An inspector draws 20 items from a production line and finds 3 defective. Two questions can be asked about this event, and they are not the same question.

The first is a probability question. If each item is independently defective with probability \(0.1\) , how likely is it that exactly 3 of 20 are defective? The answer is a single number, \(\binom{20}{3}(0.1)^3(0.9)^{17}\approx 0.190\) , and nothing about it is uncertain once the rate \(0.1\) is given.

The second is a statistical question. Having seen 3 defectives in 20, what can be said about the defect rate of the line? Here the rate is not given. It could be \(0.1\) ; it could be \(0.15\) or \(0.3\) . Each candidate rate makes the observed count possible, and some make it more probable than others. The answer cannot be a single number delivered with certainty. It will be an estimate, a range, or a decision, together with a statement of how often the method that produced it goes wrong.

This chapter builds the vocabulary for the second kind of question. Its central object is the statistical model: a family of probability laws, one of which is supposed to have generated the data. Its central warning is that some features of a model cannot be learned from data however much data there is, and that this has to be settled before any estimation begins.

1.1 A model is a family of laws

In Probability Theory a random experiment is described by one probability law. In statistics the law is unknown, and the description is a set of laws considered possible.

A model is an assumption, and it has two parts that are easy to run together. One part says what form the law takes: binomial, normal, and so on. The other says which values of the index are allowed. The statement “the count is binomial with 20 trials and success probability \(p\) ” is incomplete as a model until \(\Theta\) is named: \(p\in(0,1)\) , or \(p\in[0,1]\) , or \(p\in[0,0.5]\) are three different models, and they can lead to different estimates from the same data.

The word data refers to everything observed, taken together. Most often the data are a list \(X=(X_1,\dots,X_n)\) .

Independence and identical distribution are modelling assumptions about how the data were collected, not facts about numbers in a table. Twenty items taken from one hour of production by one machine may be close to a random sample; twenty items of which ten come from each of two machines are not identically distributed, and twenty consecutive items from a machine whose tool is wearing are not independent of their position in time.

Three objects must be kept apart, and a good habit is to write them on three separate lines when starting a problem. The parameter \(\theta\) is fixed and unknown. The statistic \(T(X)\) is random and has a distribution, which depends on \(\theta\) . The estimate \(T(x)\) is a known number. The sample mean \(\bar X\) is a statistic; the population mean \(\mu\) is a parameter; \(\bar X-\mu\) is neither, since it cannot be computed without knowing \(\mu\) , although it will be very useful in Chapter 2 exactly because its distribution can be simple.

1.2 Which model, and how large

Models differ in how much they assume. For measurements \(X_1,\dots,X_n\) of a physical constant, here are three models in decreasing order of strength.

The first says the \(X_i\) are a random sample from \(N(\mu,\sigma^2)\) with \(\theta=(\mu,\sigma^2)\in\mathbb R\times(0,\infty)\) . Two numbers describe the whole law.

The second says the \(X_i\) are a random sample from a distribution with density \(f_0(x-\mu)\) , where \(f_0\) is some unknown density symmetric about zero. The parameter is the pair \((\mu,f_0)\) , and \(f_0\) ranges over an infinite-dimensional set. The estimand of interest is still the single number \(\mu\) .

The third says only that the \(X_i\) are a random sample from some distribution \(F\) with finite variance. The parameter is \(F\) itself, and typical estimands are functionals of it: the mean \(\int x\,dF(x)\) , the median, the value \(F(c)\) at a fixed point.

A stronger model buys sharper conclusions, and the price is that the conclusions are wrong if the assumption is. Two families of parametric models occur so often that they have names.

The normal distributions form the location-scale family generated by the standard normal density. The exponential distributions form a scale family with \(\sigma=1/\lambda\) . The uniform distributions on \((0,\theta)\) form a scale family generated by the uniform density on \((0,1)\) . The representation \(X=\mu+\sigma Z\) is the reason these families are tractable: questions about \(X\) become questions about a variable \(Z\) whose law contains no unknowns.

At the far end from parametric models, a random sample from a completely unknown \(F\) can still be analysed, because \(F\) can be estimated directly.

The proposition is the first instance of a pattern that organises the next eight chapters: a statistic is proposed, its distribution under the model is derived, and that distribution tells how far the statistic can be from its target.

1.3 Identifiability

A model can be written with more parameters than the data can distinguish. When that happens no amount of data repairs it, and the defect has to be found by examining the parametrization, not the data.

Equality of distributions means equality of the probabilities assigned to every event, or equivalently of the distribution functions. To prove identifiability one usually exhibits a recipe that recovers \(\theta\) from \(P_\theta\) .

In the model \(N(\mu,\sigma^2)\) with \(\sigma^2>0\) , the mean of the distribution is \(\mu\) and its variance is \(\sigma^2\) , so the functional “mean and variance” recovers the parameter and the model is identifiable. In the binomial model, \(P_p(X=20)=p^{20}\) recovers \(p\) . The useful direction of Proposition 1.7 is this one: find any feature of the distribution, a moment, a probability, a quantile, from which the parameter can be solved.

To prove that a parametrization is not identifiable, exhibit two different parameter values with the same distribution. One pair is enough.

1.4 Why non-identifiability cannot be outrun

The claim that more data do not help deserves a proof, because it is the reason identifiability is checked first. It needs the notion of an estimator that eventually gets the right answer.

Consistency is a minimal requirement: with unlimited data, the estimator should be close to the truth with probability approaching one, and this should hold whichever member of the model is the true one. Chapter 9 studies when it holds. Here is when it cannot.

The proof uses the two hypotheses in different places. Equality of the sampling distributions transfers a probability statement from \(\theta_2\) to \(\theta_1\) . The gap between \(g(\theta_1)\) and \(g(\theta_2)\) makes the two target neighbourhoods disjoint. Remove either and the argument stops, as it should: if \(g(\theta_1)=g(\theta_2)\) the estimand does not care which of the two parameters is true, and the theorem places no obstacle in the way of estimating it. In Example 1.3 the theorem applies to \(g(\mu)=\mu\) with \(\theta_1=0.5\) and \(\theta_2=-0.5\) , and does not apply to \(g(\mu)=\lvert\mu\rvert\) .

The converse is false. Identifiability is necessary for consistent estimation and far from sufficient; an identifiable parameter can still be very hard to estimate, and a finite sample never reveals it exactly.

Non-identifiability often enters through redundant parametrization of a mean. Suppose two groups are measured, \(Y_{1j}\sim N(\mu+\alpha_1,1)\) and \(Y_{2j}\sim N(\mu+\alpha_2,1)\) , with parameter \((\mu,\alpha_1,\alpha_2)\in\mathbb R^3\) . Adding a constant \(c\) to \(\mu\) and subtracting it from both \(\alpha_1\) and \(\alpha_2\) leaves both group means unchanged, so the three-dimensional parameter is not identifiable. The group means \(\mu+\alpha_1\) and \(\mu+\alpha_2\) are identifiable, and so is the difference \(\alpha_1-\alpha_2\) . Linear models handle this either by imposing a constraint such as \(\alpha_1=0\) or by speaking only of identifiable functions, and Chapter 10 will require a design matrix of full column rank for exactly this reason.

1.5 The sampling scheme belongs to the model

Example 1.1 showed that the same record can arise from two designs. A third common design removes independence altogether.

Exercises

1. A survey records, for each of \(n\) randomly chosen households in a city, the number of cars the household owns. Propose a statistical model: give the sample space, a family of distributions with its parameter space, one statistic, and one estimand that is a function of the parameter and not the parameter itself.

2. In the model of Example 1.1, compute \(P_p(X=3)\) for \(p=0.05\) and \(p=0.2\) , and place the five rates \(0.05\) , \(0.1\) , \(0.15\) , \(0.2\) , \(0.3\) in order of how probable they make the observation.

   Answer

   \(P_{0.05}(X=3)\approx 0.0596\) and \(P_{0.2}(X=3)\approx 0.2054\) . From most to least probable observation: \(p=0.15\) (\(0.2428\) ), \(p=0.2\) (\(0.2054\) ), \(p=0.1\) (\(0.1901\) ), \(p=0.3\) (\(0.0716\) ), \(p=0.05\) (\(0.0596\) ).

3. Let \(X_1,\dots,X_n\) be a random sample from \(N(\mu,\sigma^2)\) with both parameters unknown, and let \(\mu_0\) be a known number. Decide which of the following are statistics and explain each answer: (a) \(\bar X-\mu\) ; (b) \(\max_i X_i\) ; (c) \(\sqrt n(\bar X-\mu_0)/\sigma\) ; (d) \(\sqrt n(\bar X-\mu_0)/S\) .

   Answer

   (a) Not a statistic: it requires \(\mu\) . (b) A statistic. (c) Not a statistic: \(\mu_0\) is known but \(\sigma\) is not. (d) A statistic: \(\mu_0\) is known and \(S\) is computed from the data.

4. Show that the model \(N(\mu,\sigma^2)\) with parameter \((\mu,\sigma^2)\in\mathbb R\times(0,\infty)\) is identifiable, and that the same family of laws parametrized by \((\mu,\sigma)\in\mathbb R\times(\mathbb R\setminus\{0\})\) is not.

   Solution

   Let \(P_{\mu,\sigma^2}\) denote the \(N(\mu,\sigma^2)\) distribution. Its mean is \(\mu\) and its variance is \(\sigma^2\) . If \(P_{\mu_1,\sigma_1^2}=P_{\mu_2,\sigma_2^2}\) , the two distributions have the same mean and the same variance, so \(\mu_1=\mu_2\) and \(\sigma_1^2=\sigma_2^2\) . The parametrization by \((\mu,\sigma^2)\) is therefore one-to-one, which is identifiability; in the language of Proposition 1.7 the functional is “mean and variance”.

   For the second parametrization the density is

   \[ \frac{1}{\lvert\sigma\rvert\sqrt{2\pi}}\exp\Bigl(-\frac{(x-\mu)^2}{2\sigma^2}\Bigr), \]

   which depends on \(\sigma\) only through \(\sigma^2\) . The parameter values \((0,1)\) and \((0,-1)\) are different points of \(\mathbb R\times(\mathbb R\setminus\{0\})\) and both give the standard normal distribution. One pair of distinct parameters with the same law is enough to show that the parametrization is not one-to-one. The family of laws is unchanged; what changed is the labelling, and identifiability is a property of the labelling.

5. A two-component mixture has density \(\pi\,\phi(x-\mu_1)+(1-\pi)\,\phi(x-\mu_2)\) , where \(\phi\) is the standard normal density and \((\pi,\mu_1,\mu_2)\in(0,1)\times\mathbb R^2\) . Exhibit two different parameter values with the same density, and state a restriction on the parameter space that removes this particular ambiguity.

   Hint

   Swap the labels of the two components: compare \((\pi,\mu_1,\mu_2)\) with \((1-\pi,\mu_2,\mu_1)\) . A restriction such as \(\mu_1<\mu_2\) forbids the swap. Notice that it also excludes \(\mu_1=\mu_2\) , where a second ambiguity appears: the density is then \(\phi(x-\mu_1)\) for every \(\pi\) .

6. In the two-group model \(Y_{1j}\sim N(\mu+\alpha_1,1)\) , \(Y_{2j}\sim N(\mu+\alpha_2,1)\) of Section 1.4, prove that \(\alpha_1-\alpha_2\) is identifiable and that \(\alpha_1\) is not, and show that the constraint \(\alpha_1+\alpha_2=0\) makes the whole parameter identifiable.

   Solution

   The data consist of the two groups, and their distribution is determined by the two group means \(m_1=\mu+\alpha_1\) and \(m_2=\mu+\alpha_2\) , since the variances are known. Each \(m_k\) is the expectation of an observation in group \(k\) , so by Proposition 1.7 both \(m_1\) and \(m_2\) are identifiable. Then \(\alpha_1-\alpha_2=m_1-m_2\) is a function of identifiable quantities and is identifiable.

   For \(\alpha_1\) , take \(\theta=(0,1,0)\) and \(\theta'=(1,0,-1)\) . Both give \(m_1=1\) and \(m_2=0\) , hence the same distribution of the data, while \(\alpha_1\) equals \(1\) in the first and \(0\) in the second. So \(\alpha_1\) is not identifiable.

   Under the constraint \(\alpha_1+\alpha_2=0\) the map from \((\mu,\alpha_1)\) to \((m_1,m_2)\) is \(m_1=\mu+\alpha_1\) , \(m_2=\mu-\alpha_1\) , which is inverted by \(\mu=(m_1+m_2)/2\) and \(\alpha_1=(m_1-m_2)/2\) , with \(\alpha_2=-\alpha_1\) . The parameter is recovered from identifiable quantities, so the constrained parametrization is one-to-one.

7. Let \(\theta\mapsto P_\theta\) be identifiable and let \(h:\Theta\to H\) be a bijection. Prove that the reparametrized model \(\eta\mapsto P_{h^{-1}(\eta)}\) is identifiable, and give an example showing that if \(h\) is not one-to-one the conclusion “the parameter can be recovered from \(h(\theta)\) ” fails even though \(h(\theta)\) itself remains identifiable.

   Hint

   For the first part, if \(P_{h^{-1}(\eta_1)}=P_{h^{-1}(\eta_2)}\) then identifiability of the original parametrization gives \(h^{-1}(\eta_1)=h^{-1}(\eta_2)\) ; apply \(h\) . For the second part, take \(N(\mu,1)\) and \(h(\mu)=\mu^2\) : the function \(\mu^2\) is identifiable, being a function of the mean, but knowing \(\mu^2\) does not return \(\mu\) .

8. In Example 1.4 suppose the full data are \((Y,Z)\) with \(Y\sim\operatorname{Poisson}(\lambda\pi)\) and, independently, \(Z\sim\operatorname{Binomial}(m,\pi)\) for a known \(m\ge 1\) . Prove that \((\lambda,\pi)\) is identifiable on \((0,\infty)\times(0,1]\) , and explain why the proof breaks down if \(\pi=0\) were allowed.

   Solution

   Suppose \((\lambda_1,\pi_1)\) and \((\lambda_2,\pi_2)\) give the same joint distribution of \((Y,Z)\) . Then they give the same marginal distribution of \(Z\) , hence the same mean \(E(Z)=m\pi\) ; since \(m\ge 1\) is known, \(\pi_1=\pi_2\) . They also give the same marginal distribution of \(Y\) , hence the same mean \(E(Y)=\lambda\pi\) , so \(\lambda_1\pi_1=\lambda_2\pi_2\) . Because the common value \(\pi_1=\pi_2\) is strictly positive on the parameter space \((0,\infty)\times(0,1]\) , it can be divided out, giving \(\lambda_1=\lambda_2\) . The parametrization is one-to-one.

   The division is the step that fails if \(\pi=0\) is allowed. When \(\pi=0\) , both \(Y\) and \(Z\) are zero with probability one whatever \(\lambda\) is, so \((\lambda,0)\) and \((\lambda',0)\) have the same law for every pair \(\lambda\ne\lambda'\) . A detector that registers nothing carries no information about the arrival rate, and no calibration run changes that.

9. A student argues about Example 1.3: “With a million observations of \(\lvert Y\rvert\) the histogram is essentially exact, so \(\mu\) is known to many decimal places, including its sign.” Identify precisely which part of the claim is correct and which part Theorem 1.9 rules out.

   Solution

   The correct part: with a very large sample the distribution of \(X=\lvert Y\rvert\) is known very precisely, and therefore every identifiable function of the parameter is known very precisely. In particular \(\lvert\mu\rvert\) is, because \(E_\mu X^2=1+\mu^2\) and the sample second moment converges to it.

   The incorrect part is “including its sign”. The distribution of the entire sample is the same under \(\mu\) and under \(-\mu\) , for every sample size, because each observation has the same law under both and the observations are independent. Theorem 1.9 applies with \(\theta_1=\mu\) , \(\theta_2=-\mu\) , and \(g(\mu)=\mu\) whenever \(\mu\ne 0\) : no sequence of estimators can be consistent for \(\mu\) . Concretely, any rule that reports a positive sign with some probability under \(\mu\) reports a positive sign with exactly that probability under \(-\mu\) , so it is wrong under one of them at least half the time in the worst case. The million observations sharpen the histogram; they add nothing about a feature on which the histogram does not depend.

10. For the lot of Example 1.5 with \(D=5\) , compute \(P_D(X=0)\) under the hypergeometric model and under the binomial approximation with success probability \(0.1\) , and explain from the dependence between draws why the hypergeometric value is the smaller one.

    Answer

    Hypergeometric: \(\binom{45}{10}/\binom{50}{10}\approx 0.3106\) . Binomial: \(0.9^{10}\approx 0.3487\) . Without replacement, each non-defective item drawn leaves a higher proportion of defectives among those remaining, so a run of ten non-defectives is less likely than under independent draws.

11. An inspector samples until the \(r\) -th defective item appears, for a fixed \(r\ge 2\) , and records the total number \(N\) of items drawn; each item is independently defective with probability \(p\in(0,1)\) . (a) Write the probability mass function of \(N\) . (b) Show that \((r-1)/(N-1)\) has expectation \(p\) . (c) Deduce that \(r/N\) does not have expectation \(p\) , and say in which direction it errs on average.

    Solution

    (a) The event \(N=n\) means that the first \(n-1\) items contain exactly \(r-1\) defectives and item \(n\) is defective. Hence, for \(n=r,r+1,\dots\) ,

    \[ P_p(N=n)=\binom{n-1}{r-1}p^{r}(1-p)^{n-r}. \]

    (b) Use the identity \(\frac{r-1}{n-1}\binom{n-1}{r-1}=\binom{n-2}{r-2}\) , valid for \(n\ge r\ge 2\) . Then

    \[ E_p\Bigl[\frac{r-1}{N-1}\Bigr]=\sum_{n=r}^{\infty}\binom{n-2}{r-2}p^{r}(1-p)^{n-r} =p\sum_{k=r-1}^{\infty}\binom{k-1}{r-2}p^{r-1}(1-p)^{k-(r-1)}, \]

    where \(k=n-1\) . The last sum adds the probability mass function of the waiting time for the \((r-1)\) -th defective over all its possible values, so it equals \(1\) . Therefore \(E_p[(r-1)/(N-1)]=p\) .

    (c) For every \(n\ge r\) ,

    \[ \frac{r}{n}-\frac{r-1}{n-1}=\frac{n-r}{n(n-1)}\ge 0, \]

    with strict inequality whenever \(n>r\) , an event of positive probability. Taking expectations, \(E_p[r/N]>E_p[(r-1)/(N-1)]=p\) . The natural-looking proportion \(r/N\) overestimates \(p\) on average under this design. For \(r=3\) and \(p=0.15\) a direct summation gives \(E_p[3/N]\approx 0.2026\) . Under the binomial design of Example 1.1 the same proportion, defectives over items, has expectation exactly \(p\) . The record “3 in 20” is the same; the sampling distribution of the proportion is not.

12. Let \(\widehat F_n\) be the empirical distribution function of a random sample of size \(n=25\) from a continuous distribution with median \(m\) . Find the exact distribution of \(25\,\widehat F_{25}(m)\) , the standard deviation of \(\widehat F_{25}(m)\) , and the smallest \(n\) for which the standard deviation of \(\widehat F_n(x)\) is at most \(0.02\) for every \(x\) and every \(F\) .

    Answer

    \(25\,\widehat F_{25}(m)\sim\operatorname{Binomial}(25,1/2)\) , because \(F(m)=1/2\) for a continuous distribution. The standard deviation of \(\widehat F_{25}(m)\) is \(\sqrt{0.25/25}=0.1\) . The worst case over \(x\) and \(F\) is \(1/(2\sqrt n)\) , and \(1/(2\sqrt n)\le 0.02\) requires \(n\ge 625\) .

13. Lifetimes \(T_1,\dots,T_n\) are a random sample from \(\operatorname{Exp}(\lambda)\) , but the only thing recorded for each unit is whether it survived past a fixed time \(c>0\) . Write the model for the recorded data, determine whether \(\lambda\) is identifiable, and express \(\lambda\) in terms of a probability that the recorded data can estimate.

    Answer

    The recorded data are \(B_i=\mathbf 1\{T_i>c\}\) , independent Bernoulli variables with success probability \(q=e^{-\lambda c}\) . The map \(\lambda\mapsto e^{-\lambda c}\) is strictly decreasing on \((0,\infty)\) , so \(\lambda\) is identifiable, with \(\lambda=-\log(q)/c\) where \(q=P_\lambda(B_i=1)\) .

14. A threshold device observes \(Y\sim N(\mu,\sigma^2)\) and reports only \(B=\mathbf 1\{Y>0\}\) . (a) With \(\sigma=1\) known, show \(\mu\) is identifiable from the distribution of \(B\) . (b) With both parameters unknown, show that \((\mu,\sigma)\) is not identifiable, and identify the function of the parameter that is.

    Answer

    (a) \(P_\mu(B=1)=P(Y>0)=\Phi(\mu)\) , and \(\Phi\) is strictly increasing, so \(\mu=\Phi^{-1}(P_\mu(B=1))\) . (b) \(P(B=1)=\Phi(\mu/\sigma)\) , so \((1,1)\) and \((2,2)\) give the same law. The identifiable function is the ratio \(\mu/\sigma\) .

15. In the model \(Y_i=\beta_1x_{i1}+\beta_2x_{i2}+\varepsilon_i\) with independent \(\varepsilon_i\sim N(0,1)\) and known constants satisfying \(x_{i2}=2x_{i1}\) for every \(i\) , with not all \(x_{i1}\) equal to zero, show that \((\beta_1,\beta_2)\) is not identifiable and that \(\beta_1+2\beta_2\) is.

    Solution

    Write \(x_{i2}=2x_{i1}\) in the mean: \(E\,Y_i=\beta_1x_{i1}+2\beta_2x_{i1}=(\beta_1+2\beta_2)x_{i1}\) . The observations are independent normal variables with variance \(1\) and these means, so the distribution of the data depends on \((\beta_1,\beta_2)\) only through \(\gamma=\beta_1+2\beta_2\) .

    Not identifiable: the parameter values \((1,0)\) and \((-1,1)\) both have \(\gamma=1\) , so they give the same distribution of \((Y_1,\dots,Y_n)\) , and they are different points.

    Identifiable function: choose an index \(i\) with \(x_{i1}\ne 0\) , which exists by hypothesis. Then \(\gamma=E(Y_i)/x_{i1}\) is recovered from the distribution of the data, so by Proposition 1.7 the combination \(\beta_1+2\beta_2\) is identifiable. In matrix language the design matrix with columns \((x_{i1})\) and \((x_{i2})\) has rank \(1\) , not \(2\) ; the data determine the mean vector, and the mean vector determines only one linear combination of the two coefficients.

16. Let \(f_0\) be a known density with finite mean \(m_0\) . Prove that the location family \(\{f_0(x-\mu):\mu\in\mathbb R\}\) is identifiable. Then give a different argument that does not need a finite mean, using the fact that a distribution function tends to \(0\) at \(-\infty\) and to \(1\) at \(+\infty\) .

    Solution

    With a finite mean: if \(X\) has density \(f_0(x-\mu)\) then \(X-\mu\) has density \(f_0\) , so \(E\,X=\mu+m_0\) . If two location parameters \(\mu_1\) and \(\mu_2\) give the same distribution, they give the same mean, so \(\mu_1+m_0=\mu_2+m_0\) and \(\mu_1=\mu_2\) .

    Without a finite mean: let \(F_0\) be the distribution function of \(f_0\) . The distribution function of the member with location \(\mu\) is \(F_0(x-\mu)\) . Suppose \(F_0(x-\mu_1)=F_0(x-\mu_2)\) for all \(x\) , and put \(d=\mu_1-\mu_2\) . Substituting \(x=y+\mu_1\) gives \(F_0(y)=F_0(y+d)\) for every \(y\) , and by repeating the step, \(F_0(y)=F_0(y+kd)\) for every integer \(k\) . If \(d\ne 0\) , letting \(k\to\infty\) in one direction sends \(y+kd\) to \(+\infty\) and in the other to \(-\infty\) , so \(F_0(y)=1\) and \(F_0(y)=0\) at once, a contradiction. Hence \(d=0\) . This argument covers the Cauchy location family, whose mean does not exist.

17. Prove that if \(g_1\) and \(g_2\) are identifiable functions on \(\Theta\) and \(\psi\) is any function of two variables, then \(\psi(g_1(\theta),g_2(\theta))\) is identifiable. Use this and Example 1.4 to decide, with reasons, whether \(\lambda\pi^2\) is identifiable from \(Y\) alone.

    Hint

    If \(P_{\theta_1}=P_{\theta_2}\) then \(g_1\) and \(g_2\) agree at the two parameters, so any function of the pair agrees too. For the second part the first statement cannot help, because \(\lambda\pi^2\) is not a function of \(\lambda\pi\) . Test the two parameter values used in Example 1.4.

18. To estimate the size \(N\) of a fish population, \(20\) fish are caught, marked, and released; later \(10\) fish are caught and \(4\) of them are found to be marked. Model the number of marked fish in the second catch, show that \(N\) is identifiable on \(\{20,21,22,\dots\}\) , and compute the probability of the observed count for \(N=40\) , \(N=50\) , and \(N=60\) .

    Solution

    The second catch is a sample of \(10\) drawn without replacement from a population of \(N\) fish of which \(20\) are marked, so the number \(X\) of marked fish in it is hypergeometric:

    \[ P_N(X=x)=\frac{\binom{20}{x}\binom{N-20}{10-x}}{\binom{N}{10}} . \]

    For identifiability, look at the single probability \(P_N(X=10)=\binom{20}{10}/\binom{N}{10}\) . The denominator \(\binom{N}{10}\) is strictly increasing in \(N\) for \(N\ge 10\) , so this probability is strictly decreasing in \(N\) on \(\{20,21,\dots\}\) and takes a different value at each \(N\) . By Proposition 1.7, \(N\) is identifiable.

    For the observed count \(x=4\) :

    \[ P_{40}(X=4)\approx 0.2215,\qquad P_{50}(X=4)\approx 0.2801,\qquad P_{60}(X=4)\approx 0.2467 . \]

    Among the three, \(N=50\) makes the observation most probable. That agrees with the proportion argument: the marked fraction in the second catch is \(4/10\) , and setting \(20/N=4/10\) gives \(N=50\) .