Let \(D\) be the event that the two-headed coin was drawn and \(H\) the event of three heads in three tosses. The prior is \(P(D)=\tfrac14\) , and the likelihoods are \(P(H\mid D)=1\) and \(P(H\mid D^c)=\tfrac18\) , the tosses being independent given the coin. By Bayes’ rule,

For the fourth toss, condition on the coin, using the posterior probabilities and the conditional independence of the tosses given the coin:

In odds form: prior odds \(1:3\) , likelihood ratio \(8\) , posterior odds \(8:3\) . The tosses are not independent unconditionally; three heads raise the probability of a fourth from the prior value \(\tfrac14+\tfrac34\cdot\tfrac12=\tfrac58\) to \(\tfrac{19}{22}\) .

Marginal. For \(x>0\) , \(f_X(x)=\int_x^{\infty}2e^{-x-y}\,dy=2e^{-2x}\) : \(X\sim\mathrm{Exp}(2)\) , with mean \(\tfrac12\) and variance \(\tfrac14\) .

Regression function. The conditional density of \(Y\) given \(X=x\) is \(2e^{-x-y}/(2e^{-2x})=e^{-(y-x)}\) for \(y>x\) : given \(X=x\) , \(Y\) is \(x\) plus an \(\mathrm{Exp}(1)\) variable. Hence \(E[Y\mid X=x]=x+1\) .

Mean of the second variable. By the tower property, \(E[Y]=E[X]+1=\tfrac32\) .

Covariance. By Exercise 6.10, \(\operatorname{Cov}(X,Y)=\operatorname{Cov}(X,E[Y\mid X])=\operatorname{Cov}(X,X+1)=\operatorname{Var}(X)=\tfrac14\) .

The joint law is that of the smaller and the larger of two independent \(\mathrm{Exp}(1)\) variables: the density \(e^{-x-y}\) on the quadrant, folded onto the region \(x<y\) . The structure found here is lack of memory: after the first of two exponential clocks rings, at a time that is \(\mathrm{Exp}(2)\) by Exercise 5.13, the other still has an \(\mathrm{Exp}(1)\) time to run, independent of the past. As a check, the mean of the larger is \(\tfrac12+1=\tfrac32\) , and the two means sum to \(2\) , the mean of the sum of the two original variables.

The number of heads \(X\) has mean \(200\) and variance \(100\) ; the event is \(\{X\ge 220\}\) , a deviation of \(20\) , or two standard deviations.

Chebyshev, two-sided (a bound). \(P(X\ge 220)\le P(\lvert X-200\rvert\ge 20)\le\tfrac{100}{400}=0.25\) .

One-sided Chebyshev (a bound). \(\dfrac{\sigma^2}{\sigma^2+a^2}=\dfrac{100}{100+400}=0.2\) .

Exponential bound (a bound). With \(\varepsilon=20/400=0.05\) , Proposition 7.5 gives \(e^{-2\cdot 400\cdot 0.0025}=e^{-2}\approx 0.135\) .

Normal approximation (an approximation). \(1-\Phi\bigl(\tfrac{219.5-200}{10}\bigr)=1-\Phi(1.95)\approx 0.0256\) .

Exact. \(0.0255\) .

The three bounds are valid without qualification and are loose by factors of ten, eight, and five. The approximation is accurate to the fourth decimal place and comes with no guarantee. At two standard deviations the exponential bound has barely begun to outperform the moment bounds; its advantage is at larger deviations.

Two variables. The pair \((X_1,X_2)\) has the same joint distribution as \((X_2,X_1)\) , since the variables are independent with a common distribution. So \(P(X_1<X_2)=P(X_2<X_1)\) . Because the distribution is continuous, \(P(X_1=X_2)=0\) , and the two probabilities sum to one. Hence each is \(\tfrac12\) .

All orderings. The same symmetry holds for every permutation of the indices: the joint distribution of \((X_1,\dots,X_n)\) is unchanged by permuting the variables. Ties have probability zero, so exactly one of the \(n!\) strict orderings occurs, and by symmetry they are equally likely, each with probability \(1/n!\) .

Records. Let \(R_k\) be the event that \(X_k\) exceeds \(X_1,\dots,X_{k-1}\) . Among the first \(k\) variables each is equally likely to be the largest, by the previous part applied to \(X_1,\dots,X_k\) , so \(P(R_k)=1/k\) . The number of records is \(\sum_k\mathbf 1_{R_k}\) , and by linearity its expectation is

Nothing about the common distribution was used beyond continuity. The expected number of records grows like \(\ln n\) : among a thousand independent observations, only about \(7.5\) record highs are expected.

For any \(X\) and \(Y\) with finite variances, bilinearity gives

So what equal variances give, in general, is that the sum and difference are uncorrelated. Independence of \(X\) and \(Y\) is not even needed for that.

The student’s error is in the account of the proof. Example 5.10 did not deduce independence from equal variances. It computed the joint density of the sum and difference and saw it factorize, and the factorization used the specific form of the normal density, in which \(x^2+y^2\) appears in the exponent and equals \((u^2+v^2)/2\) .

Counterexample: let \(X\) and \(Y\) be independent fair coin indicators. They have equal variances, so \(X+Y\) and \(X-Y\) are uncorrelated. But \(P(X+Y=2,\ X-Y=0)=P(X=1,Y=1)=\tfrac14\) , while \(P(X+Y=2)\,P(X-Y=0)=\tfrac14\cdot\tfrac12=\tfrac18\) . The sum and difference are dependent; indeed a sum of \(0\) or \(2\) forces the difference to be \(0\) .

Use three states, according to how much of the pattern HTH is currently in hand: nothing, H, or HT. Let \(a\) , \(b\) , \(c\) be the expected numbers of further tosses from these states. From “nothing”: a head moves to H, a tail stays. From H: a tail moves to HT, a head stays at H. From HT: a head completes the pattern, a tail returns to “nothing”. So

The first gives \(a=2+b\) and the second \(b=2+c\) , so \(a=4+c\) ; the third then gives \(c=1+\tfrac12(4+c)\) , so \(c=6\) , \(b=8\) , and

The pattern HTH takes \(10\) tosses on average, longer than the \(8\) that a pattern of three tosses with no self-overlap would take: HTH ends with the letter it begins with, so its occurrences can overlap, and overlapping patterns have longer waits, as with HH in Example 6.1.

Write \(Z_n=S_n/\sqrt n\) , the standardized sum.

\(P(S_n>0)=P(Z_n>0)\to 1-\Phi(0)=\tfrac12\) , by the central limit theorem.

\(P(S_n>\sqrt n)=P(Z_n>1)\to 1-\Phi(1)\approx 0.159\) , by the central limit theorem.

\(P(S_n>n^{0.6})=P(Z_n>n^{0.1})\) . By Chebyshev’s inequality this is at most \(\operatorname{Var}(S_n)/n^{1.2}=n^{-0.2}\to 0\) . The threshold is a growing number of standard deviations.

\(P(S_n>-n)=P(\bar X_n>-1)\to 1\) , by the weak law of large numbers, since \(\bar X_n\) converges in probability to \(0\) .

Only the second question needs the shape of the limit distribution. The first needs only its symmetry about \(0\) ; the third and fourth are settled by the scale alone, and for them Chebyshev’s inequality suffices.

The hypothesis that the variables are identically distributed is used in the first two answers and not in the last two. Chebyshev’s inequality and the weak law need only a common mean and variance and zero correlations. The central limit theorem needs more: let \(X_k\) be \(k\) or \(-k\) with probability \(\tfrac{1}{2k^2}\) each and \(0\) otherwise, independently. Each \(X_k\) has mean \(0\) and variance \(1\) , but \(P(X_k\ne 0)=k^{-2}\) has a finite sum, so by Lemma 7.9 only finitely many of the \(X_k\) are nonzero, \(S_n\) stays bounded, and \(P(S_n>\sqrt n)\to 0\) , not \(0.159\) .

Label the four aces and the four kings. For an ace \(a\) let \(I_a\) indicate that it is in the hand, and for a king \(k\) let \(J_k\) do the same. Then \(X=\sum_aI_a\) and \(Y=\sum_kJ_k\) . Each card is in the hand with probability \(\tfrac{13}{52}=\tfrac14\) , and two given cards are both in the hand with probability \(\tfrac{13}{52}\cdot\tfrac{12}{51}=\tfrac1{17}\) . So for every ace and every king,

By bilinearity, summing over the \(16\) ace–king pairs,

The number of aces is hypergeometric with \(N=52\) , \(K=4\) , \(n=13\) , so by Example 5.7

and \(\operatorname{Var}(Y)\) is the same. Therefore

Aces and kings compete for the thirteen places in the hand, so the correlation is negative, and it is small because there are \(44\) other cards competing as well.