-
Let \(U\) be uniform on \((0,1)\) and \(X_n=\mathbf 1_{\{U<1/n\}}\,n^{3/4}\) . Decide whether \(X_n\) converges almost surely, in probability, in \(L^1\) , and in \(L^2\) , and whether the family is uniformly integrable.
Answer
At each sample point \(X_n=0\) once \(n>1/U\) , so \(X_n\to 0\) almost surely and in probability. \(\mathbb E X_n=n^{-1/4}\to 0\) , so also in \(L^1\) , and hence the family is uniformly integrable (Theorem 5.6). \(\mathbb E X_n^2=n^{1/2}\to\infty\) , so not in \(L^2\) .
-
Let \(X_1,X_2,\dots\) be independent with \(\mathbb P(X_n=1)=1/n\) and \(\mathbb P(X_n=0)=1-1/n\) . Find \(\mathbb P(X_n=1\ \text{i.o.})\) , decide in which modes \(X_n\to 0\) , and find the limit in distribution of \((\sum_{k\le n}X_k-\log n)/\sqrt{\log n}\) .
Solution
The events \(\{X_n=1\}\) are independent with probabilities \(1/n\) , and \(\sum_n1/n=\infty\) , so by the second Borel–Cantelli lemma \(\mathbb P(X_n=1\ \text{i.o.})=1\) . Consequently \(X_n\) does not converge to \(0\) almost surely; in fact it converges with probability \(0\) , since it also equals \(0\) infinitely often. In probability, \(\mathbb P(\lvert X_n\rvert>\varepsilon)=1/n\to 0\) for \(0<\varepsilon<1\) , so \(X_n\to 0\) in probability, and \(\mathbb E\lvert X_n\rvert^p=1/n\to 0\) , so in every \(L^p\) , and in distribution.
The sum \(R_n=\sum_{k\le n}X_k\) has the law of the number of records of Example 10.3: independent indicators with success probabilities \(1/k\) . Its mean is \(H_n\) and its variance \(s_n^2=H_n-\sum_{k\le n}k^{-2}\to\infty\) ; the centred, scaled terms are bounded by \(1/s_n\) , so Lindeberg’s condition holds and \((R_n-H_n)/s_n\to N(0,1)\) in distribution by Theorem 10.5. Since \(H_n-\log n\) is bounded and \(s_n^2/\log n\to 1\) , Slutsky’s theorem gives \((R_n-\log n)/\sqrt{\log n}\to N(0,1)\) in distribution.
-
Let \(X\) be integrable and \(\mathcal G\) a sub-sigma-algebra. Prove that \(\mathbb E\lvert\mathbb E[X\mid\mathcal G]\rvert\le\mathbb E\lvert X\rvert\) , and characterize the cases of equality when \(X\ge 0\) fails.
Hint
\(\lvert\mathbb E[X\mid\mathcal G]\rvert\le\mathbb E[\lvert X\rvert\mid\mathcal G]\) by Theorem 6.4(b); take expectations. Equality holds if and only if there is \(G\in\mathcal G\) with \(X\ge 0\) almost surely on \(G\) and \(X\le 0\) almost surely on \(G^{c}\) : take \(G=\{\mathbb E[X\mid\mathcal G]\ge 0\}\) and integrate \(\lvert X\rvert-X\) over it.
-
A fair coin is tossed repeatedly and a gambler’s fortune changes by \(\pm1\) each toss, starting from \(0\) . The gambler stops on first reaching \(+3\) or \(-5\) . Find the probability of stopping at \(+3\) and the expected number of tosses, stating which theorem justifies each step.
Answer
With \(a=5\) and \(b=3\) in Example 7.3: the probability of stopping at \(+3\) is \(5/8\) , by optional stopping (Theorem 7.7(b)) for the bounded stopped walk, and the expected number of tosses is \(15\) , from the martingale \(S_n^2-n\) with bounded and monotone convergence.
-
Let \(X_1,X_2,\dots\) be independent and identically distributed with \(\mathbb P(\lvert X_1\rvert>x)=x^{-1/2}\) for \(x\ge 1\) . Decide whether \(S_n/n\) converges almost surely, and find the almost sure value of \(\limsup_n\lvert X_n\rvert/n^2\) .
Answer
\(\mathbb E\lvert X_1\rvert=\int_0^\infty\mathbb P(\lvert X_1\rvert>x)\,dx=\infty\) , so by Theorem 8.6 the averages are almost surely unbounded and do not converge. For every \(c>0\) , \(\mathbb P(\lvert X_n\rvert>cn^2)=c^{-1/2}/n\) for large \(n\) , which is not summable; by independence and the second Borel–Cantelli lemma \(\limsup_n\lvert X_n\rvert/n^2=\infty\) almost surely.
-
Let \(Y_n\) be uniformly distributed on \(\{1,2,\dots,n\}\) . Find the limit in distribution of \(Y_n/n\) and of \(n\,\mathbf 1_{\{Y_n=1\}}\) , and decide for each whether the expectations converge to the expectation of the limit.
Answer
\(Y_n/n\) has distribution function \(\lfloor nx\rfloor/n\to x\) , so it converges in distribution to the uniform law on \((0,1)\) , and \(\mathbb E[Y_n/n]=(n+1)/(2n)\to\tfrac12\) , as boundedness guarantees. \(n\mathbf 1_{\{Y_n=1\}}\) is nonzero with probability \(1/n\) , so it converges in distribution to \(0\) , while its expectation is \(1\) for all \(n\) : the family is not uniformly integrable.
-
Let \(M_n\) be a martingale with \(M_0=0\) and \(\lvert M_n-M_{n-1}\rvert\le 1/n\) for all \(n\) . Prove that \(M_n\) converges almost surely and in \(L^2\) .
Solution
Let \(D_k=M_k-M_{k-1}\) , so \(\lvert D_k\rvert\le 1/k\) and each \(M_n=\sum_{k\le n}D_k\) is bounded, hence square-integrable. By Theorem 7.15 the increments of a square-integrable martingale are orthogonal and
\[ \mathbb E M_n^2=\sum_{k=1}^{n}\mathbb E D_k^2\le\sum_{k=1}^{n}\frac1{k^2}\le\frac{\pi^2}{6}. \]So the martingale is bounded in \(L^2\) , and the same theorem gives convergence almost surely and in \(L^2\) . The bound on the increments is deterministic, but the increments need not be independent; the random harmonic series of Example 7.6 is the special case \(D_k=\varepsilon_k/k\) . Note that the cruder bound \(\lvert M_n\rvert\le\sum_{k\le n}1/k\) diverges and would prove nothing: orthogonality, which makes squares of increments add, is what turns the divergent series \(\sum 1/k\) into the convergent \(\sum 1/k^2\) .
-
Suppose \(X_n\to X\) in distribution and \(\sup_n\mathbb E X_n^2<\infty\) . Prove that \(\mathbb E X_n\to\mathbb E X\) , and show by example that \(\mathbb E X_n^2\to\mathbb E X^2\) can fail.
Answer
A bounded second moment gives uniform integrability (Proposition 5.3), and Corollary 9.4(c) gives \(\mathbb E X_n\to\mathbb E X\) . For the example let \(X_n=\sqrt n\) with probability \(1/n\) and \(0\) otherwise: \(X_n\to 0\) in distribution, \(\mathbb E X_n^2=1\) , and \(\mathbb E X^2=0\) .
-
Let \(U\) and \(V\) be independent and uniform on \((0,1)\) . Find \(\mathbb E[U\mid U+V]\) and \(\mathbb E[UV\mid U]\) , and verify the first against the defining property on the events \(\{U+V\le t\}\) for \(t\le 1\) .
Solution
First conditional expectation. By Exercise 6.7 with \(n=2\) , symmetry of the joint law gives \(\mathbb E[U\mid U+V]=\tfrac12(U+V)\) . To verify the defining property on \(\{U+V\le t\}\) with \(0\le t\le 1\) : by Theorem 2.6,
\[ \mathbb E[U;U+V\le t]=\int_0^tu\,(t-u)\,du=\frac{t^3}{2}-\frac{t^3}{3}=\frac{t^3}{6}. \]The sum \(U+V\) has density \(s\) on \((0,1)\) (Example 2.4), so
\[ \mathbb E\bigl[\tfrac12(U+V);U+V\le t\bigr]=\int_0^t\frac s2\cdot s\,ds=\frac{t^3}{6}. \]The two agree, as required.
Second conditional expectation. \(U\) is \(\sigma(U)\) -measurable and \(V\) is independent of \(\sigma(U)\) , so taking out what is known and Theorem 6.4(g) give \(\mathbb E[UV\mid U]=U\,\mathbb E[V\mid U]=U\,\mathbb E V=\tfrac12U\) . Equivalently, Proposition 6.7 with \(h(x,y)=xy\) gives \(H(x)=x/2\) .
-
Let \(\xi_1,\xi_2,\dots\) be independent with \(\mathbb P(\xi_k=1)=\mathbb P(\xi_k=-1)=\tfrac12\) . For each of the series \(\sum_k\xi_k/k^{\alpha}\) , \(\alpha>0\) , decide whether it converges almost surely, naming the theorem used in each range of \(\alpha\) .
Answer
For \(\alpha>\tfrac12\) the partial sums are a martingale with \(\sum_kk^{-2\alpha}<\infty\) , bounded in \(L^2\) , and the series converges almost surely by Theorem 7.15. For \(\alpha\le\tfrac12\) the variances \(\sum_{k\le n}k^{-2\alpha}\) diverge while the terms are bounded, so Theorem 10.5 and the zero-one law give almost sure divergence, exactly as in Exercise 10.17.
-
A student claims: “The events \(A_n\) satisfy \(\mathbb P(A_n)\ge\tfrac13\) for all \(n\) , so by the second Borel–Cantelli lemma infinitely many occur almost surely.” Say what is true, what is false, and what can be concluded about \(\mathbb P(A_n\ \text{i.o.})\) .
Solution
It is true that \(\sum_n\mathbb P(A_n)=\infty\) , since the terms are at least \(\tfrac13\) . It is false that the second Borel–Cantelli lemma applies: that lemma requires the events to be independent, or at least pairwise independent (Theorem 3.5), and nothing of the kind is assumed. The conclusion is also false in general: if \(A_n=A\) for every \(n\) , with \(\mathbb P(A)=\tfrac13\) , then \(\{A_n\ \text{i.o.}\}=A\) has probability \(\tfrac13\) .
What can be concluded is the bound of Lemma 3.2, which needs no independence:
\[ \mathbb P(A_n\ \text{i.o.})\ge\limsup_n\mathbb P(A_n)\ge\tfrac13 . \]The repeated-event example shows that this bound cannot be improved under the stated hypothesis. So infinitely many of the events occur with probability at least \(\tfrac13\) , and with probability one if they are in addition pairwise independent.
-
Let \(X_n\) be normal with mean \(0\) and variance \(1+(-1)^n/2\) . Decide whether the sequence of laws is tight, find its weakly convergent subsequences and their limits, and decide whether the whole sequence converges weakly.
Answer
The variances are bounded by \(\tfrac32\) , so the sequence is tight (Exercise 9.8). Along even \(n\) the laws equal \(N(0,\tfrac32)\) and along odd \(n\) they equal \(N(0,\tfrac12)\) . A subsequence converges weakly exactly when its indices are eventually all even or eventually all odd, with those two limits. The whole sequence does not converge, having two different subsequential limits; equivalently its characteristic functions \(e^{-(1+(-1)^n/2)t^2/2}\) do not converge.
-
Let \(X_1,X_2,\dots\) be independent and identically distributed with mean \(\mu\ne 0\) and finite variance \(\sigma^2>0\) . Find the limit in distribution of \(\sqrt n\,(\bar X_n^{-1}-\mu^{-1})\) and of \(\sqrt n\,(\bar X_n-\mu)/\bar X_n\) .
Answer
By the delta method with \(g(x)=1/x\) , \(g'(\mu)=-\mu^{-2}\) : \(\sqrt n(\bar X_n^{-1}-\mu^{-1})\to N(0,\sigma^2/\mu^4)\) . By Slutsky’s theorem, since \(\bar X_n\to\mu\) in probability: \(\sqrt n(\bar X_n-\mu)/\bar X_n\to N(0,\sigma^2/\mu^2)\) .
-
Let \((X_n)\) be a uniformly integrable martingale and \(\tau\) a stopping time that may take the value \(\infty\) . Explain how to define \(X_\tau\) , and prove that \(\mathbb E X_\tau=\mathbb E X_0\) , naming the result of Chapter 7 that replaces the hypotheses of the optional stopping theorem.
Hint
By Theorem 7.13 the martingale has an almost sure and \(L^1\) limit \(X_\infty\) with \(X_n=\mathbb E[X_\infty\mid\mathcal F_n]\) ; define \(X_\tau=X_\infty\) on \(\{\tau=\infty\}\) and follow the hint to Exercise 7.15.
-
Let \(X_n\) have characteristic function \(\varphi_n(t)=\bigl(1-\tfrac{t^2}{2n}\bigr)^n\) for \(\lvert t\rvert\le\sqrt{2n}\) . Without knowing whether such variables exist, say what the continuity theorem would give, and exhibit a genuine sequence with \(\varphi_n(t)=\cos(t/\sqrt n)^n\) that has the same limit.
Hint
By Lemma 9.16 the functions converge to \(e^{-t^2/2}\) , so if they are characteristic functions, Theorem 9.17 gives convergence in distribution to \(N(0,1)\) ; the theorem needs each \(\varphi_n\) to be a characteristic function, which has not been shown. For independent signs, \(S_n/\sqrt n\) has characteristic function \(\cos(t/\sqrt n)^n\) and the same limit (Example 10.1).
-
Let \(X\ge 0\) and suppose \(\mathbb E[X\mid\mathcal G_n]\to Y\) almost surely for an increasing sequence of sigma-algebras \(\mathcal G_n\) . Identify \(Y\) when \(X\) is integrable, and explain what can go wrong for the product martingale of Example 7.4, which is also nonnegative and almost surely convergent.
Hint
By Theorem 7.14, \(Y=\mathbb E[X\mid\mathcal G_\infty]\) . The product martingale is not of the form \(\mathbb E[X\mid\mathcal F_n]\) for any integrable \(X\) : by Theorem 7.13 that would make it uniformly integrable, and it is not, so its limit \(0\) does not close it.