Skip to content
Date: Oct 2, 2026
Builds on Chapter 2.

Run an infinite sequence of independent trials in which trial \(n\) succeeds with probability \(p_n\) , for \(n\ge 2\) . If \(p_n=1/n\) , the chance that no trial from the \(m\) -th to the \(N\) -th succeeds is the telescoping product \(\prod_{n=m}^{N}(1-\tfrac1n)=\tfrac{m-1}{N}\) , which tends to \(0\) as \(N\) grows: after any trial whatever, another success is certain to come. If \(p_n=1/n^2\) , the chance that no trial at all succeeds is \(\prod_{n\ge 2}(1-\tfrac1{n^2})=\tfrac12\) , and the chance of no success from the \(m\) -th trial on is \(\tfrac{m-1}{m}\) , which tends to \(1\) : successes are certain to stop.

Both sequences of success probabilities tend to zero. One produces infinitely many successes with probability one, the other with probability zero, and no choice of \(p_n\) produces an answer strictly between. This chapter explains all three facts. The first two are the Borel–Cantelli lemmas, which decide whether events happen infinitely often by testing whether their probabilities are summable. The third is Kolmogorov’s zero-one law, which says that for independent sequences every event that does not depend on any finite stretch of the sequence is trivial.

3.1 Infinitely often and eventually

Definition 3.1 (upper and lower limits of events). For events \(A_1,A_2,\dots\) ,

\[ \limsup_n A_n=\bigcap_{m\ge 1}\bigcup_{n\ge m}A_n,\qquad \liminf_n A_n=\bigcup_{m\ge 1}\bigcap_{n\ge m}A_n . \]

The first is the event that \(A_n\) occurs infinitely often, written \(\{A_n\ \text{i.o.}\}\) ; the second is the event that \(A_n\) occurs eventually, that is, for all but finitely many \(n\) .

Unwinding the quantifiers justifies the names. A point \(\omega\) lies in \(\limsup_nA_n\) when for every \(m\) there is an \(n\ge m\) with \(\omega\in A_n\) , which says that \(\omega\) belongs to infinitely many of the sets. It lies in \(\liminf_nA_n\) when there is an \(m\) such that \(\omega\in A_n\) for all \(n\ge m\) . Both are events, being countable intersections and unions of events. Three reformulations are used freely: the complement of \(\{A_n\ \text{i.o.}\}\) is \(\{A_n^{c}\ \text{eventually}\}\) ; the indicator of \(\limsup_nA_n\) is \(\limsup_n\mathbf 1_{A_n}\) ; and \(\omega\in\limsup_nA_n\) exactly when the count \(N(\omega)=\sum_n\mathbf 1_{A_n}(\omega)\) is infinite.

Lemma 3.2 (probabilities of upper and lower limits). For any events \(A_1,A_2,\dots\) ,

\[ \mathbb P\bigl(\liminf_nA_n\bigr)\le\liminf_n\mathbb P(A_n)\le\limsup_n\mathbb P(A_n)\le\mathbb P\bigl(\limsup_nA_n\bigr). \]

Proof. The events \(B_m=\bigcup_{n\ge m}A_n\) decrease to \(\limsup_nA_n\) , so \(\mathbb P(B_m)\to\mathbb P(\limsup_nA_n)\) by continuity of a finite measure along decreasing sequences. Since \(\mathbb P(B_m)\ge\sup_{n\ge m}\mathbb P(A_n)\) , letting \(m\to\infty\) gives the right-hand inequality. The left-hand one follows by applying this to the complements, and the middle one is trivial.

The lemma has a consequence worth stating in words: if every \(A_n\) has probability at least \(\delta\) , then with probability at least \(\delta\) infinitely many of them occur. What it cannot do is promise probability one, and it says nothing useful when \(\mathbb P(A_n)\to 0\) .

Example 3.1: Upper and lower limits of oscillating intervals

On \([0,1]\) with Lebesgue measure let \(A_n=[0,a_n]\) with

\[ a_n=\frac12+(-1)^n\Bigl(\frac14-\frac1{n+3}\Bigr). \]

The even-numbered endpoints \(a_2=0.55\) , \(a_4=\tfrac34-\tfrac17\) , and so on increase to \(3/4\) without reaching it; the odd-numbered endpoints \(a_1=\tfrac12\) , \(a_3=\tfrac14+\tfrac16\) , and so on decrease to \(1/4\) without reaching it.

A point \(x\) lies in infinitely many \(A_n\) if and only if \(x\le a_n\) for infinitely many \(n\) . For \(x<3/4\) this holds along the even \(n\) ; for \(x\ge 3/4\) it never holds. So \(\limsup_nA_n=[0,\tfrac34)\) . A point \(x\) lies in all but finitely many \(A_n\) if and only if \(x\le a_n\) for all large \(n\) , and the odd \(n\) are the constraint: this holds exactly for \(x\le 1/4\) . So \(\liminf_nA_n=[0,\tfrac14]\) . One limit set is half-open and the other closed, according to whether the endpoints approach their limit from inside or from outside. Here Lemma 3.2 holds with equality at both ends: \(\lambda(\liminf_nA_n)=\tfrac14=\liminf_n a_n\) and \(\lambda(\limsup_nA_n)=\tfrac34=\limsup_n a_n\) .

The inequalities can also be strict. Let \(B_n=[0,\tfrac12)\) for even \(n\) and \(B_n=[\tfrac12,1)\) for odd \(n\) . No point is in all but finitely many \(B_n\) , and every point of \([0,1)\) is in infinitely many, so \(\liminf_nB_n=\emptyset\) and \(\limsup_nB_n=[0,1)\) . The four quantities of the lemma are \(0<\tfrac12=\tfrac12<1\) .

3.2 Summable probabilities: the first lemma

Theorem 3.3 (first Borel–Cantelli lemma). If \(\sum_n\mathbb P(A_n)<\infty\) , then \(\mathbb P(A_n\ \text{i.o.})=0\) .

Proof. For every \(m\) , \(\limsup_nA_n\subseteq\bigcup_{n\ge m}A_n\) , so by countable subadditivity

\[ \mathbb P\bigl(\limsup_nA_n\bigr)\le\sum_{n\ge m}\mathbb P(A_n). \]

The right side is the tail of a convergent series and tends to \(0\) as \(m\to\infty\) .

No hypothesis relates the events to one another. They may be independent, nested, or identical. There is a second proof that explains why: the count \(N=\sum_n\mathbf 1_{A_n}\) has expectation \(\mathbb E N=\sum_n\mathbb P(A_n)\) , by the Monotone Convergence Theorem, and a nonnegative random variable with finite expectation is finite almost surely. The first lemma is the statement that a count with finite mean is finite. It is the workhorse for proving almost sure convergence: to show that \(X_n\to X\) almost surely it suffices to show that \(\sum_n\mathbb P(\lvert X_n-X\rvert>\varepsilon)<\infty\) for each \(\varepsilon>0\) , a criterion that Chapter 4 develops and Chapter 8 uses to prove the strong law of large numbers.

Example 3.2: An upper envelope for exponential variables

Let \(X_1,X_2,\dots\) each have the exponential law with rate \(1\) ; nothing is assumed about their joint law. For a constant \(c>0\) and \(n\ge 2\) ,

\[ \mathbb P(X_n>c\log n)=e^{-c\log n}=n^{-c}. \]

If \(c>1\) these probabilities are summable, so by Theorem 3.3, almost surely \(X_n\le c\log n\) for all but finitely many \(n\) . Hence \(\limsup_nX_n/\log n\le c\) almost surely. Applying this with \(c=1+1/k\) for \(k=1,2,\dots\) and intersecting the countably many events of probability one,

\[ \limsup_{n\to\infty}\frac{X_n}{\log n}\le 1\qquad\text{almost surely.} \]

The counting proof even bounds how often the envelope \(2\log n\) is exceeded: the expected number of \(n\ge 1\) with \(X_n>2\log n\) is \(\sum_{n\ge 1}n^{-2}=\pi^2/6\approx 1.645\) .

For \(c=1\) the probabilities are \(1/n\) , the series diverges, and the first lemma is silent. Whether \(X_n>\log n\) happens infinitely often now depends on how the variables are related. If \(X_n=X_1\) for every \(n\) , then \(X_n/\log n\to 0\) and the event happens only finitely often. If the variables are independent, the next theorem shows that it happens infinitely often.

3.3 Divergent probabilities and the role of independence

Theorem 3.4 (second Borel–Cantelli lemma). If the events \(A_1,A_2,\dots\) are independent and \(\sum_n\mathbb P(A_n)=\infty\) , then \(\mathbb P(A_n\ \text{i.o.})=1\) .

Proof. We show that the complement, the event that \(A_n^{c}\) occurs eventually, has probability \(0\) . It is the countable union over \(m\) of the events \(\bigcap_{n\ge m}A_n^{c}\) , so it is enough to show that each of these is null. The complements \(A_n^{c}\) are independent, since independence was defined for the sigma-algebras \(\{\emptyset,A_n,A_n^{c},\Omega\}\) . Using \(1-x\le e^{-x}\) , for \(N\ge m\) ,

\[ \mathbb P\Bigl(\bigcap_{n=m}^{N}A_n^{c}\Bigr)=\prod_{n=m}^{N}\bigl(1-\mathbb P(A_n)\bigr)\le\exp\Bigl(-\sum_{n=m}^{N}\mathbb P(A_n)\Bigr). \]

As \(N\to\infty\) the sum diverges and the bound tends to \(0\) ; the events on the left decrease to \(\bigcap_{n\ge m}A_n^{c}\) , whose probability is therefore \(0\) .

Together the two lemmas give a dichotomy for independent events: \(\mathbb P(A_n\ \text{i.o.})\) is \(0\) or \(1\) according as \(\sum_n\mathbb P(A_n)\) converges or diverges.

Example 3.3: The two schedules of the opening, computed exactly

Let \(A_2,A_3,\dots\) be independent with \(\mathbb P(A_n)=p_n\) .

Schedule \(p_n=1/n\) . The probability that none of \(A_m,\dots,A_N\) occurs is

\[ \prod_{n=m}^{N}\frac{n-1}{n}=\frac{m-1}{N}, \]

because each numerator cancels the preceding denominator. For \(m=1000\) and \(N=10^6\) it equals \(999/10^6\) , so a success between trials \(1000\) and \(10^6\) has probability \(0.999001\) . Letting \(N\to\infty\) , no stretch of trials is the last to contain a success, and \(\mathbb P(A_n\ \text{i.o.})=1\) , in agreement with Theorem 3.4.

Schedule \(p_n=1/n^2\) . Now

\[ \prod_{n=m}^{N}\Bigl(1-\frac1{n^2}\Bigr)=\prod_{n=m}^{N}\frac{(n-1)(n+1)}{n\cdot n}=\frac{m-1}{m}\cdot\frac{N+1}{N}\ \longrightarrow\ \frac{m-1}{m}\quad(N\to\infty). \]

With \(m=2\) : the probability that no trial ever succeeds is \(1/2\) . With general \(m\) : the probability that all successes occur before trial \(m\) is \((m-1)/m\) , which tends to \(1\) . So \(\mathbb P(A_n\ \text{i.o.})=0\) , in agreement with Theorem 3.3, and the calculation shows more than the lemma: it gives the law of the time of the last success.

Independence cannot simply be deleted from Theorem 3.4. Let \(A\) be an event of probability \(1/2\) and let \(A_n=A\) for all \(n\) . Then \(\sum_n\mathbb P(A_n)=\infty\) and \(\{A_n\ \text{i.o.}\}=A\) has probability \(1/2\) . Divergence of the series says that the expected count is infinite, and an infinite expectation does not force an infinite value: here the count is \(0\) on \(A^{c}\) and \(\infty\) on \(A\) . What independence contributes is control of the fluctuations of the count around its mean, and for that purpose much less than full independence is enough.

Theorem 3.5 (second lemma under pairwise independence). If the events \(A_1,A_2,\dots\) are pairwise independent and \(\sum_n\mathbb P(A_n)=\infty\) , then \(\mathbb P(A_n\ \text{i.o.})=1\) .

Proof. Let \(N_n=\sum_{k\le n}\mathbf 1_{A_k}\) and \(m_n=\mathbb E N_n=\sum_{k\le n}\mathbb P(A_k)\) , so \(m_n\uparrow\infty\) . The indicators are pairwise independent, hence by Corollary 2.7

\[ \operatorname{Var}(N_n)=\sum_{k\le n}\mathbb P(A_k)\bigl(1-\mathbb P(A_k)\bigr)\le m_n . \]

Chebyshev’s inequality gives \(\mathbb P(N_n\le m_n/2)\le\mathbb P(\lvert N_n-m_n\rvert\ge m_n/2)\le 4\operatorname{Var}(N_n)/m_n^2\le 4/m_n\) . Let \(N_\infty=\lim_nN_n\) be the total count and fix an integer \(K\) . For all \(n\) so large that \(m_n/2\ge K\) ,

\[ \mathbb P(N_\infty\le K)\le\mathbb P(N_n\le K)\le\mathbb P(N_n\le m_n/2)\le\frac{4}{m_n}. \]

Letting \(n\to\infty\) gives \(\mathbb P(N_\infty\le K)=0\) for every \(K\) , so \(N_\infty=\infty\) almost surely.

Example 3.4: The exact growth rate of independent exponential variables

Return to Example 3.2 and assume now that \(X_1,X_2,\dots\) are independent. The events \(\{X_n>\log n\}\) are independent with probabilities \(1/n\) , whose sum diverges, so by Theorem 3.4 almost surely \(X_n>\log n\) for infinitely many \(n\) , and therefore \(\limsup_nX_n/\log n\ge 1\) . Combined with the upper bound of Example 3.2, which needed no independence,

\[ \limsup_{n\to\infty}\frac{X_n}{\log n}=1\qquad\text{almost surely.} \]

The two halves use the two lemmas, and the constant \(1\) is exactly the exponent at which \(\sum_nn^{-c}\) changes from divergent to convergent. As in Example 3.3, the probability that \(X_n\le\log n\) for every \(n\) from \(m\) to \(N\) is \(\prod_{n=m}^{N}(1-\tfrac1n)=\tfrac{m-1}{N}\) .

The lower limit behaves differently: \(\mathbb P(X_n\le\varepsilon)=1-e^{-\varepsilon}\) does not depend on \(n\) , so by Theorem 3.4 the event \(\{X_n\le\varepsilon\}\) occurs infinitely often almost surely for each \(\varepsilon>0\) , and \(\liminf_nX_n=0\) . The sequence \(X_n/\log n\) has no almost sure limit, although \(\mathbb P(X_n/\log n>\varepsilon)=n^{-\varepsilon}\to 0\) for each fixed \(\varepsilon>0\) . Chapter 4 gives names to these two kinds of behaviour.

Example 3.5: Records

Let \(X_1,X_2,\dots\) be independent with a common continuous distribution function, and let \(A_n\) be the event that \(X_n\) is a record: \(X_n>X_k\) for all \(k<n\) . We claim that the events \(A_n\) are independent and \(\mathbb P(A_n)=1/n\) .

By Exercise 2.8, \(\mathbb P(X_i=X_j)=0\) for \(i\ne j\) , so almost surely there are no ties. Fix \(n\) . The joint law of \((X_1,\dots,X_n)\) is a product of \(n\) equal factors, so it is unchanged by permuting the coordinates, and each of the \(n!\) possible orderings of the values is equally likely. For \(k\le n\) let \(\rho_k\in\{1,\dots,k\}\) be the rank of \(X_k\) among \(X_1,\dots,X_k\) , counted from the largest, so that \(A_k=\{\rho_k=1\}\) . The ordering of \(X_1,\dots,X_n\) determines \((\rho_1,\dots,\rho_n)\) , and conversely the ordering of the first \(k\) values can be rebuilt from the ordering of the first \(k-1\) and the position \(\rho_k\) at which the \(k\) -th is inserted. So orderings correspond one-to-one with vectors in \(\{1\}\times\{1,2\}\times\dots\times\{1,\dots,n\}\) , a set of exactly \(n!\) elements, and \((\rho_1,\dots,\rho_n)\) is uniformly distributed on that product set. Therefore the \(\rho_k\) are independent, each uniform on \(\{1,\dots,k\}\) , which proves the claim. For instance \(\mathbb P(A_2\cap A_3\cap A_5)=\tfrac12\cdot\tfrac13\cdot\tfrac15=\tfrac1{30}\) : of the \(120\) orderings of five values, exactly \(4\) have records at the second, third, and fifth places.

Both lemmas now apply. Since \(\sum_n1/n=\infty\) and the \(A_n\) are independent, Theorem 3.4 shows that almost surely there are infinitely many records. On the other hand the event of a double record, \(A_n\cap A_{n+1}\) , has probability \(\tfrac1{n(n+1)}=\tfrac1n-\tfrac1{n+1}\) , and these probabilities sum to \(1\) . By Theorem 3.3, almost surely only finitely many records are immediately followed by another, and the counting proof says that the expected number of double records is exactly \(1\) . The double-record events overlap and are not independent of one another, which is no obstacle, because the first lemma asks for nothing but summability.

Common mistake. Using the second lemma without independence, or reading the lemmas backwards. From \(\sum_n\mathbb P(A_n)=\infty\) alone nothing follows about \(\{A_n\ \text{i.o.}\}\) beyond Lemma 3.2; the events \(A_n=\{U<1/n\}\) for one uniform variable \(U\) have divergent probabilities and occur only finitely often at every sample point (Exercise 9). In the other direction, \(\mathbb P(A_n)\to 0\) does not make the events occur finitely often: the independent schedule \(p_n=1/n\) has \(p_n\to 0\) and infinitely many successes. Before applying either lemma, write down which one it is and what it requires: the first needs a convergent series and nothing else; the second needs a divergent series and independence, pairwise at least.

3.4 Tail events and Kolmogorov’s zero-one law

For independent events the probability of “infinitely often” is \(0\) or \(1\) . The same turns out to be true of every event determined by the far future of an independent sequence.

Definition 3.6 (tail sigma-algebra). Let \(X_1,X_2,\dots\) be random elements. Their tail sigma-algebra is

\[ \mathcal T=\bigcap_{n\ge 1}\sigma(X_n,X_{n+1},X_{n+2},\dots). \]

Events in \(\mathcal T\) are tail events, and a \(\mathcal T\) -measurable random variable is a tail random variable.

An event is a tail event when, for every \(n\) , its occurrence can be decided without looking at \(X_1,\dots,X_{n-1}\) . For real random variables the events \(\{X_n\in B_n\ \text{i.o.}\}\) , \(\{X_n\ \text{converges}\}\) , and \(\{\sum_nX_n\ \text{converges}\}\) are tail events, because altering finitely many terms changes neither infinitely-often statements nor the convergence of a sequence or a series. With \(S_n=X_1+\dots+X_n\) , the variables \(\limsup_nX_n\) and \(\limsup_nS_n/n\) are tail random variables with values in \([-\infty,\infty]\) ; for the second, write \(S_n/n=S_m/n+(S_n-S_m)/n\) and observe that the first term tends to \(0\) for each fixed \(m\) . By contrast \(\{X_1>0\}\) , \(\{\sup_nX_n>1\}\) , and \(\{\sum_nX_n\ \text{converges to a sum exceeding }1\}\) are not tail events in general: changing \(X_1\) can change each of them.

Theorem 3.7 (Kolmogorov’s zero-one law). Let \(X_1,X_2,\dots\) be independent random elements with tail sigma-algebra \(\mathcal T\) . Then \(\mathbb P(A)\in\{0,1\}\) for every \(A\in\mathcal T\) , and every tail random variable with values in \([-\infty,\infty]\) is almost surely constant.

Proof. Let \(\mathcal F_n=\sigma(X_1,\dots,X_n)\) and \(\mathcal T_n=\sigma(X_{n+1},X_{n+2},\dots)\) . By grouping (Corollary 2.4), \(\mathcal F_n\) and \(\mathcal T_n\) are independent. Since \(\mathcal T\subseteq\mathcal T_n\) , the sigma-algebras \(\mathcal F_n\) and \(\mathcal T\) are independent, and this holds for every \(n\) . Consequently the class \(\mathcal C=\bigcup_n\mathcal F_n\) is independent of \(\mathcal T\) .

The class \(\mathcal C\) is a \(\pi\) -system, because the \(\mathcal F_n\) increase, so two members of \(\mathcal C\) lie in a common \(\mathcal F_n\) and their intersection lies there too. It generates \(\mathcal F_\infty=\sigma(X_1,X_2,\dots)\) . By Theorem 2.2, applied to the \(\pi\) -systems \(\mathcal C\) and \(\mathcal T\) , the sigma-algebras \(\mathcal F_\infty\) and \(\mathcal T\) are independent.

But \(\mathcal T\subseteq\mathcal F_\infty\) . So every \(A\in\mathcal T\) is independent of itself: \(\mathbb P(A)=\mathbb P(A\cap A)=\mathbb P(A)^2\) , whence \(\mathbb P(A)\in\{0,1\}\) . If \(Z\) is a tail random variable, then \(\mathbb P(Z\le z)\in\{0,1\}\) for each \(z\) , and with \(c=\inf\{z:\mathbb P(Z\le z)=1\}\in[-\infty,\infty]\) the argument of Exercise 2.3 shows \(\mathbb P(Z=c)=1\) .

The proof is a small paradox turned into a theorem: a tail event is independent of every finite stretch of the sequence, hence of the whole sequence, hence of itself. The law has striking consequences. For independent \(X_n\) the series \(\sum_nX_n\) converges with probability \(0\) or \(1\) ; there is no independent sequence whose series converges with probability one half. The upper limit of the averages \(S_n/n\) is almost surely a constant, possibly infinite. For independent random signs \(\varepsilon_n=\pm1\) , each with probability \(1/2\) , the series \(\sum_n\varepsilon_n/n\) either converges almost surely or diverges almost surely.

What the law does not do is say which. It is a statement of form, and deciding between \(0\) and \(1\) needs a quantitative tool: the Borel–Cantelli lemmas in the examples above, the martingale convergence theorem for the random harmonic series (Chapter 7 shows that it converges almost surely), the strong law for averages (Chapter 8). The law is also strictly about independence and about tail events. If \(X_n=Y\) for every \(n\) , where \(Y\) takes two values with probability \(1/2\) each, the tail sigma-algebra is \(\sigma(Y)\) and contains events of probability \(1/2\) . And the event that a random walk \(S_n\) returns to \(0\) infinitely often is not a tail event of the steps, since changing the first step shifts the whole walk; its probability is nevertheless \(0\) or \(1\) for identically distributed steps, by a different zero-one law due to Hewitt and Savage, which we mention without proof.

A sequence of events can thus occur infinitely often with certainty while each event becomes ever less likely. The same tension between what happens along the whole sequence and what is likely at each index is the difference between almost sure convergence and convergence in probability, and it is the starting point of the next chapter.

Exercises

  1. Prove that \((\limsup_nA_n)^{c}=\liminf_nA_n^{c}\) and that \(\mathbf 1_{\limsup_nA_n}=\limsup_n\mathbf 1_{A_n}\) pointwise, and deduce Lemma 3.2 from Fatou’s lemma applied to indicators.

    Hint

    A point fails to lie in infinitely many \(A_n\) exactly when it lies in all but finitely many \(A_n^{c}\) . For indicators, \(\limsup_n\mathbf 1_{A_n}(\omega)=1\) exactly when \(\mathbf 1_{A_n}(\omega)=1\) infinitely often. Fatou’s lemma for \(\mathbf 1_{A_n}\) gives the left inequality of Lemma 3.2; applied to \(1-\mathbf 1_{A_n}\) it gives the right one.

  2. On the real line let \(A_n\) be the closed interval with endpoints \((-1)^n/n\) and \(1+(-1)^n/n\) . Find \(\limsup_nA_n\) and \(\liminf_nA_n\) , paying attention to the two endpoints \(0\) and \(1\) .

    Answer

    \(\limsup_nA_n=[0,1]\) and \(\liminf_nA_n=(0,1)\) . The point \(0\) lies in \(A_n\) for odd \(n\) only and the point \(1\) for even \(n\) only, so each is in infinitely many of the sets and misses infinitely many; every \(x\in(0,1)\) lies in \(A_n\) once \(1/n<\min(x,1-x)\) .

  3. Construct events \(A_n\) in \(([0,1),\mathcal B,\lambda)\) with \(\lambda(A_n)\to 0\) and \(\lambda(A_n\ \text{i.o.})=1\) , and explain why this does not contradict Lemma 3.2 or Theorem 3.3.

    Solution

    Number the dyadic subintervals of \([0,1)\) generation by generation: for \(k\ge 0\) and \(0\le j<2^k\) let

    \[ A_{2^k+j}=\bigl[j2^{-k},(j+1)2^{-k}\bigr). \]

    Every integer \(n\ge 1\) is \(2^k+j\) for exactly one such pair, so this defines \(A_n\) for all \(n\) , and \(\lambda(A_n)=2^{-k}\) when \(2^k\le n<2^{k+1}\) . Hence \(\lambda(A_n)\to 0\) . For each \(k\) the \(2^k\) intervals of generation \(k\) partition \([0,1)\) , so every point \(\omega\) lies in exactly one set of each generation and therefore in infinitely many \(A_n\) . Thus \(\{A_n\ \text{i.o.}\}=[0,1)\) and has probability \(1\) .

    There is no contradiction with Lemma 3.2, which only asserts \(\mathbb P(A_n\ \text{i.o.})\ge\limsup_n\mathbb P(A_n)=0\) . Nor with Theorem 3.3: its hypothesis fails, since each generation contributes total probability \(2^k\cdot 2^{-k}=1\) to the series, so \(\sum_n\lambda(A_n)=\infty\) . The events are also far from independent, so Theorem 3.4 is not what produces the conclusion; the geometry does.

  4. Let \(X_1,X_2,\dots\) be arbitrary real random variables on one probability space. Prove that there are constants \(c_n>0\) such that \(X_n/c_n\to 0\) almost surely.

    Solution

    Each \(X_n\) is finite at every sample point, so the events \(\{\lvert X_n\rvert>a\}\) decrease to the empty set as \(a\to\infty\) , and \(\mathbb P(\lvert X_n\rvert>a)\to 0\) by continuity of \(\mathbb P\) . Choose \(a_n>0\) with \(\mathbb P(\lvert X_n\rvert>a_n)\le 2^{-n}\) and put \(c_n=na_n\) . Then

    \[ \sum_n\mathbb P\Bigl(\frac{\lvert X_n\rvert}{c_n}>\frac1n\Bigr)=\sum_n\mathbb P(\lvert X_n\rvert>a_n)\le\sum_n2^{-n}<\infty . \]

    By the first Borel–Cantelli lemma, almost surely \(\lvert X_n\rvert/c_n\le 1/n\) for all but finitely many \(n\) , and on that event \(X_n/c_n\to 0\) . No assumption on the joint law was used; the constants depend only on the individual laws. The result says that any sequence of random variables can be normalized to converge to zero almost surely, so a statement of the form “\(X_n/c_n\to 0\) ” carries information only when the \(c_n\) are specified.

  5. Let \(X_1,X_2,\dots\) be identically distributed. Prove that if \(\mathbb E\lvert X_1\rvert<\infty\) then \(X_n/n\to 0\) almost surely, with no independence assumption, and that if the \(X_n\) are independent and \(\mathbb E\lvert X_1\rvert=\infty\) then \(\mathbb P(\lvert X_n\rvert>n\ \text{i.o.})=1\) .

    Solution

    Assume \(\mathbb E\lvert X_1\rvert<\infty\) and fix \(\varepsilon>0\) . Since \(X_n\) has the law of \(X_1\) , and using the sum form of Proposition 1.12 for \(Y=\lvert X_1\rvert/\varepsilon\) ,

    \[ \sum_{n\ge 1}\mathbb P(\lvert X_n\rvert>\varepsilon n)=\sum_{n\ge 1}\mathbb P(Y>n)\le\sum_{n\ge 1}\mathbb P(Y\ge n)\le\mathbb E Y=\frac{\mathbb E\lvert X_1\rvert}{\varepsilon}<\infty . \]

    By Theorem 3.3, almost surely \(\lvert X_n\rvert\le\varepsilon n\) for all large \(n\) , so \(\limsup_n\lvert X_n\rvert/n\le\varepsilon\) almost surely. Taking \(\varepsilon=1/k\) and intersecting over \(k\) gives \(X_n/n\to 0\) almost surely. Only the common law was used.

    Now let the \(X_n\) be independent with \(\mathbb E\lvert X_1\rvert=\infty\) . The other half of Proposition 1.12 gives \(\sum_{n\ge 0}\mathbb P(\lvert X_1\rvert>n)\ge\mathbb E\lvert X_1\rvert=\infty\) , so \(\sum_{n\ge 1}\mathbb P(\lvert X_n\rvert>n)=\infty\) . The events \(\{\lvert X_n\rvert>n\}\) are independent, each being determined by a different \(X_n\) , and Theorem 3.4 gives \(\mathbb P(\lvert X_n\rvert>n\ \text{i.o.})=1\) . In particular \(X_n/n\) does not tend to \(0\) , a fact that Chapter 8 turns into the converse of the strong law.

  6. Let \(A_1,A_2,\dots\) be independent events with \(\mathbb P(A_n)<1\) for every \(n\) . Prove that \(\mathbb P(\bigcup_nA_n)=1\) if and only if \(\mathbb P(A_n\ \text{i.o.})=1\) .

    Hint

    One direction is trivial. For the other, if \(\sum_n\mathbb P(A_n)<\infty\) then \(\prod_n(1-\mathbb P(A_n))>0\) because every factor is positive and the series converges, so \(\mathbb P(\bigcup_nA_n)<1\) ; hence \(\mathbb P(\bigcup_nA_n)=1\) forces the series to diverge, and Theorem 3.4 applies.

  7. Let \(A_1,A_2,\dots\) be independent with \(\mathbb P(A_n)=1/(4n^2)\) . Find the exact probability that none of the events occurs, using the product formula \(\prod_{n\ge 1}(1-\tfrac{x^2}{n^2\pi^2})=\tfrac{\sin x}{x}\) .

    Answer

    By independence the probability is \(\prod_{n\ge 1}(1-\tfrac1{4n^2})\) , which is the product formula at \(x=\pi/2\) : \(\sin(\pi/2)/(\pi/2)=2/\pi\approx 0.637\) .

  8. In a sequence of independent fair coin tosses let \(L_n\) be the number of consecutive heads starting at toss \(n\) . Prove that \(\limsup_nL_n/\log_2n=1\) almost surely, using disjoint blocks of tosses to obtain independent events for the lower bound.

    Hint

    \(\mathbb P(L_n\ge r)=2^{-r}\) . For the upper bound, \(\mathbb P(L_n\ge(1+\varepsilon)\log_2n)\le n^{-(1+\varepsilon)}\) is summable; use Theorem 3.3. For the lower bound define positions \(t_1=2\) , \(t_{k+1}=t_k+r_k\) with \(r_k=\lfloor(1-\varepsilon)\log_2t_k\rfloor+1\) , and let \(B_k\) be the event that tosses \(t_k,\dots,t_k+r_k-1\) are all heads. The blocks are disjoint, so the \(B_k\) are independent, \(\mathbb P(B_k)\ge\tfrac12t_k^{-(1-\varepsilon)}\) , and \(t_k\le Ck\log k\) makes the series diverge; use Theorem 3.4.

  9. Let \(U\) be uniform on \((0,1)\) and \(A_n=\{U<1/n\}\) . A student argues that \(\sum_n\mathbb P(A_n)=\infty\) , so \(A_n\) occurs infinitely often almost surely. Find \(\mathbb P(A_n\ \text{i.o.})\) exactly, name the hypothesis that fails, and verify its failure with a specific pair of events.

    Solution

    The events decrease: \(A_1\supseteq A_2\supseteq\cdots\) . For a decreasing sequence, a point lies in infinitely many of the sets exactly when it lies in all of them, so

    \[ \{A_n\ \text{i.o.}\}=\bigcap_n\{U<1/n\}=\{U\le 0\}=\emptyset , \]

    since \(U\) takes values in \((0,1)\) . Thus \(\mathbb P(A_n\ \text{i.o.})=0\) ; indeed at each sample point \(A_n\) occurs exactly for \(n<1/U(\omega)\) , finitely many times. The series \(\sum_n\mathbb P(A_n)=\sum_n1/n\) does diverge, so the first lemma does not apply, and the student has used the second lemma. Its independence hypothesis fails. For example \(A_2\cap A_3=A_3\) , so \(\mathbb P(A_2\cap A_3)=\tfrac13\) , while \(\mathbb P(A_2)\,\mathbb P(A_3)=\tfrac12\cdot\tfrac13=\tfrac16\) . The events are not even pairwise independent, so Theorem 3.5 does not apply either. Lemma 3.2 is consistent with the answer: it gives only \(\mathbb P(A_n\ \text{i.o.})\ge\limsup_n\mathbb P(A_n)=0\) .

  10. Show that the converse of Theorem 3.3 is false: give events with \(\sum_n\mathbb P(A_n)=\infty\) of which only finitely many occur at every sample point, and say what the expected number of occurrences is.

    Answer

    The events \(A_n=\{U<1/n\}\) of Exercise 9 serve: at each sample point only finitely many occur, yet the expected number of occurrences is \(\sum_n1/n=\infty\) . The count is finite everywhere and not integrable.

  11. Show directly from Theorems 3.3 and 3.4 that \(\mathbb P(A_n\ \text{i.o.})\in\{0,1\}\) for independent events, and construct dependent events with \(\mathbb P(A_n\ \text{i.o.})=1/3\) .

    Answer

    For independent events the series \(\sum_n\mathbb P(A_n)\) either converges, giving probability \(0\) by Theorem 3.3, or diverges, giving probability \(1\) by Theorem 3.4. For the example take an event \(A\) of probability \(1/3\) and \(A_n=A\) for every \(n\) .

  12. For a sequence of real random variables \(X_n\) with partial sums \(S_n\) , decide which of the following are tail events: (a) \(\{X_n\to 0\}\) ; (b) \(\{\sup_nX_n>1\}\) ; (c) \(\{\sum_nX_n\ \text{converges}\}\) ; (d) \(\{\sum_nX_n\ \text{converges and the sum is positive}\}\) ; (e) \(\{\limsup_nS_n/n\le 0\}\) ; (f) \(\{S_n>0\ \text{i.o.}\}\) .

    Answer

    Tail events: (a), (c), (e). Not tail events in general: (b), (d), (f); each can be changed by altering \(X_1\) alone.

  13. Let \(X_0,X_1,X_2,\dots\) be independent random variables. Prove that the radius of convergence of the random power series \(\sum_nX_nz^n\) is almost surely equal to a constant in \([0,\infty]\) .

    Solution

    By the Cauchy–Hadamard formula the radius of convergence is

    \[ R=\Bigl(\limsup_{n\to\infty}\lvert X_n\rvert^{1/n}\Bigr)^{-1}, \]

    with the conventions \(1/0=\infty\) and \(1/\infty=0\) . The upper limit of a sequence is unchanged when finitely many of its terms are altered or discarded, so for every \(m\) the variable \(\limsup_n\lvert X_n\rvert^{1/n}\) equals \(\limsup_{n\ge m}\lvert X_n\rvert^{1/n}\) , which is a measurable function of \(X_m,X_{m+1},\dots\) with values in \([0,\infty]\) . Hence it is measurable with respect to \(\sigma(X_m,X_{m+1},\dots)\) for every \(m\) , that is, with respect to the tail sigma-algebra \(\mathcal T\) , and so is \(R\) . The \(X_n\) are independent, so by Theorem 3.7 the tail random variable \(R\) is almost surely equal to a constant in \([0,\infty]\) . The theorem does not say what the constant is; for identically distributed \(X_n\) that are not almost surely zero, the Borel–Cantelli lemmas show it is \(1\) when \(\mathbb E\log^{+}\lvert X_0\rvert<\infty\) and \(0\) otherwise.

  14. Let \(X_1,X_2,\dots\) be independent and suppose \(S_n/n\) converges almost surely to a random variable \(Z\) . Prove that \(Z\) is almost surely constant.

    Hint

    On the event of convergence, \(Z=\limsup_nS_n/n\) , which is a tail random variable by the decomposition \(S_n/n=S_m/n+(S_n-S_m)/n\) . Apply Theorem 3.7.

  15. Prove that in an infinite sequence of independent fair coin tosses, almost surely every finite pattern of heads and tails appears infinitely often.

    Solution

    Let \(\xi_1,\xi_2,\dots\) be the tosses and fix a pattern \(w=(w_1,\dots,w_r)\) of length \(r\) . Cut the sequence into consecutive disjoint blocks of length \(r\) and let \(E_k\) be the event that the \(k\) -th block equals \(w\) :

    \[ E_k=\{\xi_{(k-1)r+1}=w_1,\dots,\xi_{kr}=w_r\}. \]

    Each \(E_k\) is determined by its own block of tosses, the blocks are disjoint, and so the events \(E_k\) are independent by grouping (Corollary 2.4). Each has probability \(2^{-r}\) , so \(\sum_k\mathbb P(E_k)=\infty\) . By the second Borel–Cantelli lemma, almost surely \(E_k\) occurs for infinitely many \(k\) , and in particular the pattern \(w\) appears infinitely often. There are countably many finite patterns, and a countable intersection of events of probability one has probability one. Hence almost surely every finite pattern appears infinitely often. Restricting to aligned blocks discards many appearances, but it is what produces independent events; overlapping windows are dependent and Theorem 3.4 would not apply to them directly.

  16. Let \(R_n\) be the number of records among \(X_1,\dots,X_n\) in Example 3.5. Find \(\mathbb E R_n\) and \(\operatorname{Var}(R_n)\) , evaluate them for \(n=100\) , and prove that \(\mathbb P(\lvert R_n/\log n-1\rvert>\varepsilon)\to 0\) for every \(\varepsilon>0\) .

    Solution

    By Example 3.5, \(R_n=\sum_{k\le n}\mathbf 1_{A_k}\) with the \(A_k\) independent and \(\mathbb P(A_k)=1/k\) . Writing \(H_n=\sum_{k\le n}1/k\) ,

    \[ \mathbb E R_n=H_n,\qquad \operatorname{Var}(R_n)=\sum_{k\le n}\frac1k\Bigl(1-\frac1k\Bigr)=H_n-\sum_{k\le n}\frac1{k^2}, \]

    the variance by Corollary 2.7. For \(n=100\) these are \(H_{100}\approx 5.187\) and \(5.187-1.635\approx 3.552\) : among a hundred observations one expects only about five records.

    Since \(\operatorname{Var}(R_n)\le H_n\) , Chebyshev’s inequality gives, for \(\delta>0\) ,

    \[ \mathbb P\bigl(\lvert R_n-H_n\rvert>\delta H_n\bigr)\le\frac{\operatorname{Var}(R_n)}{\delta^2H_n^2}\le\frac1{\delta^2H_n}\longrightarrow 0 , \]

    because \(H_n\to\infty\) . Finally \(H_n/\log n\to 1\) , so given \(\varepsilon>0\) choose \(\delta=\varepsilon/3\) and \(n\) so large that \(\lvert H_n/\log n-1\rvert<\varepsilon/3\) and \(H_n/\log n<2\) ; then \(\lvert R_n/\log n-1\rvert>\varepsilon\) forces \(\lvert R_n-H_n\rvert>\delta H_n\) . Hence \(\mathbb P(\lvert R_n/\log n-1\rvert>\varepsilon)\to 0\) .

  17. Let \(Y\) take the values \(0\) and \(1\) with probability \(1/2\) each and let \(X_n=Y\) for all \(n\) . Identify the tail sigma-algebra of \((X_n)\) , exhibit a tail event of probability \(1/2\) , and say which step of the proof of Theorem 3.7 fails.

    Answer

    \(\sigma(X_n,X_{n+1},\dots)=\sigma(Y)\) for every \(n\) , so \(\mathcal T=\sigma(Y)\) and \(\{Y=1\}\) is a tail event of probability \(1/2\) . The step that fails is the independence of \(\mathcal F_n\) and \(\mathcal T_n\) : both equal \(\sigma(Y)\) .

  18. Let \(X_1,X_2,\dots\) be independent with \(\mathbb P(X_n=1)=p_n\) and \(\mathbb P(X_n=0)=1-p_n\) . Prove that \(X_n\to 0\) almost surely if and only if \(\sum_np_n<\infty\) , whereas \(\mathbb P(\lvert X_n\rvert>\varepsilon)\to 0\) for every \(\varepsilon>0\) if and only if \(p_n\to 0\) .

    Solution

    Each \(X_n\) takes only the values \(0\) and \(1\) , so \(X_n(\omega)\to 0\) exactly when \(X_n(\omega)=1\) for only finitely many \(n\) . Thus \(\{X_n\to 0\}\) is the complement of \(\{X_n=1\ \text{i.o.}\}\) . The events \(\{X_n=1\}\) are independent with probabilities \(p_n\) . If \(\sum_np_n<\infty\) , Theorem 3.3 gives \(\mathbb P(X_n=1\ \text{i.o.})=0\) and so \(X_n\to 0\) almost surely. If \(\sum_np_n=\infty\) , Theorem 3.4 gives \(\mathbb P(X_n=1\ \text{i.o.})=1\) , so \(X_n\to 0\) with probability \(0\) . This proves the first equivalence, and shows incidentally that the probability of convergence is never strictly between \(0\) and \(1\) .

    For the second, if \(0<\varepsilon<1\) then \(\{\lvert X_n\rvert>\varepsilon\}=\{X_n=1\}\) has probability \(p_n\) , and if \(\varepsilon\ge 1\) the event is empty. So \(\mathbb P(\lvert X_n\rvert>\varepsilon)\to 0\) for every \(\varepsilon>0\) exactly when \(p_n\to 0\) . With \(p_n=1/n\) the second condition holds and the first fails: the probability of a nonzero value tends to zero, yet almost surely nonzero values keep occurring.

Search knowledge base

Explore

On this page

Note