The constants \(X_n=1/n\) converge to \(0\) in every sense one could ask for. Their distribution functions are \(F_n(x)=\mathbf 1_{\{x\ge 1/n\}}\) , and that of the limit is \(F(x)=\mathbf 1_{\{x\ge 0\}}\) . At the point \(x=0\) we have \(F_n(0)=0\) for every \(n\) and \(F(0)=1\) . So the most innocent convergent sequence there is fails the most natural definition of convergence in distribution, “the distribution functions converge”. Had the constants been \(-1/n\) , the definition would have been satisfied.
Chapter 4 avoided the difficulty by defining convergence in distribution through expectations of bounded continuous functions: \(f(1/n)\to f(0)\) and \(f(-1/n)\to f(0)\) alike. This chapter develops that definition into a theory. It shows that the definition is equivalent to convergence of the distribution functions at the points where the limit is continuous, and to several other conditions. It identifies the one obstruction to extracting convergent subsequences from a sequence of laws, the escape of mass to infinity. And it constructs the tool with which convergence in distribution is proved in practice: the characteristic function, a complete description of a law that turns sums of independent variables into products. The chapter ends with the theorem that ties the two together, and Chapter 10 applies it.
9.1 Weak convergence of laws
Definition 9.1 (weak convergence). Probability measures \(\mu_n\) on \((\mathbb R,\mathcal B(\mathbb R))\) converge weakly to a probability measure \(\mu\) , written \(\mu_n\Rightarrow\mu\) , if
By Theorem 1.11, random variables \(X_n\) converge in distribution to \(X\) , in the sense of Definition 4.1, exactly when their laws converge weakly. The statement concerns laws only.
Theorem 9.2 (portmanteau theorem). Let \(\mu_n\) and \(\mu\) be probability measures on \(\mathbb R\) with distribution functions \(F_n\) and \(F\) . The following are equivalent.
(i) \(\mu_n\Rightarrow\mu\) .
(ii) \(\int f\,d\mu_n\to\int f\,d\mu\) for every bounded Lipschitz function \(f\) .
(iii) \(\limsup_n\mu_n(C)\le\mu(C)\) for every closed set \(C\) .
(iv) \(\liminf_n\mu_n(G)\ge\mu(G)\) for every open set \(G\) .
(v) \(\mu_n(A)\to\mu(A)\) for every Borel set \(A\) whose boundary \(\partial A\) has \(\mu(\partial A)=0\) .
(vi) \(F_n(x)\to F(x)\) at every point \(x\) at which \(F\) is continuous.
Proof. (i) implies (ii) because Lipschitz functions are continuous.
(ii) implies (iii). Let \(C\) be closed and \(f_k(x)=\max\bigl(0,1-k\,\operatorname{dist}(x,C)\bigr)\) . Each \(f_k\) is bounded and Lipschitz, \(f_k\ge\mathbf 1_C\) , and \(f_k\downarrow\mathbf 1_C\) pointwise as \(k\to\infty\) , since a point outside a closed set is at positive distance from it. For each \(k\) , \(\limsup_n\mu_n(C)\le\lim_n\int f_k\,d\mu_n=\int f_k\,d\mu\) , and letting \(k\to\infty\) the right side tends to \(\mu(C)\) by dominated convergence.
(iii) and (iv) are equivalent by passing to complements.
(iii) and (iv) imply (v). Let \(A^{\circ}\) be the interior and \(\bar A\) the closure of \(A\) . Then
and the two ends differ by \(\mu(\bar A\setminus A^{\circ})=\mu(\partial A)=0\) .
(v) implies (vi). The boundary of \((-\infty,x]\) is \(\{x\}\) , and \(\mu\{x\}=0\) exactly when \(F\) is continuous at \(x\) .
(vi) implies (i). Let \(f\) be continuous with \(\lvert f\rvert\le M\) , and \(\varepsilon>0\) . The points of discontinuity of \(F\) are countable (Exercise 1.4), so its continuity points are dense. Choose continuity points \(a<b\) with \(F(a)<\varepsilon\) and \(1-F(b)<\varepsilon\) , and, since \(f\) is uniformly continuous on \([a,b]\) , continuity points \(a=t_0<t_1<\dots<t_m=b\) such that \(f\) varies by at most \(\varepsilon\) on each \([t_{i-1},t_i]\) . Let \(g=\sum_{i\le m}f(t_i)\mathbf 1_{(t_{i-1},t_i]}\) . Then \(\lvert f-g\rvert\le\varepsilon\) on \((a,b]\) , and \(\lvert f-g\rvert\le M\) elsewhere, so
and likewise for \(\mu\) . By (vi), \(\int g\,d\mu_n=\sum_if(t_i)(F_n(t_i)-F_n(t_{i-1}))\to\int g\,d\mu\) , and \(F_n(a)+1-F_n(b)\to F(a)+1-F(b)<2\varepsilon\) . Hence \(\limsup_n\lvert\int f\,d\mu_n-\int f\,d\mu\rvert\le 2\varepsilon+4M\varepsilon\) .
The theorem’s odd name is that of a travelling bag with many compartments. Each condition is the convenient one in some argument: (ii) for estimates, (iii) and (iv) for inequalities, (vi) for explicit laws on the line. The inequalities in (iii) and (iv) can be strict, and that is exactly what happens at the origin in the opening example.
Example 9.1: Point masses and a discrete approximation to the uniform law
Point masses. Let \(\mu_n=\delta_{1/n}\) and \(\mu=\delta_0\) . Then \(\int f\,d\mu_n=f(1/n)\to f(0)\) , so \(\mu_n\Rightarrow\mu\) . For the closed set \(C=\{0\}\) , \(\mu_n(C)=0\) for all \(n\) and \(\mu(C)=1\) : condition (iii) holds with strict inequality. For the open set \(G=(0,\infty)\) , \(\mu_n(G)=1\) and \(\mu(G)=0\) : condition (iv) holds with strict inequality. The only discontinuity point of \(F\) is \(x=0\) , and it is the only point where \(F_n(x)\not\to F(x)\) , as (vi) allows. Mass may arrive at a closed set in the limit and may leave an open one; it cannot do the opposite.
A discrete uniform law. Let \(\mu_n\) put mass \(1/n\) at each of the points \(1/n,2/n,\dots,n/n\) . Its distribution function is \(F_n(x)=\lfloor nx\rfloor/n\) on \([0,1]\) , which differs from \(F(x)=x\) by at most \(1/n\) . By (vi), \(\mu_n\) converges weakly to the uniform law \(\mu\) on \((0,1)\) . In terms of (i) this is the convergence of Riemann sums: for instance \(\int x^2\,d\mu_n=\tfrac1n\sum_{k\le n}(k/n)^2=\tfrac{(n+1)(2n+1)}{6n^2}\to\tfrac13\) .
Yet every \(\mu_n\) is carried by the rational numbers, \(\mu_n(\mathbb Q)=1\) , while \(\mu(\mathbb Q)=0\) . There is no contradiction with (v): the boundary of \(\mathbb Q\) is the whole line. Weak convergence gives convergence of the probabilities of sets with negligible boundary and of nothing more.
Example 9.2: The largest of \(n\) uniform observations
Let \(M_n\) be the maximum of \(n\) independent uniform variables on \((0,1)\) . Then \(M_n\to 1\) in probability, and the question is how fast. For \(0\le x\le n\) ,
The limit is the tail of the exponential law with rate \(1\) , whose distribution function is continuous, so by (vi) \(n(1-M_n)\) converges in distribution to an exponential variable. The gap between the maximum and \(1\) is of order \(1/n\) , with an explicit limiting shape. Numerically, at \(x=1\) the probabilities are \(0.9^{10}\approx 0.3487\) for \(n=10\) and \(0.99^{100}\approx 0.3660\) for \(n=100\) , against the limit \(e^{-1}\approx 0.3679\) .
No characteristic function or moment was needed here, because the distribution function of a maximum of independent variables is a product. For sums it is a convolution, and a different tool will be required.
9.2 A common sample space, and what it buys
Convergence in distribution says nothing about the variables as functions. One can always choose variables, with the given laws, that do converge as functions.
Theorem 9.3 (Skorokhod representation on the line). If \(\mu_n\Rightarrow\mu\) , there are random variables \(Y_n\) and \(Y\) on \(((0,1),\mathcal B,\lambda)\) with laws \(\mu_n\) and \(\mu\) such that \(Y_n\to Y\) almost surely.
Proof. Let \(Y_n=F_n^{-1}\) and \(Y=F^{-1}\) be the quantile functions; by Theorem 1.9 they have the required laws under Lebesgue measure. The function \(F^{-1}\) is nondecreasing, so it has at most countably many discontinuities, a set of Lebesgue measure zero. We show \(F_n^{-1}(u)\to F^{-1}(u)\) at every \(u\in(0,1)\) at which \(F^{-1}\) is continuous. Recall from Theorem 1.9 that \(G^{-1}(u)\le x\) if and only if \(u\le G(x)\) , for any distribution function \(G\) .
Lower bound. Let \(\varepsilon>0\) and choose a continuity point \(x\) of \(F\) with \(F^{-1}(u)-\varepsilon<x<F^{-1}(u)\) . Since \(x<F^{-1}(u)\) , we have \(F(x)<u\) . By Theorem 9.2(vi), \(F_n(x)\to F(x)\) , so \(F_n(x)<u\) for large \(n\) , which means \(F_n^{-1}(u)>x>F^{-1}(u)-\varepsilon\) .
Upper bound. Let \(u'\in(u,1)\) and choose a continuity point \(x\) of \(F\) with \(F^{-1}(u')<x<F^{-1}(u')+\varepsilon\) . Then \(F(x)\ge u'>u\) , so \(F_n(x)\ge u\) for large \(n\) , which means \(F_n^{-1}(u)\le x<F^{-1}(u')+\varepsilon\) . Hence \(\limsup_nF_n^{-1}(u)\le F^{-1}(u')\) , and letting \(u'\downarrow u\) , continuity of \(F^{-1}\) at \(u\) gives \(\limsup_nF_n^{-1}(u)\le F^{-1}(u)\) .
The representation converts statements about convergence in distribution into statements about almost sure convergence, to which the theorems of earlier chapters apply.
Corollary 9.4 (continuous mapping and moments). Let \(X_n\to X\) in distribution.
(a) If \(g:\mathbb R\to\mathbb R\) is Borel measurable and continuous outside a Borel set \(D\) with \(\mathbb P(X\in D)=0\) , then \(g(X_n)\to g(X)\) in distribution.
(b) \(\mathbb E\lvert X\rvert\le\liminf_n\mathbb E\lvert X_n\rvert\) .
(c) If \((X_n)\) is uniformly integrable, then \(X\) is integrable and \(\mathbb E X_n\to\mathbb E X\) .
Proof. Take \(Y_n\to Y\) almost surely as in Theorem 9.3. Since \(\lambda(Y\in D)=\mathbb P(X\in D)=0\) , \(g(Y_n)\to g(Y)\) almost surely, and for bounded continuous \(f\) the bounded convergence theorem gives \(\mathbb E f(g(X_n))=\mathbb E f(g(Y_n))\to\mathbb E f(g(Y))=\mathbb E f(g(X))\) ; this is (a). Part (b) is Fatou’s lemma for \(\lvert Y_n\rvert\) . For (c), the \(Y_n\) have the same laws as the \(X_n\) and so are uniformly integrable, and Corollary 5.7 gives \(\mathbb E Y_n\to\mathbb E Y\) .
The second tool combines a sequence converging in distribution with one converging in probability to a constant. It will be used whenever an estimated quantity is substituted for an unknown one.
Theorem 9.5 (Slutsky’s theorem). Let \(X_n\to X\) in distribution and \(Y_n\to c\) in probability, where \(c\) is a constant and \(X_n,Y_n\) are defined on the same space for each \(n\) . Then \(X_n+Y_n\to X+c\) and \(X_nY_n\to cX\) in distribution.
Proof. Sums. By Theorem 9.2(ii) it is enough to test with a function \(f\) bounded by \(M\) and Lipschitz with constant \(L\) . For \(\varepsilon>0\) ,
which is at most \(2L\varepsilon\) for large \(n\) . And \(\mathbb E f(X_n+c)\to\mathbb E f(X+c)\) because \(x\mapsto f(x+c)\) is bounded and continuous.
Products. Write \(X_nY_n=cX_n+X_n(Y_n-c)\) . The first term converges in distribution to \(cX\) by Corollary 9.4(a). The second tends to \(0\) in probability: for \(K>0\) and \(\varepsilon>0\) ,
and by Theorem 9.2(iii) for the closed set \(\{\lvert x\rvert\ge K\}\) , \(\limsup_n\mathbb P(\lvert X_n\rvert\ge K)\le\mathbb P(\lvert X\rvert\ge K)\) , which is small for large \(K\) . Now apply the statement about sums to \(cX_n\) and \(X_n(Y_n-c)\) .
The constant is essential. If \(Y_n\) converges only in distribution to a nonconstant limit, the law of \(X_n+Y_n\) depends on the joint law of the pair, which the marginal convergences do not control (Exercise 16).
9.3 Tightness
When does a sequence of laws have a weakly convergent subsequence? A bounded sequence of real numbers always has a convergent subsequence. For laws there is one way to fail: mass can leave every bounded set.
Definition 9.6 (tightness). A family of probability measures \((\mu_i)_{i\in I}\) on \(\mathbb R\) is tight if for every \(\varepsilon>0\) there is \(K\) with \(\mu_i([-K,K]^{c})\le\varepsilon\) for all \(i\in I\) .
Theorem 9.7 (Helly’s selection theorem). Every sequence of distribution functions \(F_n\) has a subsequence \(F_{n_k}\) such that, for some nondecreasing right-continuous function \(G\) with \(0\le G\le 1\) , \(F_{n_k}(x)\to G(x)\) at every continuity point \(x\) of \(G\) .
Proof. Enumerate the rationals as \(q_1,q_2,\dots\) . The bounded sequence \(F_n(q_1)\) has a convergent subsequence; along it, \(F_n(q_2)\) has a further convergent subsequence; and so on. The diagonal subsequence \(F_{n_k}\) converges at every rational \(q\) , to a limit \(H(q)\in[0,1]\) , and \(H\) is nondecreasing on the rationals. Define
\(G\) is nondecreasing with values in \([0,1]\) . It is right-continuous: given \(x\) and \(\varepsilon\) , pick a rational \(q>x\) with \(H(q)<G(x)+\varepsilon\) ; then \(G(y)\le H(q)<G(x)+\varepsilon\) for all \(y\in[x,q)\) .
Let \(x\) be a continuity point of \(G\) and \(\varepsilon>0\) . Choose \(y<x\) with \(G(y)>G(x)-\varepsilon\) , and rationals \(r,s\) with \(y<r<x<s\) and \(H(s)<G(x)+\varepsilon\) . By definition of \(G\) , \(H(r)\ge G(y)>G(x)-\varepsilon\) . Since \(F_{n_k}(r)\le F_{n_k}(x)\le F_{n_k}(s)\) , letting \(k\to\infty\) gives
The limit \(G\) need not be a distribution function: its limits at \(\pm\infty\) may fall short of \(0\) and \(1\) . Tightness is exactly what prevents that.
Theorem 9.8 (Prokhorov’s theorem on the line). (a) If \((\mu_n)\) is tight, every subsequence has a further subsequence that converges weakly to a probability measure. (b) If \(\mu_n\Rightarrow\mu\) , then \((\mu_n)\) is tight.
Proof. (a) A subsequence of a tight sequence is tight, so it suffices to extract one weakly convergent subsequence. Let \(F_{n_k}\) and \(G\) be as in Helly’s theorem. Given \(\varepsilon\) , let \(K\) be as in the definition of tightness and choose continuity points \(a<-K\) and \(b>K\) of \(G\) . Then \(G(b)-G(a)=\lim_k(F_{n_k}(b)-F_{n_k}(a))\ge 1-\varepsilon\) . As \(\varepsilon\) is arbitrary and \(0\le G\le 1\) , \(G\) tends to \(0\) at \(-\infty\) and to \(1\) at \(+\infty\) . So \(G\) is a distribution function, of a probability measure \(\mu\) by Theorem 1.9, and \(\mu_{n_k}\Rightarrow\mu\) by Theorem 9.2(vi).
(b) Given \(\varepsilon\) , choose \(K\) with \(\pm K\) continuity points of the distribution function of \(\mu\) and \(\mu([-K,K]^{c})<\varepsilon\) . By Theorem 9.2, \(\mu_n([-K,K]^{c})\to\mu([-K,K]^{c})\) , so \(\mu_n([-K,K]^{c})<\varepsilon\) for all \(n\ge n_0\) . Each of the finitely many remaining measures gives mass less than \(\varepsilon\) to the complement of some bounded interval; enlarge \(K\) to cover them.
Corollary 9.9 (subsequence principle). If \((\mu_n)\) is tight and every weakly convergent subsequence of it has the same limit \(\mu\) , then \(\mu_n\Rightarrow\mu\) .
Proof. If not, there are a bounded continuous \(f\) , an \(\varepsilon>0\) , and a subsequence along which \(\lvert\int f\,d\mu_{n_k}-\int f\,d\mu\rvert\ge\varepsilon\) . By Theorem 9.8(a) it has a further subsequence converging weakly, by hypothesis to \(\mu\) , which contradicts the lower bound.
The corollary is the pattern for proving weak convergence: show tightness, then identify every possible limit. The second step needs a way to recognise a law, and that is what characteristic functions provide.
Example 9.3: Mass that escapes
Let \(\mu_n\) be the normal law with mean \(0\) and variance \(n\) , whose distribution function is \(F_n(x)=\Phi(x/\sqrt n)\) with \(\Phi\) the standard normal distribution function. For every fixed \(x\) , \(F_n(x)\to\Phi(0)=\tfrac12\) . The limit \(G\equiv\tfrac12\) is nondecreasing and right-continuous, as Helly’s theorem promises, and it is not a distribution function: half the mass has gone to \(-\infty\) and half to \(+\infty\) . The sequence is not tight, since \(\mu_n([-K,K])=2\Phi(K/\sqrt n)-1\to 0\) for every \(K\) , and it has no weakly convergent subsequence.
Partial escape is also possible. For \(\nu_n=\tfrac12\delta_0+\tfrac12\delta_n\) the distribution functions converge to \(\tfrac12\mathbf 1_{\{x\ge 0\}}\) : half the mass stays at the origin and half is lost. In both cases \(\int f\,d\mu_n\) does converge for continuous \(f\) vanishing at infinity; what fails is convergence for the constant function \(1\) against a limit of total mass less than one.
9.4 Characteristic functions
Definition 9.10 (characteristic function). The characteristic function of a random variable \(X\) , or of its law \(\mu\) , is
The integrand is bounded, so the characteristic function exists for every law, with no moment condition; that is its advantage over the moment generating function of a first course. Expectations of complex-valued variables are taken componentwise, and the inequality \(\lvert\mathbb E Z\rvert\le\mathbb E\lvert Z\rvert\) remains true.
Proposition 9.11 (elementary properties). Let \(\varphi\) be the characteristic function of \(X\) .
(a) \(\varphi(0)=1\) , \(\lvert\varphi(t)\rvert\le 1\) , and \(\varphi(-t)=\overline{\varphi(t)}\) .
(b) \(\varphi\) is uniformly continuous on \(\mathbb R\) .
(c) For real \(a,b\) , \(\varphi_{aX+b}(t)=e^{itb}\varphi_X(at)\) .
(d) If \(X\) and \(Y\) are independent, then \(\varphi_{X+Y}(t)=\varphi_X(t)\,\varphi_Y(t)\) .
Proof. (a) and (c) are immediate from the definition. (b) \(\lvert\varphi(t+h)-\varphi(t)\rvert=\lvert\mathbb E[e^{itX}(e^{ihX}-1)]\rvert\le\mathbb E\lvert e^{ihX}-1\rvert\) , which does not depend on \(t\) and tends to \(0\) as \(h\to 0\) by dominated convergence. (d) \(e^{it(X+Y)}=e^{itX}e^{itY}\) , and Theorem 2.6, applied to the real and imaginary parts, gives \(\mathbb E[e^{itX}e^{itY}]=\mathbb E e^{itX}\,\mathbb E e^{itY}\) .
Property (d) is the reason for the whole construction: the law of a sum of independent variables is a convolution, which is hard to compute and harder to take limits of, and its characteristic function is a product.
Example 9.4: Five characteristic functions
Bernoulli and binomial. If \(\mathbb P(X=1)=p=1-\mathbb P(X=0)\) , then \(\varphi(t)=1-p+pe^{it}\) . A binomial variable is a sum of \(n\) independent such variables, so its characteristic function is \((1-p+pe^{it})^n\) .
Poisson. For mean \(\lambda\) , \(\varphi(t)=\sum_ke^{itk}e^{-\lambda}\lambda^k/k!=\exp\bigl(\lambda(e^{it}-1)\bigr)\) .
Uniform on \((-1,1)\) . \(\varphi(t)=\tfrac12\int_{-1}^{1}\cos(tx)\,dx=\dfrac{\sin t}{t}\) , the sine part vanishing by symmetry.
Exponential with rate \(1\) . \(\varphi(t)=\int_0^\infty e^{itx}e^{-x}\,dx=\dfrac{1}{1-it}=\dfrac{1+it}{1+t^2}\) .
Standard normal. By symmetry \(\varphi(t)=\int\cos(tx)\,\tfrac{1}{\sqrt{2\pi}}e^{-x^2/2}\,dx\) . Differentiating under the integral sign, which dominated convergence permits because \(\lvert x\sin(tx)\rvert\le\lvert x\rvert\) is integrable against the normal density, and then integrating by parts,
With \(\varphi(0)=1\) this differential equation has the unique solution \(\varphi(t)=e^{-t^2/2}\) . By property (c), the normal law with mean \(m\) and variance \(s^2\) has characteristic function \(e^{itm-s^2t^2/2}\) , and by (d) a sum of independent normal variables has a characteristic function of the same form, with the means and the variances added.
The last remark would show that sums of independent normal variables are normal, if a law could be recovered from its characteristic function. It can.
Theorem 9.12 (inversion formula). Let \(\mu\) be a probability measure on \(\mathbb R\) with characteristic function \(\varphi\) . For all real \(a<b\) ,
Proof. The function \(t\mapsto(e^{-ita}-e^{-itb})/(it)=\int_a^be^{-itx}\,dx\) is bounded by \(b-a\) , so the integrand in
is bounded on \([-T,T]\times\mathbb R\) and Fubini’s theorem allows the order of integration to be exchanged. For fixed \(x\) split the numerator into cosines and sines. The function \(t\mapsto\bigl(\cos t(x-a)-\cos t(x-b)\bigr)/(it)\) is bounded and odd, so it integrates to zero over \([-T,T]\) ; the function \(t\mapsto\bigl(\sin t(x-a)-\sin t(x-b)\bigr)/t\) is even. Since \(\int_0^T\frac{\sin(t\theta)}{t}\,dt=\operatorname{sgn}(\theta)\,S(T\lvert\theta\rvert)\) for real \(\theta\) , where \(S(y)=\int_0^y\frac{\sin t}{t}\,dt\) , the inner integral over \(t\) equals \(2\operatorname{sgn}(x-a)S(T\lvert x-a\rvert)-2\operatorname{sgn}(x-b)S(T\lvert x-b\rvert)\) . Therefore
The function \(S\) is continuous with \(S(y)\to\pi/2\) as \(y\to\infty\) , and hence bounded. The limit is the value of the Dirichlet integral \(\int_0^\infty\frac{\sin t}{t}\,dt\) , an improper Riemann integral that we quote from Real Analysis without proof. As \(T\to\infty\) the bracket tends to \(0\) if \(x<a\) or \(x>b\) , to \(\pi\) if \(a<x<b\) , and to \(\pi/2\) if \(x=a\) or \(x=b\) . The integrand is bounded uniformly in \(T\) , so by the bounded convergence theorem \(I_T\) tends to the right-hand side of the formula.
Corollary 9.13 (uniqueness). Two probability measures on \(\mathbb R\) with the same characteristic function are equal.
Proof. Let \(\mu\) and \(\nu\) have the same characteristic function. The points that are atoms of \(\mu\) or of \(\nu\) form a countable set \(D\) . For \(a<b\) outside \(D\) , the inversion formula gives \(\mu((a,b))=\nu((a,b))\) . The open intervals with endpoints outside \(D\) form a \(\pi\) -system, and they generate the Borel sigma-algebra because the complement of \(D\) is dense. By uniqueness on a \(\pi\) -system, \(\mu=\nu\) .
Corollary 9.14 (integrable characteristic functions). If \(\int\lvert\varphi(t)\rvert\,dt<\infty\) , then \(\mu\) has the bounded continuous density
Proof. With \(\varphi\) integrable, the limit in the inversion formula is an absolutely convergent integral, bounded in modulus by \((b-a)\int\lvert\varphi\rvert\,dt/(2\pi)\) . Letting \(b\downarrow a\) shows that \(\mu\) has no atoms. Then, writing \((e^{-ita}-e^{-itb})/(it)=\int_a^be^{-itx}\,dx\) and exchanging the integrals by Fubini’s theorem, \(\mu((a,b))=\int_a^bf(x)\,dx\) for all \(a<b\) . Continuity and boundedness of \(f\) follow from dominated convergence.
Example 9.5: The Cauchy characteristic function by duality
The two-sided exponential density \(\tfrac12e^{-\lvert x\rvert}\) has the characteristic function
the real part of the exponential characteristic function of Example 9.4. This function of \(t\) is integrable, so Corollary 9.14 applies and returns the density:
Now read the identity with the roles of the two variables exchanged. Multiplying by \(2\) and using the symmetry of the integrand, it says \(\int e^{itx}\,\frac{1}{\pi(1+x^2)}\,dx=e^{-\lvert t\rvert}\) . The left side is the characteristic function of the Cauchy density. So
This function is not differentiable at \(t=0\) , which reflects the absence of a mean (Exercise 14 shows that an integrable variable has a differentiable characteristic function). By Proposition 9.11(c) the Cauchy law with scale \(a\) has characteristic function \(e^{-a\lvert t\rvert}\) , and by (d) the sum of independent Cauchy variables with scales \(a\) and \(b\) has characteristic function \(e^{-(a+b)\lvert t\rvert}\) ; by Corollary 9.13 it is Cauchy with scale \(a+b\) . In particular the average of \(n\) independent standard Cauchy variables has characteristic function \((e^{-\lvert t\rvert/n})^n=e^{-\lvert t\rvert}\) , which is the claim of Example 8.3.
9.5 The continuity theorem
Weak convergence implies pointwise convergence of characteristic functions, simply because \(\cos(tx)\) and \(\sin(tx)\) are bounded continuous functions of \(x\) . The converse is the deep direction. Its proof follows the pattern of Corollary 9.9, and the tightness comes from an inequality that reads the tails of a law off the behaviour of its characteristic function near the origin.
Lemma 9.15 (tails from the characteristic function). For a probability measure \(\mu\) with characteristic function \(\varphi\) and any \(u>0\) ,
Proof. The right side is real, because the imaginary part of \(\varphi\) is odd. For \(x\ne 0\) , \(\frac1u\int_{-u}^{u}(1-e^{itx})\,dt=2\bigl(1-\frac{\sin(ux)}{ux}\bigr)\) , and for \(x=0\) the integral is \(0\) . By Fubini’s theorem,
The integrand is nonnegative everywhere, since \(\lvert\sin y\rvert\le\lvert y\rvert\) . Where \(\lvert ux\rvert\ge 2\) it is at least \(1-\frac{1}{\lvert ux\rvert}\ge\frac12\) . So the right side is at least \(2\cdot\tfrac12\,\mu(\lvert x\rvert\ge 2/u)\) .
One more elementary fact is needed, here and in Chapter 10, to take limits of \(n\) -th powers.
Lemma 9.16 (complex exponential limit). If \(c_n\) are complex numbers with \(c_n\to c\) , then \(\bigl(1+\tfrac{c_n}{n}\bigr)^n\to e^{c}\) .
Proof. For complex \(z,w\) of modulus at most \(\rho\) , \(\lvert z^n-w^n\rvert=\lvert z-w\rvert\,\lvert\sum_{j<n}z^{j}w^{n-1-j}\rvert\le n\rho^{n-1}\lvert z-w\rvert\) . Take \(z=1+c_n/n\) and \(w=e^{c_n/n}\) ; both have modulus at most \(\rho=e^{\lvert c_n\rvert/n}\) . From the power series, \(\lvert e^{\zeta}-1-\zeta\rvert\le\lvert\zeta\rvert^2\) when \(\lvert\zeta\rvert\le 1\) , so for large \(n\) , \(\lvert z-w\rvert\le\lvert c_n\rvert^2/n^2\) . Hence \(\lvert(1+c_n/n)^n-e^{c_n}\rvert\le n\,e^{\lvert c_n\rvert}\,\lvert c_n\rvert^2/n^2\to 0\) , and \(e^{c_n}\to e^{c}\) .
Theorem 9.17 (Lévy’s continuity theorem). Let \(\mu_n\) be probability measures on \(\mathbb R\) with characteristic functions \(\varphi_n\) .
(a) If \(\mu_n\Rightarrow\mu\) , then \(\varphi_n(t)\to\varphi(t)\) for every \(t\) , where \(\varphi\) is the characteristic function of \(\mu\) .
(b) If \(\varphi_n(t)\) converges for every \(t\) to a limit \(\varphi(t)\) , and \(\varphi\) is continuous at \(0\) , then \(\varphi\) is the characteristic function of a probability measure \(\mu\) , and \(\mu_n\Rightarrow\mu\) .
Proof. (a) Apply the definition of weak convergence to the bounded continuous functions \(x\mapsto\cos(tx)\) and \(x\mapsto\sin(tx)\) .
(b) Tightness. Let \(\varepsilon>0\) . Since \(\varphi(0)=\lim\varphi_n(0)=1\) and \(\varphi\) is continuous at \(0\) , there is \(u>0\) with \(\lvert 1-\varphi(t)\rvert\le\varepsilon/4\) for \(\lvert t\rvert\le u\) , so that \(\frac1u\int_{-u}^{u}\lvert 1-\varphi(t)\rvert\,dt\le\varepsilon/2\) . The functions \(1-\varphi_n\) are bounded by \(2\) and converge pointwise to \(1-\varphi\) , so by dominated convergence and Lemma 9.15,
So \(\mu_n(\lvert x\rvert\ge 2/u)\le\varepsilon\) for all \(n\ge n_0\) , and enlarging the interval to accommodate the first \(n_0\) measures shows that \((\mu_n)\) is tight.
Identification of the limit. By Theorem 9.8(a) some subsequence converges weakly to a probability measure \(\mu\) , and by (a) the characteristic function of \(\mu\) is \(\lim_k\varphi_{n_k}=\varphi\) . So \(\varphi\) is a characteristic function. If another subsequence converges weakly to \(\nu\) , the characteristic function of \(\nu\) is again \(\varphi\) , and \(\nu=\mu\) by Corollary 9.13. Corollary 9.9 gives \(\mu_n\Rightarrow\mu\) .
Continuity at the origin is the hypothesis that carries the tightness, and it cannot be omitted.
Example 9.6: A Poisson limit, and a limit that is no characteristic function
Rare events. Let \(X_n\) be binomial with parameters \(n\) and \(\lambda/n\) . By Example 9.4 and Lemma 9.16 with the constant sequence \(c_n=\lambda(e^{it}-1)\) ,
the characteristic function of the Poisson law with mean \(\lambda\) , which is continuous. By Theorem 9.17, \(X_n\) converges in distribution to that law; and for integer-valued variables this means convergence of each point probability, by Theorem 9.2(v) applied to the intervals \((k-\tfrac12,k+\tfrac12)\) . With \(\lambda=2\) and \(n=100\) : \(\mathbb P(X_n=0)=0.98^{100}\approx 0.1326\) against \(e^{-2}\approx 0.1353\) , and \(\mathbb P(X_n=2)\approx 0.2734\) against \(2e^{-2}\approx 0.2707\) .
Escaping mass, seen through the transform. For the normal laws with variance \(n\) of Example 9.3, \(\varphi_n(t)=e^{-nt^2/2}\) , which converges for every \(t\) : to \(1\) at \(t=0\) and to \(0\) elsewhere. The limit exists everywhere and is discontinuous at the origin. It is not a characteristic function, since those are continuous, and the laws do not converge. The flattening of the laws appears in the transform as the collapse of \(\varphi_n\) away from the origin, and the connection can be made exact. Lemma 9.15 bounds the tails from above only, so it cannot be quoted here; but the identity in its proof also gives a bound in the other direction. Since \(0\le 1-\frac{\sin y}{y}\le\min(y^2/6,\,2)\) for real \(y\) , splitting the integral at \(\lvert x\rvert=K\) gives, for every law \(\mu\) and all \(u,K>0\) ,
So a characteristic function whose average distance from \(1\) over \([-u,u]\) is large for small \(u\) belongs to a law with substantial mass beyond every \(K\) of smaller order than \(1/u\) . For the normal laws above the left side tends to \(2\) for each fixed \(u\) , and the inequality with \(u=1/K\) returns \(\liminf_n\mu_n(\lvert x\rvert>K)\ge\tfrac5{12}\) for every \(K\) .
Common mistake. Asking weak convergence for more than it gives, or accepting a limit of transforms without checking that it is one. Convergence in distribution does not mean \(F_n(x)\to F(x)\) at every \(x\) ; it can fail at the jumps of \(F\) , as for the constants \(1/n\) . It does not give \(\mathbb P(X_n\in A)\to\mathbb P(X\in A)\) for every Borel set \(A\) , only for sets whose boundary the limit does not charge; nor convergence of densities; nor convergence of moments, which needs uniform integrability (Corollary 9.4). On the transform side, pointwise convergence of \(\varphi_n\) proves convergence in distribution only when the limit is continuous at \(0\) ; the limit \(\mathbf 1_{\{t=0\}}\) of Example 9.6 is the standard warning. When using Theorem 9.17, state which characteristic function the limit is, or verify its continuity at the origin.
Everything is now in place for the central limit theorem. If \(X_1,X_2,\dots\) are independent with a common law of mean \(0\) and variance \(1\) and characteristic function \(\varphi\) , then \(S_n/\sqrt n\) has characteristic function \(\varphi(t/\sqrt n)^n\) by Proposition 9.11. What does the behaviour of \(\varphi\) near the origin say about the limit of that expression?
Exercises
-
Show that \(\delta_{x_n}\Rightarrow\delta_x\) if and only if \(x_n\to x\) , and describe the behaviour of the distribution functions at the point \(x\) when \(x_n\) decreases to \(x\) and when it increases to \(x\) .
Answer
If \(x_n\to x\) then \(f(x_n)\to f(x)\) for continuous \(f\) . Conversely, test with \(f(y)=\min(\lvert y-x\rvert,1)\) : \(f(x_n)\to f(x)=0\) forces \(x_n\to x\) . If \(x_n\downarrow x\) with \(x_n>x\) , then \(F_n(x)=0\) for all \(n\) while \(F(x)=1\) ; if \(x_n\uparrow x\) , then \(F_n(x)=1=F(x)\) . Convergence fails only at the discontinuity point \(x\) of \(F\) , and only from one side.
-
Prove that a sequence of probability measures on \(\mathbb R\) has at most one weak limit.
Hint
If \(\mu_n\Rightarrow\mu\) and \(\mu_n\Rightarrow\nu\) , then \(\int f\,d\mu=\int f\,d\nu\) for bounded continuous \(f\) . Approximate \(\mathbf 1_{(-\infty,x]}\) from above by continuous functions as in Exercise 2.13 to get equal distribution functions.
-
Let \(X_n\) have density \(1+\cos(2\pi nx)\) on \((0,1)\) . Compute its distribution function, prove that \(X_n\) converges in distribution to the uniform law, and compute \(\int_0^1\lvert f_n(x)-1\rvert\,dx\) to show that the densities do not converge in \(L^1\) .
Solution
The function \(f_n(x)=1+\cos(2\pi nx)\) is nonnegative and integrates to \(1\) over \((0,1)\) , so it is a density. For \(0\le x\le 1\) ,
\[ F_n(x)=\int_0^x\bigl(1+\cos(2\pi ny)\bigr)\,dy=x+\frac{\sin(2\pi nx)}{2\pi n}, \]so \(\lvert F_n(x)-x\rvert\le\tfrac{1}{2\pi n}\) for all \(x\) . The uniform distribution function \(F(x)=x\) is continuous everywhere, and \(F_n\to F\) at every point, indeed uniformly; by Theorem 9.2(vi), \(X_n\) converges in distribution to the uniform law.
The densities do not converge. At a typical point \(f_n(x)\) oscillates between \(0\) and \(2\) , and in \(L^1\) ,
\[ \int_0^1\lvert f_n(x)-1\rvert\,dx=\int_0^1\lvert\cos(2\pi nx)\rvert\,dx=\frac2\pi\qquad\text{for every }n, \]because the integral of \(\lvert\cos\rvert\) over each of its \(n\) periods of length \(1/n\) is \(\tfrac{2}{\pi n}\) . So weak convergence holds while the distance in total variation stays at the constant \(\tfrac2\pi\) . The oscillations are invisible to continuous test functions, which average over them, and fully visible to the set \(\{x:\cos(2\pi nx)>0\}\) , whose probability under \(f_n\) is \(\tfrac12+\tfrac1\pi\) against \(\tfrac12\) under the uniform law. That set changes with \(n\) , which is why Theorem 9.2(v), about a fixed set, is not contradicted.
-
Let \(X_n\) have densities \(f_n\) with \(f_n\to f\) almost everywhere, where \(f\) is a probability density. Use Exercise 5.10 to prove that \(X_n\) converges in distribution to the law with density \(f\) , and that in fact \(\mathbb P(X_n\in A)\to\int_Af\,dx\) uniformly over Borel sets \(A\) .
Hint
By Exercise 5.10, \(\int\lvert f_n-f\rvert\,dx\to 0\) , and \(\lvert\mathbb P(X_n\in A)-\int_Af\,dx\rvert\le\int\lvert f_n-f\rvert\,dx\) for every Borel \(A\) ; for bounded continuous \(g\) , \(\lvert\int g(f_n-f)\,dx\rvert\le\sup\lvert g\rvert\int\lvert f_n-f\rvert\,dx\) .
-
Let \(X_n\) and \(X\) take values in the integers. Prove that \(X_n\to X\) in distribution if and only if \(\mathbb P(X_n=k)\to\mathbb P(X=k)\) for every integer \(k\) .
Solution
Write \(p_n(k)=\mathbb P(X_n=k)\) and \(p(k)=\mathbb P(X=k)\) .
Necessity. Fix an integer \(k\) and let \(A=(k-\tfrac12,k+\tfrac12)\) . Its boundary \(\{k\pm\tfrac12\}\) contains no integer, so \(\mathbb P(X\in\partial A)=0\) , and Theorem 9.2(v) gives \(p_n(k)=\mathbb P(X_n\in A)\to\mathbb P(X\in A)=p(k)\) .
Sufficiency. Let \(f\) be continuous with \(\lvert f\rvert\le M\) , and \(\varepsilon>0\) . Choose a finite set \(K\) of integers with \(\sum_{k\in K}p(k)>1-\varepsilon\) . Since \(K\) is finite, \(\sum_{k\in K}p_n(k)\to\sum_{k\in K}p(k)\) , so \(\mathbb P(X_n\notin K)<2\varepsilon\) for large \(n\) . Then
\[ \lvert\mathbb E f(X_n)-\mathbb E f(X)\rvert\le\sum_{k\in K}\lvert f(k)\rvert\,\lvert p_n(k)-p(k)\rvert+M\,\mathbb P(X_n\notin K)+M\,\mathbb P(X\notin K). \]The finite sum tends to \(0\) , and the other two terms are at most \(3M\varepsilon\) for large \(n\) . Hence \(\mathbb E f(X_n)\to\mathbb E f(X)\) . The finite set \(K\) supplies tightness, which convergence of the individual point probabilities to a probability law gives automatically: no mass can escape, because the limit already accounts for all of it.
-
Prove directly from Definition 9.1, without the Skorokhod representation, that \(X_n\to X\) in distribution implies \(\mathbb E\lvert X\rvert\le\liminf_n\mathbb E\lvert X_n\rvert\) .
Solution
For \(M>0\) the function \(f_M(x)=\lvert x\rvert\wedge M\) is bounded and continuous, so by Definition 9.1,
\[ \mathbb E\bigl[\lvert X\rvert\wedge M\bigr]=\lim_n\mathbb E\bigl[\lvert X_n\rvert\wedge M\bigr]\le\liminf_n\mathbb E\lvert X_n\rvert , \]since \(\lvert X_n\rvert\wedge M\le\lvert X_n\rvert\) . The right side does not depend on \(M\) . Let \(M\uparrow\infty\) : \(\lvert X\rvert\wedge M\uparrow\lvert X\rvert\) , and the Monotone Convergence Theorem gives \(\mathbb E\lvert X\rvert\le\liminf_n\mathbb E\lvert X_n\rvert\) . The argument is the one behind Fatou’s lemma: truncate to make the functional continuous for the mode of convergence at hand, pass to the limit, and remove the truncation by monotone convergence. Nothing better than an inequality can be expected, as the laws \((1-\tfrac1n)\delta_0+\tfrac1n\delta_n\) show: they converge weakly to \(\delta_0\) and all have mean \(1\) .
-
Show that the normal laws with means \(m_n\) and variances \(s_n^2\) form a tight sequence if and only if both sequences are bounded, and that they converge weakly if and only if \(m_n\) and \(s_n^2\) converge, the limit being a point mass when \(s_n^2\to 0\) .
Hint
Use characteristic functions \(e^{itm_n-s_n^2t^2/2}\) and Theorem 9.17 for the convergence statement; for tightness, if \(s_n^2\) or \(\lvert m_n\rvert\) is unbounded, mass leaves every interval along a subsequence as in Example 9.3.
-
Prove that \(\sup_n\mathbb E\lvert X_n\rvert^p<\infty\) for some \(p>0\) implies that the laws of the \(X_n\) are tight, and give a tight sequence for which no such \(p\) exists.
Answer
By Markov’s inequality \(\mathbb P(\lvert X_n\rvert>K)\le K^{-p}\sup_n\mathbb E\lvert X_n\rvert^p\) , which is small for large \(K\) uniformly in \(n\) . Conversely a single law is always tight, and the law with tail \(\mathbb P(X>x)=1/\log x\) for \(x\ge e\) has \(\mathbb E X^p=\infty\) for every \(p>0\) .
-
A student argues: “The characteristic functions \(e^{-nt^2/2}\) converge for every \(t\) , so the normal laws with variance \(n\) converge in distribution.” Name the hypothesis of the continuity theorem that fails, prove directly that the laws have no weak limit, and say where the mass goes.
Solution
The pointwise limit of \(\varphi_n(t)=e^{-nt^2/2}\) is \(\varphi(t)=1\) for \(t=0\) and \(\varphi(t)=0\) for \(t\ne 0\) . Part (b) of Theorem 9.17 requires the limit to be continuous at \(0\) , and this \(\varphi\) is not. The hypothesis is not a technicality here: if the laws \(\mu_n\) converged weakly to some \(\mu\) , then by part (a) the functions \(\varphi_n\) would converge pointwise to the characteristic function of \(\mu\) , which is continuous by Proposition 9.11(b); the actual pointwise limit is discontinuous, so no weak limit exists. The same argument applies to every subsequence.
Directly: for every \(K\) , \(\mu_n([-K,K])=2\Phi(K/\sqrt n)-1\to 0\) . If \(\mu_n\Rightarrow\mu\) , choose \(K\) with \(\mu((-K,K))>\tfrac12\) ; by Theorem 9.2(iv), \(\liminf_n\mu_n((-K,K))\ge\mu((-K,K))>\tfrac12\) , a contradiction. The mass goes to infinity, half in each direction: \(\mu_n((K,\infty))\to\tfrac12\) and \(\mu_n((-\infty,-K))\to\tfrac12\) for every \(K\) . In the language of Lemma 9.15, \(\frac1u\int_{-u}^{u}(1-\varphi_n(t))\,dt\to 2\) for every \(u>0\) , the largest value possible.
-
Find the characteristic function of the uniform law on \((-\tfrac12,\tfrac12)\) and of the sum of two independent such variables, and verify the second by integrating \(\cos(tx)\) against the triangular density \((1-\lvert x\rvert)^{+}\) .
Solution
For \(X\) uniform on \((-\tfrac12,\tfrac12)\) the sine part vanishes by symmetry and
\[ \varphi_X(t)=\int_{-1/2}^{1/2}\cos(tx)\,dx=\frac{2\sin(t/2)}{t}=\frac{\sin(t/2)}{t/2}. \]For the sum of two independent copies, Proposition 9.11(d) gives
\[ \varphi_{X+Y}(t)=\Bigl(\frac{\sin(t/2)}{t/2}\Bigr)^2=\frac{4\sin^2(t/2)}{t^2}=\frac{2(1-\cos t)}{t^2}. \]By the convolution computation of Example 2.4, shifted to be centred at \(0\) , the sum has the triangular density \((1-\lvert x\rvert)^{+}\) on \((-1,1)\) . Integrating directly, with two integrations by parts,
\[ \int_{-1}^{1}(1-\lvert x\rvert)\cos(tx)\,dx=2\int_0^1(1-x)\cos(tx)\,dx=2\Bigl[\frac{(1-x)\sin(tx)}{t}\Bigr]_0^1+\frac2t\int_0^1\sin(tx)\,dx=\frac{2(1-\cos t)}{t^2}. \]The two computations agree, as they must by Proposition 9.11(d). The characteristic function is nonnegative and integrable, decaying like \(t^{-2}\) , in keeping with Corollary 9.14: the triangular density is continuous, while the uniform density, which is not, has the non-integrable characteristic function \(\sin(t/2)/(t/2)\) .
-
Prove that a characteristic function is real-valued if and only if the law is symmetric, that is, \(X\) and \(-X\) have the same law.
Hint
\(\varphi_{-X}(t)=\overline{\varphi_X(t)}\) . If \(\varphi_X\) is real then \(\varphi_X=\varphi_{-X}\) and Corollary 9.13 applies; the converse is immediate.
-
Suppose \(\varphi_X(t_0)=1\) for some \(t_0\ne 0\) . Prove that \(X\) takes values in the lattice \(\{2\pi k/t_0:k\in\mathbb Z\}\) almost surely.
Hint
Taking real parts, \(\mathbb E[1-\cos(t_0X)]=0\) with a nonnegative integrand, so \(\cos(t_0X)=1\) almost surely.
-
Assuming that \((1-\lvert t\rvert)^{+}\) is a characteristic function, use Corollary 9.14 to find the corresponding density, and explain how Exercise 10 and the duality argument of Example 9.5 justify the assumption.
Answer
\(f(x)=\frac{1}{2\pi}\int_{-1}^{1}(1-\lvert t\rvert)\cos(tx)\,dt=\frac{1-\cos x}{\pi x^2}\) , by the integral of Exercise 10. That exercise shows the triangular density has the integrable characteristic function \(2(1-\cos t)/t^2\) ; applying Corollary 9.14 to it and exchanging the roles of the variables, as in Example 9.5, shows that \((1-\cos x)/(\pi x^2)\) is a probability density with characteristic function \((1-\lvert t\rvert)^{+}\) .
-
Let \(\mathbb E\lvert X\rvert<\infty\) . Prove that \(\varphi_X\) is continuously differentiable with \(\varphi_X'(t)=\mathbb E[iXe^{itX}]\) , so that \(\varphi_X'(0)=i\,\mathbb E X\) .
Solution
Fix \(t\) and \(h\ne 0\) . Then
\[ \frac{\varphi_X(t+h)-\varphi_X(t)}{h}=\mathbb E\Bigl[e^{itX}\,\frac{e^{ihX}-1}{h}\Bigr]. \]For real \(y\) , \(\lvert e^{iy}-1\rvert=\lvert\int_0^yie^{is}\,ds\rvert\le\lvert y\rvert\) , so the integrand is bounded in modulus by \(\lvert X\rvert\) , which is integrable by hypothesis, uniformly in \(h\) . As \(h\to 0\) the integrand converges pointwise to \(iXe^{itX}\) . The Dominated Convergence Theorem, applied to the real and imaginary parts, gives
\[ \varphi_X'(t)=\mathbb E\bigl[iXe^{itX}\bigr]. \]The right side is continuous in \(t\) , again by dominated convergence with the same dominating variable \(\lvert X\rvert\) , so \(\varphi_X\) is continuously differentiable, and at \(t=0\) the formula reads \(\varphi_X'(0)=i\,\mathbb E X\) . The same argument, repeated, shows that \(\mathbb E\lvert X\rvert^k<\infty\) makes \(\varphi_X\) \(k\) times continuously differentiable with \(\varphi_X^{(k)}(0)=i^k\,\mathbb E X^k\) . The Cauchy characteristic function \(e^{-\lvert t\rvert}\) has a corner at \(0\) , consistent with the absence of a first moment.
-
Use Exercise 14, Lemma 9.16, and the continuity theorem to give a proof by characteristic functions that \(S_n/n\to\mu\) in probability for independent identically distributed integrable variables with mean \(\mu\) , and check the limit explicitly for exponential variables.
Solution
Let \(\varphi\) be the common characteristic function. By Proposition 9.11(c) and (d), \(S_n/n\) has characteristic function \(\varphi(t/n)^n\) . Fix \(t\) . By Exercise 14, \(\varphi\) is differentiable at \(0\) with \(\varphi'(0)=i\mu\) , so
\[ c_n:=n\bigl(\varphi(t/n)-1\bigr)=t\cdot\frac{\varphi(t/n)-\varphi(0)}{t/n}\longrightarrow t\,\varphi'(0)=i\mu t . \]Then \(\varphi(t/n)^n=(1+c_n/n)^n\to e^{i\mu t}\) by Lemma 9.16. The limit \(e^{i\mu t}\) is the characteristic function of the point mass at \(\mu\) and is continuous, so Theorem 9.17(b) gives \(S_n/n\to\mu\) in distribution, and Theorem 4.7, convergence in distribution to a constant, upgrades this to convergence in probability.
For exponential variables with rate \(1\) , \(\varphi(t)=1/(1-it)\) and
\[ \varphi(t/n)^n=\Bigl(1-\frac{it}{n}\Bigr)^{-n}\longrightarrow e^{it}, \]the characteristic function of the constant \(\mu=1\) . This is a third proof of the weak law under a first moment, after truncation (Theorem 8.3) and the strong law. It uses full independence, through the product formula, and gives no almost sure statement.
-
Show by example that \(X_n\to X\) and \(Y_n\to Y\) in distribution do not imply \(X_n+Y_n\to X+Y\) in distribution, even when all the variables are defined on one space, and say what Slutsky’s theorem requires in addition.
Answer
Let \(Z\) be standard normal, \(X_n=Z\) , and \(Y_n=-Z\) for all \(n\) . Then \(X_n\to Z\) and \(Y_n\to Z\) in distribution, since \(-Z\) has the law of \(Z\) , but \(X_n+Y_n=0\) , which is not the law of a sum of two independent normals, nor of \(2Z\) . Slutsky’s theorem requires one of the two limits to be a constant.
-
Evaluate both sides of Lemma 9.15 for the standard normal law with \(u=1\) .
Answer
Right side: \(\int_{-1}^{1}(1-e^{-t^2/2})\,dt=2-\sqrt{2\pi}\,\bigl(2\Phi(1)-1\bigr)\approx 0.2888\) . Left side: \(\mathbb P(\lvert X\rvert\ge 2)\approx 0.0455\) . The inequality holds with room to spare.
-
Let \(X_n\to X\) in distribution and let \(a_n\to a\) and \(b_n\to b\) be real sequences. Prove that \(a_nX_n+b_n\to aX+b\) in distribution.
Hint
Write \(a_nX_n+b_n=aX_n+b+(a_n-a)X_n+(b_n-b)\) . The first two terms converge in distribution to \(aX+b\) by Corollary 9.4(a); the remainder tends to \(0\) in probability, using tightness of \((X_n)\) as in the proof of Theorem 9.5; conclude with Slutsky’s theorem.