Which straight line \(a+bx\) best approximates \(e^x\) on \([0,1]\) ? The answer depends on how error is measured. If it is the largest deviation, the problem lives in \(C[0,1]\) with the supremum norm, and Chapter 3 showed how such problems are solved: guess an approximant, then certify it with a cleverly chosen functional. If instead the error is the mean square \(\int_0^1\lvert e^x-a-bx\rvert^2\,dx\) , the problem can be solved by a formula. One computes two integrals, and the best line is \((4e-10)+(18-6e)x\) . No guessing and no certificate are required.

The difference is that the mean-square norm comes from an inner product, and an inner product brings with it the notion of perpendicularity. The best approximation from a subspace is the foot of the perpendicular, exactly as in Euclidean geometry, and perpendicularity is a linear condition that can be solved. This chapter develops that geometry in infinite dimensions. Its central result, the projection theorem, gives every closed convex set a unique nearest point and every closed subspace a complement, two things that Chapters 5 and 6 showed to be false or delicate in general Banach spaces. Its proof needs neither the Hahn–Banach theorem nor weak compactness, only one identity satisfied by the norm.

7.1 Inner products and the parallelogram law

The first two conditions make the inner product conjugate-linear in its second argument: \(\langle x,\alpha y\rangle=\overline\alpha\langle x,y\rangle\) . The standard examples are \(\mathbb K^n\) and \(\ell^2\) with \(\langle x,y\rangle=\sum_kx_k\overline{y_k}\) , and \(L^2\) of a measure space with \(\langle f,g\rangle=\int f\,\overline g\) ; these are complete by Theorem 1.3 and by the completeness of \(L^p\) quoted there. The same integral makes \(C[0,1]\) an inner product space, but not a Hilbert space, since the norm is then \(\lVert\cdot\rVert_2\) and the space is incomplete by the argument of Example 1.2. That \(\lVert\cdot\rVert\) is in fact a norm follows from the first theorem.

The inequality shows that the inner product is continuous in both variables: if \(x_n\to x\) and \(y_n\to y\) then \(\langle x_n,y_n\rangle\to\langle x,y\rangle\) . The choice of \(t\) in the proof is not a trick. The vector \(ty\) is the point of the line through \(y\) nearest to \(x\) , and the inequality says that the distance from \(x\) to that line is real.

Expanding \(\lVert x\pm y\rVert^2\) and adding gives the parallelogram law

the sum of the squares of the diagonals of a parallelogram equals the sum of the squares of its sides. Subtracting instead recovers the inner product from the norm, the polarization identity: \(\langle x,y\rangle=\tfrac14(\lVert x+y\rVert^2-\lVert x-y\rVert^2)\) in a real space, with the additional term \(\tfrac i4(\lVert x+iy\rVert^2-\lVert x-iy\rVert^2)\) in a complex one. A theorem of Jordan and von Neumann, which we shall not need, states that conversely a norm satisfying the parallelogram law arises from an inner product. So the law is a test for whether a given Banach space is secretly a Hilbert space.

7.2 The nearest point of a closed convex set

The inequality \(\operatorname{Re}\langle x-z,k-z\rangle\le 0\) says that, seen from the nearest point \(z\) , the direction to \(x\) makes an obtuse angle with the direction to every other point of \(K\) . It is called the variational inequality for the projection. The proof should be compared with that of Theorem 6.11, which gave existence of a nearest point in any reflexive space through weak compactness. Here the parallelogram law does more with less: it makes a minimizing sequence converge in norm, and it gives uniqueness in the same stroke.

7.3 Orthogonal complements

Vectors \(x,y\) in an inner product space are orthogonal, written \(x\perp y\) , if \(\langle x,y\rangle=0\) . For orthogonal vectors the expansion of \(\lVert x+y\rVert^2\) reduces to the Pythagorean identity \(\lVert x+y\rVert^2=\lVert x\rVert^2+\lVert y\rVert^2\) . The orthogonal complement of a set \(A\subseteq H\) is

It is always a closed subspace, being an intersection of kernels of the continuous functionals \(x\mapsto\langle x,a\rangle\) .

The operator \(P_M\) is the orthogonal projection onto \(M\) , and \(I-P_M\) is the orthogonal projection onto \(M^\perp\) . In \(\mathbb R^n\) this is the familiar projection that solves least-squares problems (math.linear-algebra.projection-as-best-approximation); what is new is that it exists for every closed subspace of an infinite-dimensional space.

Compare Theorem 7.4 with the situation in Banach spaces. There a closed subspace need not have any closed complement, and when it has one the projection can have large norm (Example 5.5). In a Hilbert space every closed subspace has a canonical complement, at a right angle, with a projection of norm \(1\) . Closedness is still indispensable: for the dense subspace \(c_{00}\) of \(\ell^2\) the complement \(c_{00}^\perp\) is \(\{0\}\) , and \(c_{00}\oplus c_{00}^\perp\) is not all of \(\ell^2\) .

7.4 Orthonormal sequences and expansions

A family of vectors is orthonormal if its members have norm \(1\) and are pairwise orthogonal. Projection onto the span of finitely many orthonormal vectors is explicit.

The numbers \(\langle x,e_k\rangle\) are the Fourier coefficients of \(x\) with respect to the orthonormal family. Any linearly independent sequence \(v_1,v_2,\dots\) can be converted into an orthonormal one with the same successive spans by the Gram–Schmidt process: put \(u_1=v_1\) , \(u_n=v_n-\sum_{k<n}\langle v_n,e_k\rangle e_k\) , and \(e_n=u_n/\lVert u_n\rVert\) . Each step subtracts from \(v_n\) its projection onto the span of the earlier vectors, which by the proposition is the nearest point of that span, and what remains is nonzero by independence.

Example 7.5: The best line and the best parabola for the exponential

In \(L^2[0,1]\) apply Gram–Schmidt to \(1,x,x^2\) . First \(q_0=1\) . Then \(x-\langle x,q_0\rangle q_0=x-\tfrac12\) , with \(\lVert x-\tfrac12\rVert^2=\tfrac1{12}\) , so \(q_1=\sqrt3\,(2x-1)\) . The next step, subtracting from \(x^2\) its components \(\tfrac13\) along \(q_0\) and \(\tfrac1{2\sqrt3}\) along \(q_1\) , leaves \(x^2-x+\tfrac16\) , whose norm is \(1/(6\sqrt5)\) , so \(q_2=\sqrt5\,(6x^2-6x+1)\) .

Now take \(f(x)=e^x\) . Its Fourier coefficients are

\[ \langle f,q_0\rangle=e-1,\qquad\langle f,q_1\rangle=\sqrt3\int_0^1e^x(2x-1)\,dx=\sqrt3\,(3-e),\qquad\langle f,q_2\rangle=\sqrt5\,(7e-19), \]

numerically \(1.7183\) , \(0.4880\) , and \(0.0625\) . By Proposition 7.6 the best approximation by a polynomial of degree at most \(1\) is

\[ p_1(x)=(e-1)+3(3-e)(2x-1)=(4e-10)+(18-6e)x\approx 0.8731+1.6903x , \]

which is the line announced at the start of the chapter. The squared error is \(\lVert f\rVert^2-(e-1)^2-3(3-e)^2\) with \(\lVert f\rVert^2=\tfrac12(e^2-1)\) , giving \(\lVert f-p_1\rVert_2\approx 0.0628\) . Adding the term \(\langle f,q_2\rangle q_2\) gives the best parabola, and the error drops to \(\lVert f-p_2\rVert_2\approx 0.0053\) , because the squared error decreases by exactly \(\langle f,q_2\rangle^2\) . The first two coefficients did not change when the third basis function was added. That is the practical advantage of an orthonormal basis: approximations of increasing order are nested.

Theorem 7.7 (orthonormal bases). Let \((e_k)_{k\ge 1}\) be an orthonormal sequence in a Hilbert space \(H\) . For every \((c_k)\in\ell^2\) the series \(\sum_kc_ke_k\) converges in \(H\) , and \(\lVert\sum_kc_ke_k\rVert^2=\sum_k\lvert c_k\rvert^2\) . Moreover the following are equivalent: (a) the linear span of the \(e_k\) is dense in \(H\) ; (b) \(x=\sum_{k=1}^\infty\langle x,e_k\rangle e_k\) for every \(x\in H\) ; (c) \(\lVert x\rVert^2=\sum_{k=1}^\infty\lvert\langle x,e_k\rangle\rvert^2\) for every \(x\in H\) (Parseval’s identity); (d) the only vector orthogonal to every \(e_k\) is \(0\) . An orthonormal sequence with these properties is an orthonormal basis of \(H\) .

Proof. By Pythagoras, \(\lVert\sum_{k=m+1}^nc_ke_k\rVert^2=\sum_{k=m+1}^n\lvert c_k\rvert^2\) , so the partial sums are Cauchy when \((c_k)\in\ell^2\) ; they converge because \(H\) is complete, and the norm identity passes to the limit. Now fix \(x\) and let \(s_n=\sum_{k\le n}\langle x,e_k\rangle e_k\) , which converges to some \(s\) by Bessel’s inequality and what was just proved. By Proposition 7.6, \(\lVert x-s_n\rVert^2=\lVert x\rVert^2-\sum_{k\le n}\lvert\langle x,e_k\rangle\rvert^2\) , and this is the squared distance from \(x\) to the span of \(e_1,\dots,e_n\) . So (b) and (c) each say that this distance tends to \(0\) for every \(x\) , which is (a). For each fixed \(k\) , \(\langle x-s,e_k\rangle=\lim_n\langle x-s_n,e_k\rangle=0\) . If (d) holds this forces \(x=s\) , which is (b). Finally if (b) holds and \(x\perp e_k\) for all \(k\) , then \(x=\sum_k0\cdot e_k=0\) , which is (d).

Theorem 7.8 (separable Hilbert spaces). Every infinite-dimensional separable Hilbert space \(H\) has an orthonormal basis \((e_k)_{k\ge 1}\) , and the map \(x\mapsto(\langle x,e_k\rangle)_{k\ge 1}\) is a linear isometry of \(H\) onto \(\ell^2\) that preserves inner products.

Proof. Take a dense sequence in \(H\) , discard each member that is a linear combination of earlier ones, and apply Gram–Schmidt to what remains. The resulting orthonormal sequence has the same span as the dense sequence, so it is a basis by criterion (a). The coefficient map is linear, isometric by Parseval’s identity, and onto \(\ell^2\) by the first statement of Theorem 7.7; it preserves inner products by polarization.

All infinite-dimensional separable Hilbert spaces are therefore the same space in different coordinates. The interest of \(L^2[0,1]\) is not that it differs from \(\ell^2\) but that it carries several useful orthonormal bases at once.

Exercises

1. For \(x=(1,2)\) and \(y=(3,-1)\) in \(\mathbb R^2\) compute \(\lVert x+y\rVert_2^2\) and \(\lVert x-y\rVert_2^2\) , verify the parallelogram law, and recover \(\langle x,y\rangle\) from the polarization identity.

   Answer

   \(x+y=(4,1)\) and \(x-y=(-2,3)\) , so \(\lVert x+y\rVert_2^2=17\) and \(\lVert x-y\rVert_2^2=13\) . Their sum is \(30=2\cdot 5+2\cdot 10\) , as the parallelogram law requires. Polarization gives \(\langle x,y\rangle=\tfrac14(17-13)=1\) , in agreement with \(1\cdot 3+2\cdot(-1)=1\) .

2. Prove the polarization identity \(\langle x,y\rangle=\tfrac14\sum_{j=0}^{3}i^j\lVert x+i^jy\rVert^2\) in a complex inner product space.

   Hint

   Expand \(\lVert x+i^jy\rVert^2=\lVert x\rVert^2+\lVert y\rVert^2+2\operatorname{Re}\bigl(\overline{i^j}\langle x,y\rangle\bigr)\) , multiply by \(i^j\) , and sum over \(j=0,1,2,3\) . The terms with \(\lVert x\rVert^2+\lVert y\rVert^2\) cancel because \(1+i-1-i=0\) ; writing \(\langle x,y\rangle=a+ib\) , the remaining terms give \(2(a+a)+2i(b+b)=4\langle x,y\rangle\) .

3. Prove that equality holds in the Cauchy–Schwarz inequality if and only if \(x\) and \(y\) are linearly dependent, by examining the proof of Theorem 7.2.

   Hint

   If \(y\ne 0\) , equality means \(\lVert x-ty\rVert^2=0\) for \(t=\langle x,y\rangle/\lVert y\rVert^2\) , so \(x=ty\) . Conversely if \(x=ty\) both sides equal \(\lvert t\rvert\lVert y\rVert^2\) .

4. Show that \(L^p[0,1]\) with \(1\le p<\infty\) and \(p\ne 2\) fails the parallelogram law, by testing it on the indicator functions of \([0,\tfrac12]\) and \([\tfrac12,1]\) .

   Answer

   With \(f=\mathbf 1_{[0,1/2]}\) and \(g=\mathbf 1_{[1/2,1]}\) : \(\lVert f\rVert_p=\lVert g\rVert_p=2^{-1/p}\) and \(\lVert f\pm g\rVert_p=1\) . The law would require \(1+1=4\cdot 2^{-2/p}\) , that is \(2^{2/p}=2\) , which holds only for \(p=2\) .

5. In \(\ell^2\) let \(x=(2,-1,3,0,0,\dots)\) . Find the nearest point to \(x\) in the closed ball of radius \(2\) about the origin and in the cone of sequences with nonnegative coordinates, with the two distances.

   Answer

   \(\lVert x\rVert_2=\sqrt{14}\approx 3.742\) . Ball of radius \(2\) : nearest point \(2x/\sqrt{14}\) , distance \(\sqrt{14}-2\approx 1.742\) . Cone: nearest point \((2,0,3,0,\dots)\) , distance \(1\) .

6. In \(\mathbb R^2\) with the Euclidean inner product let \(K=\{x:x_1+x_2\le 1\}\) . Find \(P_K(2,2)\) and verify the variational inequality of Theorem 7.3 for it.

   Solution

   The point \(x=(2,2)\) is not in \(K\) , since \(2+2>1\) . The boundary of \(K\) is the line \(x_1+x_2=1\) with normal direction \((1,1)\) , so the candidate is the foot of the perpendicular from \(x\) to that line: \(z=x-t(1,1)\) with \((2-t)+(2-t)=1\) , giving \(t=\tfrac32\) and

   \[ z=\bigl(\tfrac12,\tfrac12\bigr),\qquad x-z=\bigl(\tfrac32,\tfrac32\bigr),\qquad\lVert x-z\rVert_2=\tfrac32\sqrt2\approx 2.121 . \]

   To verify that \(z=P_Kx\) by Theorem 7.3, take any \(k\in K\) :

   \[ \langle x-z,k-z\rangle=\tfrac32\bigl((k_1-\tfrac12)+(k_2-\tfrac12)\bigr)=\tfrac32(k_1+k_2-1)\le 0 , \]

   because \(k_1+k_2\le 1\) on \(K\) . Since \(z\in K\) and the variational inequality holds for all \(k\in K\) , the theorem gives \(P_K(2,2)=(\tfrac12,\tfrac12)\) . Equality in the variational inequality occurs exactly for \(k\) on the boundary line, which is the geometric statement that \(x-z\) is perpendicular to the face of \(K\) on which \(z\) lies.

7. Let \(M\) be the set of even functions in \(L^2[-1,1]\) . Show that \(M\) is a closed subspace, identify \(M^\perp\) and the projection \(P_M\) , and compute \(P_Mf\) and \(\operatorname{dist}(f,M)\) for \(f(x)=e^x\) .

   Solution

   Define the reflection \((Rf)(x)=f(-x)\) . The substitution \(x\mapsto-x\) shows \(\lVert Rf\rVert_2=\lVert f\rVert_2\) , so \(R\) is a bounded linear operator on \(L^2[-1,1]\) , and \(M=\{f:Rf=f\}=\ker(I-R)\) is a closed subspace.

   Let \(N\) be the odd functions, \(N=\ker(I+R)\) . If \(f\) is even and \(g\) odd, then \(f\overline g\) is odd and \(\langle f,g\rangle=\int_{-1}^1f\overline g=0\) ; so \(N\subseteq M^\perp\) . Every \(f\) decomposes as

   \[ f=\tfrac12(f+Rf)+\tfrac12(f-Rf), \]

   with the first term even and the second odd. This is a decomposition \(f=m+n\) with \(m\in M\) and \(n\in M^\perp\) , and by the uniqueness in Theorem 7.4 it is the orthogonal decomposition. Hence \(P_M=\tfrac12(I+R)\) and \(M^\perp=N\) exactly.

   For \(f(x)=e^x\) : \(P_Mf=\tfrac12(e^x+e^{-x})=\cosh x\) , and \(f-P_Mf=\sinh x\) . The distance is

   \[ \operatorname{dist}(f,M)=\lVert\sinh\rVert_2=\Bigl(\int_{-1}^1\sinh^2x\,dx\Bigr)^{1/2}=\Bigl(\tfrac12\sinh 2-1\Bigr)^{1/2}\approx 0.9019 , \]

   using \(\sinh^2x=\tfrac12(\cosh 2x-1)\) . As a check, \(\lVert e^x\rVert_2^2=\sinh 2\approx 3.627\) and \(\lVert\cosh\rVert_2^2=\tfrac12\sinh 2+1\approx 2.813\) ; the two squared norms \(2.813\) and \(0.813\) add to \(3.627\) , as Pythagoras requires.

8. Prove that \(c_{00}^\perp=\{0\}\) in \(\ell^2\) , and explain why this shows that the hypothesis “closed” cannot be dropped from Theorem 7.4.

   Solution

   Let \(x\in\ell^2\) be orthogonal to \(c_{00}\) . Each \(e_k\) belongs to \(c_{00}\) , so \(x_k=\langle x,e_k\rangle=0\) for every \(k\) , and \(x=0\) . Hence \(c_{00}^\perp=\{0\}\) .

   If Theorem 7.4 held for the subspace \(M=c_{00}\) , every \(x\in\ell^2\) would decompose as \(x=m+n\) with \(m\in c_{00}\) and \(n\in c_{00}^\perp=\{0\}\) , which would mean \(\ell^2=c_{00}\) . That is false: \(a=(1/k)\) is in \(\ell^2\) and not in \(c_{00}\) . What goes wrong in the proof is the existence step of Theorem 7.3. A minimizing sequence for the distance from \(a\) to \(c_{00}\) , for instance the truncations \(a^{(n)}\) , is Cauchy and converges in \(\ell^2\) , but its limit \(a\) is not in \(c_{00}\) because \(c_{00}\) is not closed. The infimum \(\operatorname{dist}(a,c_{00})=0\) is not attained. By Corollary 7.5, \((c_{00}^\perp)^\perp=\ell^2\) is the closure of \(c_{00}\) , consistent with density.

9. Apply the Gram–Schmidt process to \((1,1,0)\) , \((1,0,1)\) , \((0,1,1)\) in \(\mathbb R^3\) .

   Answer

   \(e_1=(1,1,0)/\sqrt2\) . Then \((1,0,1)-\tfrac12(1,1,0)=\tfrac12(1,-1,2)\) , so \(e_2=(1,-1,2)/\sqrt6\) . Then \((0,1,1)-\tfrac12(1,1,0)-\tfrac16(1,-1,2)=\tfrac23(-1,1,1)\) , so \(e_3=(-1,1,1)/\sqrt3\) .

10. Find the polynomial of degree at most \(1\) nearest to \(x^2\) in \(L^2[0,1]\) and the distance, using \(q_0\) and \(q_1\) from Example 7.5; compare with the best line for the supremum norm found in Example 3.4.

    Solution

    With \(q_0=1\) and \(q_1=\sqrt3(2x-1)\) ,

    \[ \langle x^2,q_0\rangle=\int_0^1x^2\,dx=\tfrac13,\qquad\langle x^2,q_1\rangle=\sqrt3\int_0^1(2x^3-x^2)\,dx=\sqrt3\bigl(\tfrac12-\tfrac13\bigr)=\tfrac{\sqrt3}{6} . \]

    By Proposition 7.6 the nearest polynomial of degree at most \(1\) is

    \[ p(x)=\tfrac13+\tfrac{\sqrt3}{6}\cdot\sqrt3(2x-1)=\tfrac13+x-\tfrac12=x-\tfrac16 , \]

    and the squared distance is \(\lVert x^2\rVert^2-\tfrac19-\tfrac1{12}=\tfrac15-\tfrac7{36}=\tfrac1{180}\) . So the distance is \(1/\sqrt{180}=1/(6\sqrt5)\approx 0.0745\) . The error \(x^2-x+\tfrac16\) is the unnormalised third Gram–Schmidt vector of Example 7.5, as it must be: it is what remains of \(x^2\) after its projection onto the span of \(1\) and \(x\) is removed.

    For the supremum norm, Example 3.4 found the best line \(x-\tfrac18\) with error \(\tfrac18=0.125\) . The two best lines are parallel and differ by the constant \(\tfrac16-\tfrac18=\tfrac1{24}\) . The mean-square line sits lower because it balances areas, tolerating a larger error at the endpoints, where \(x^2-x+\tfrac16\) equals \(\tfrac16\) , in exchange for a smaller error over most of the interval. “Best approximation” has no meaning until the norm is fixed.

11. Expand \(f(x)=x(1-x)\) in the orthonormal basis \(s_k(x)=\sqrt2\sin(k\pi x)\) of \(L^2[0,1]\) and use Parseval’s identity to evaluate \(\sum_{k\text{ odd}}k^{-6}\) .

    Solution

    Integrating by parts twice, or differentiating under the known integrals of Example 6.2,

    \[ \langle f,s_k\rangle=\sqrt2\int_0^1x(1-x)\sin(k\pi x)\,dx=\frac{2\sqrt2\,\bigl(1-(-1)^k\bigr)}{k^3\pi^3} , \]

    which is \(4\sqrt2/(k^3\pi^3)\) for odd \(k\) and \(0\) for even \(k\) . The vanishing for even \(k\) reflects symmetry: \(f\) is symmetric about \(x=\tfrac12\) and \(\sin(k\pi x)\) is antisymmetric about it when \(k\) is even.

    The norm is \(\lVert f\rVert^2=\int_0^1x^2(1-x)^2\,dx=\tfrac13-\tfrac12+\tfrac15=\tfrac1{30}\) . Parseval’s identity (Theorem 7.7) gives

    \[ \frac1{30}=\sum_{k\text{ odd}}\frac{32}{k^6\pi^6},\qquad\text{hence}\qquad\sum_{k\text{ odd}}\frac1{k^6}=\frac{\pi^6}{960}\approx 1.00145 . \]

    The coefficients decay like \(k^{-3}\) , much faster than the \(k^{-1}\) of Example 7.6, because \(x(1-x)\) vanishes at both endpoints, as every \(s_k\) does. The first term alone carries almost all of the norm: \(32/\pi^6\approx 0.03329\) against \(\tfrac1{30}\approx 0.03333\) , so the single function \(\tfrac{8}{\pi^3}\sin(\pi x)\) approximates \(x(1-x)\) with relative mean-square error below four percent.

12. Let \((e_k)\) be an orthonormal sequence in a Hilbert space. Show that \(\langle x,e_k\rangle\to 0\) for every \(x\) , that \((e_k)\) has no norm-convergent subsequence, and that \(\lVert e_k-e_j\rVert=\sqrt2\) for \(k\ne j\) .

    Answer

    \(\langle x,e_k\rangle\to 0\) because Bessel’s inequality makes \(\sum_k\lvert\langle x,e_k\rangle\rvert^2\) convergent. By Pythagoras \(\lVert e_k-e_j\rVert^2=1+1=2\) for \(k\ne j\) , so no subsequence is Cauchy.

13. For \(v_1=(1,0)\) and \(v_2=(1,1)/\sqrt2\) in \(\mathbb R^2\) compute \(\langle x,v_1\rangle v_1+\langle x,v_2\rangle v_2\) at \(x=(0,1)\) , explain why the result differs from \(x\) , and find the correct coefficients \(c_1,c_2\) with \(x=c_1v_1+c_2v_2\) .

    Answer

    \(\langle x,v_1\rangle=0\) and \(\langle x,v_2\rangle=1/\sqrt2\) , so the sum is \((\tfrac12,\tfrac12)\ne x\) . The vectors have norm \(1\) but \(\langle v_1,v_2\rangle=1/\sqrt2\ne 0\) , so they are not orthonormal. Solving \(c_1(1,0)+c_2(1,1)/\sqrt2=(0,1)\) gives \(c_2=\sqrt2\) and \(c_1=-1\) .

14. Let \(H_0\) be \(c_{00}\) with the inner product of \(\ell^2\) , and \(M=\{x\in c_{00}:\sum_kx_k/2^k=0\}\) . Show that \(M\) is a closed proper subspace of \(H_0\) whose orthogonal complement in \(H_0\) is \(\{0\}\) , so that the projection theorem fails in an incomplete inner product space.

    Solution

    Let \(\varphi(x)=\sum_kx_k/2^k=\langle x,a\rangle\) with \(a=(2^{-k})\in\ell^2\) . On \(H_0\) this is a bounded functional, by the Cauchy–Schwarz inequality, so \(M=\ker\varphi\) is closed in \(H_0\) . It is proper because \(\varphi(e_1)=\tfrac12\ne 0\) .

    Let \(z\in H_0\) be orthogonal to \(M\) . Since \(z\in c_{00}\) , there is \(n\) with \(z_k=0\) for \(k>n\) . Fix \(j\le n\) and choose any \(m>n\) . The vector

    \[ u=e_j-2^{m-j}e_m \]

    is in \(c_{00}\) and satisfies \(\varphi(u)=2^{-j}-2^{m-j}2^{-m}=0\) , so \(u\in M\) . Then \(0=\langle z,u\rangle=z_j-2^{m-j}z_m=z_j\) , because \(z_m=0\) . Hence every coordinate of \(z\) vanishes and \(M^\perp=\{0\}\) in \(H_0\) .

    So \(M\oplus M^\perp=M\ne H_0\) , and the conclusion of Theorem 7.4 fails although \(M\) is closed. The vector \(e_1\) , for example, has no nearest point in \(M\) . In the completion \(\ell^2\) the closure of \(M\) is the hyperplane of Example 7.4, whose complement is spanned by \(a\) ; but \(a\) has infinitely many nonzero coordinates and is not available in \(c_{00}\) . Completeness is what guarantees that the normal direction to a closed hyperplane exists as a vector of the space.

15. Let \((e_k)\) be an orthonormal sequence in a Hilbert space. Decide whether \(\sum_ke_k/k\) and \(\sum_ke_k/\sqrt k\) converge, and compute the norm of the sum when it exists.

    Answer

    \(\sum_ke_k/k\) converges, since \((1/k)\in\ell^2\) , and the norm of its sum is \((\sum_kk^{-2})^{1/2}=\pi/\sqrt6\approx 1.2825\) . \(\sum_ke_k/\sqrt k\) does not converge: its partial sums have squared norms \(\sum_{k\le n}1/k\to\infty\) .

16. In \(\mathbb R^2\) with the norm \(\lVert\cdot\rVert_\infty\) find all points of the closed unit ball nearest to \(x=(2,0)\) , and say which step of the proof of Theorem 7.3 is unavailable.

    Answer

    The distance is \(1\) and the nearest points are \((1,t)\) with \(\lvert t\rvert\le 1\) . The uniqueness step, and the proof that a minimizing sequence is Cauchy, both use the parallelogram law, which \(\lVert\cdot\rVert_\infty\) does not satisfy.

17. Let \(P\) and \(Q\) be the orthogonal projections onto closed subspaces \(M\) and \(N\) of a Hilbert space. Prove that \(PQ=0\) if and only if \(M\perp N\) , and that in this case \(P+Q\) is the orthogonal projection onto \(M+N\) , which is closed.

    Solution

    Suppose \(PQ=0\) . For \(n\in N\) we have \(n=Qn\) , so \(Pn=PQn=0\) , which means \(n\in\ker P=M^\perp\) . Thus \(N\perp M\) . Conversely if \(M\perp N\) , then for every \(x\) the vector \(Qx\in N\subseteq M^\perp=\ker P\) , so \(PQx=0\) . By the symmetry of the relation \(M\perp N\) , also \(QP=0\) .

    Assume now \(M\perp N\) and put \(E=P+Q\) . First, \(M+N=\{x:Ex=x\}\) . If \(x=m+n\) then \(Ex=Pm+Pn+Qm+Qn=m+0+0+n=x\) . If \(Ex=x\) then \(x=Px+Qx\in M+N\) . So \(M+N=\ker(I-E)\) is closed.

    Second, for any \(x\) the vector \(x-Ex\) is orthogonal to \(M+N\) . For \(m\in M\) , using \(\langle Pu,v\rangle=\langle u,Pv\rangle\) from Theorem 7.4,

    \[ \langle x-Px-Qx,m\rangle=\langle x-Px,m\rangle-\langle Qx,m\rangle=0-0 , \]

    since \(x-Px\in M^\perp\) and \(Qx\in N\subseteq M^\perp\) . The same holds with \(N\) in place of \(M\) . So \(x=Ex+(x-Ex)\) with \(Ex\in M+N\) and \(x-Ex\in(M+N)^\perp\) . By the uniqueness of the orthogonal decomposition for the closed subspace \(M+N\) , \(Ex\) is the orthogonal projection of \(x\) onto \(M+N\) . Orthogonality of \(M\) and \(N\) is what makes the sum closed; Example 5.6 showed that without it the sum of two closed subspaces of \(\ell^2\) can fail to be closed.

18. Find the minimum of \(\int_{-1}^{1}\lvert x^3-a-bx-cx^2\rvert^2\,dx\) over real \(a,b,c\) and the minimizing coefficients, by projecting \(x^3\) onto the orthogonal polynomials \(1\) , \(x\) , \(x^2-\tfrac13\) .

    Answer

    On \([-1,1]\) , \(x^3\) is orthogonal to \(1\) and to \(x^2-\tfrac13\) by oddness, and \(\langle x^3,x\rangle/\langle x,x\rangle=\tfrac25\big/\tfrac23=\tfrac35\) . The minimizer is \(a=0\) , \(b=\tfrac35\) , \(c=0\) , and the minimum is \(\int_{-1}^1(x^3-\tfrac35x)^2\,dx=\tfrac27-\tfrac6{25}=\tfrac8{175}\approx 0.0457\) .