Beta Phase: Square45 is currently in beta testing. Expect some features or content to be incomplete or missing.
45

Hilbert Spaces

Field: Functional Analysis

Complete inner product spaces, central to functional analysis and quantum mechanics.

Sequence of Expressions

One of the most familiar examples of a Hilbert space is the Euclidean vector space consisting of three-dimensional vectors, denoted by R3\mathbf {R} ^{3} , and equipped with the dot product. The dot product takes two vectors x and y, and produces a real number x ⋅ y. If x and y are represented in Cartesian coordinates, then the dot product is defined by: (x1x2x3)⋅(y1y2y3)=x1y1+x2y2+x3y3 .{\begin{pmatrix}x_{1}\\x_{2}\\x_{3}\end{pmatrix}}\cdot {\begin{pmatrix}y_{1}\\y_{2}\\y_{3}\end{pmatrix}}=x_{1}y_{1}+x_{2}y_{2}+x_{3}y_{3}\,. The dot product satisfies the properties: - It is symmetric in x and y: x ⋅ y = y ⋅ x. - It is linear in its first argument: (ax_{1} + bx_{2}) ⋅ y = a(x_{1} ⋅ y) + b(x_{2} ⋅ y) for any scalarsa, b, and vectors x_{1}, x_{2}, and y. - It is positive definite: for all vectors x, x ⋅ x ≥ 0 , with equality if and only ifx = 0. An operation on pairs of vectors that, like the dot product, satisfies these three properties is known as a (real) inner product. A vector space equipped with such an inner product is known as a (real) inner product space. Every finite-dimensional inner product space is also a Hilbert space. The basic feature of the dot product that connects it with Euclidean geometry is that it is related to both the length (or norm) of a vector, denoted ‖x‖, and to the angle θ between two vectors x and y by means of the formula x⋅y=∥x∥∥y∥ cos⁡θ .\mathbf {x} \cdot \mathbf {y} =\left\|\mathbf {x} \right\|\left\|\mathbf {y} \right\|\,\cos \theta \,. Completeness means that a series of vectors (in blue) results in a well-defined net displacement vector (in orange). Multivariable calculus in Euclidean space relies on the ability to compute limits, and to have useful criteria for concluding that limits exist. A mathematical series ∑n=0∞xn\sum _{n=0}^{\infty }\mathbf {x} _{n} consisting of vectors in R^{3} is absolutely convergent provided that the sum of the lengths converges as an ordinary series of real numbers: ∑k=0∞∥xk∥<∞ .\sum _{k=0}^{\infty }\|\mathbf {x} _{k}\|<\infty \,. Just as with a series of scalars, a series of vectors that converges absolutely also converges to some limit vector L in the Euclidean space, in the sense that lim⁡N→∞∥L−∑k=0Nxk∥=0.\lim _{N\to \infty }\left\|\mathbf {L} -\sum _{k=0}^{N}\mathbf {x} _{k}\right\|=0. This property expresses the completeness of Euclidean space: that a series that converges absolutely also converges in the ordinary sense. Hilbert spaces are often taken over the complex numbers. The complex plane denoted by C is equipped with a notion of magnitude, the complex modulus|z|, which is defined as the square root of the product of z with its complex conjugate: ∣z∣2=zz‾ .|z|^{2}=z{\overline {z}}\,. If z = x + iy is a decomposition of z into its real and imaginary parts, then the modulus is the usual Euclidean two-dimensional length: ∣z∣=x2+y2 .|z|={\sqrt {x^{2}+y^{2}}}\,. The inner product of a pair of complex numbers z and w is the product of z with the complex conjugate of w: ⟨z,w⟩=zw‾ .\langle z,w\rangle =z{\overline {w}}\,. This is complex-valued. The real part of ⟨z, w⟩ gives the usual two-dimensional Euclidean dot product. A second example is the space C^{2} whose elements are pairs of complex numbers z = (z_{1}, z_{2}). Then an inner product of z with another such vector w = (w_{1}, w_{2}) is given by ⟨z,w⟩=z1w‾1+z2w‾2 .\langle z,w\rangle =z_{1}{\overline {w}}_{1}+z_{2}{\overline {w}}_{2}\,. The real part of ⟨z, w⟩ is then the four-dimensional Euclidean dot product. This inner product is Hermitian symmetric, which means that the result of interchanging z and w is the complex conjugate: ⟨w,z⟩=⟨z,w⟩‾ .\langle w,z\rangle ={\overline {\langle z,w\rangle }}\,. A Hilbert space is a real or complexinner product space that is also a complete metric space with respect to the distance function induced by the inner product. To say that a complex vector space H is a complex inner product space means that there is an inner product ⟨x,y⟩\langle x,y\rangle associating a complex number to each pair of elements x,yx,y of H that satisfies the following properties: - The inner product is conjugate symmetric; that is, the inner product of a pair of elements is equal to the complex conjugate of the inner product of the swapped elements: ⟨y,x⟩=⟨x,y⟩‾ .\langle y,x\rangle ={\overline {\langle x,y\rangle }}\,. Importantly, this implies that ⟨x,x⟩\langle x,x\rangle is a real number. - The inner product is linear in its first argument. For all complex numbers aa and b,b, ⟨ax1+bx2,y⟩=a⟨x1,y⟩+b⟨x2,y⟩ .\langle ax_{1}+bx_{2},y\rangle =a\langle x_{1},y\rangle +b\langle x_{2},y\rangle \,. - The inner product of an element with itself is positive definite: ⟨x,x⟩>0 if x≠0,⟨x,x⟩=0 if x=0 .{\begin{alignedat}{4}\langle x,x\rangle >0&\quad {\text{ if }}x\neq 0,\\\langle x,x\rangle =0&\quad {\text{ if }}x=0\,.\end{alignedat}} It follows from properties 1 and 2 that a complex inner product is antilinear, also called conjugate linear, in its second argument, meaning that ⟨x,ay1+by2⟩=aˉ⟨x,y1⟩+bˉ⟨x,y2⟩ .\langle x,ay_{1}+by_{2}\rangle ={\bar {a}}\langle x,y_{1}\rangle +{\bar {b}}\langle x,y_{2}\rangle \,. A real inner product space is defined in the same way, except that H is a real vector space and the inner product takes real values. Such an inner product will be a bilinear map and (H,H,⟨⋅,⋅⟩)(H,H,\langle \cdot ,\cdot \rangle ) will form a dual system. Illustration of triangle inequality with distance function on each side The norm is the real-valued function ∥x∥=⟨x,x⟩ ,\|x\|={\sqrt {\langle x,x\rangle }}\,, and the distance dd between two points x,yx,y in H is defined in terms of the norm by d(x,y)=∥x−y∥=⟨x−y,x−y⟩ .d(x,y)=\|x-y\|={\sqrt {\langle x-y,x-y\rangle }}\,. Here, d(x,y)d(x,y) is a distance function meaning firstly that it is symmetric in xx and y,y, secondly that the distance between xx and itself is zero, and otherwise the distance between xx and yy must be positive, and lastly that the triangle inequality holds, meaning that the length of one leg of a triangle xyz cannot exceed the sum of the lengths of the other two legs: d(x,z)≤d(x,y)+d(y,z) .d(x,z)\leq d(x,y)+d(y,z)\,. This last property is ultimately a consequence of the more fundamental Cauchy–Schwarz inequality, which asserts ∣⟨x,y⟩∣≤∥x∥∥y∥\left|\langle x,y\rangle \right|\leq \|x\|\|y\| with equality if and only if xx and yy are linearly dependent. With a distance function defined in this way, any inner product space is a metric space, and sometimes is known as a pre-Hilbert space. Any pre-Hilbert space that is additionally also a complete space is a Hilbert space. The completeness of H is expressed using a form of the Cauchy criterion for sequences in H: a pre-Hilbert space H is complete if every Cauchy sequenceconverges with respect to this norm to an element in the space. Completeness can be characterized by the following equivalent condition: if a series of vectors ∑k=0∞uk\sum _{k=0}^{\infty }u_{k} converges absolutely in the sense that ∑k=0∞∥uk∥<∞ ,\sum _{k=0}^{\infty }\|u_{k}\|<\infty \,, then the series converges in H, in the sense that the partial sums converge to an element of H. As a complete normed space, Hilbert spaces are by definition also Banach spaces. As such they are topological vector spaces, in which topological notions like the openness and closedness of subsets are well defined. Of special importance is the notion of a closed linear subspace of a Hilbert space that, with the inner product induced by restriction, is also complete (being a closed set in a complete metric space) and therefore a Hilbert space in its own right. The sequence space ℓ2{\textstyle \ell ^{2}} consists of all infinite sequencesz = (z_{1}, z_{2}, ...) of complex numbers such that the series of its squared norms converges: ∑n=1∞∣zn∣2\sum _{n=1}^{\infty }|z_{n}|^{2} The inner product on l^{2} is defined by: ⟨z,w⟩=∑n=1∞znw‾n ,\langle \mathbf {z} ,\mathbf {w} \rangle =\sum _{n=1}^{\infty }z_{n}{\overline {w}}_{n}\,, The series for the inner product converges as a consequence of the Cauchy–Schwarz inequality and the assumed convergence of the two series of squared norms. Completeness of the space holds provided that whenever a series of elements from ℓ2{\textstyle \ell ^{2}} converges absolutely (in norm), then it converges to an element of ℓ2{\textstyle \ell ^{2}} . The proof is basic in mathematical analysis, and permits mathematical series of elements of the space to be manipulated with the same ease as series of complex numbers (or vectors in a finite-dimensional Euclidean space). - ^ ^{a}^{b}Axler 2024, p. 182, 6A Inner Products and Norms - ^However, some sources call finite-dimensional spaces with these properties pre-Hilbert spaces, reserving the term "Hilbert space" for infinite-dimensional spaces; see, e.g., Levitan 2001. - ^Olver, Peter J.; Shakiban, Chehrzad (2018), Applied Linear Algebra, Undergraduate Texts in Mathematics, Springer International Publishing, p. 137, doi:10.1007/978-3-319-91041-3, ISBN9783319910413 - ^Marsden 1974, §2.8 - ^Kainth (2023). For the completeness of Euclidean space, see Definition 4.37 and Example 4.38, p. 108; for the equivalence of completeness with the property that absolutely convergent series converge, see Theorem 4.44, p. 110. - ^ ^{a}^{b}Vince, John (2018), Imaginary Mathematics for Computer Science, Springer International Publishing, doi:10.1007/978-3-319-94637-5, ISBN9783319946375 - ^ ^{a}^{b}^{c}^{d}Andrilli, Stephen; Hecker, David (2010), Elementary Linear Algebra, Elsevier, pp. 446–447, ISBN9780080886251 - ^The mathematical material in this section can be found in any good textbook on functional analysis, such as Dieudonné (1960), Hewitt & Stromberg (1965), Reed & Simon (1980) or Rudin (1987). - ^ ^{a}^{b}^{c}^{d}Axler (2024), pp. 183–184. - ^Axler (2024), p. 185, Properties 6(d) and 6(e). - ^Schaefer & Wolff 1999, pp. 122–202 - ^Axler (2024), p. 186. - ^Dieudonné (1960), Section V.1. - ^Dieudonné (1960), Section III.1. - ^ ^{a}^{b}Dieudonné (1960), Section VI.2. - ^Roman 2008, p. 327 - ^Roman 2008, p. 330 Theorem 13.8 - ^Kainth (2023), p. 108, Definition 4.37. - ^Schaefer & Wolff 1999, pp. 40–41. - ^ ^{a}^{b}Stein & Shakarchi 2005, p. 163 - ^Deitmar 2005, p. 26. - ^Dieudonné 1960 Cite error: There are tags or {{efn}} templates on this page, but the references will not show without a {{reflist|group=lower-alpha}} template or {{notelist}} template (see the help page).
Definition

Definition

A real or complex inner product space that is also a complete metric space with respect to the distance function induced by the inner product.
Property

Properties

Two vectors u and v in a Hilbert space H are orthogonal when ⟨u, v⟩ = 0. The notation for this is u ⊥ v. More generally, when S is a subset in H, the notation u ⊥ S means that u is orthogonal to every element from S. When u and v are orthogonal, one has ∥u+v∥2=⟨u+v,u+v⟩=⟨u,u⟩+2 Re⁡⟨u,v⟩+⟨v,v⟩=∥u∥2+∥v∥2 .\|u+v\|^{2}=\langle u+v,u+v\rangle =\langle u,u\rangle +2\,\operatorname {Re} \langle u,v\rangle +\langle v,v\rangle =\|u\|^{2}+\|v\|^{2}\,. By induction on n, this is extended to any family u_{1}, ..., u_{n} of n orthogonal vectors, ∥u1+⋯+un∥2=∥u1∥2+⋯+∥un∥2.\left\|u_{1}+\cdots +u_{n}\right\|^{2}=\left\|u_{1}\right\|^{2}+\cdots +\left\|u_{n}\right\|^{2}. Whereas the Pythagorean identity as stated is valid in any inner product space, completeness is required for the extension of the Pythagorean identity to series. A series Σu_{k} of orthogonal vectors converges in H if and only if the series of squares of norms converges, and ∥∑k=0∞uk∥2=∑k=0∞∥uk∥2 .{\Biggl \|}\sum _{k=0}^{\infty }u_{k}{\Biggr \|}^{2}=\sum _{k=0}^{\infty }\left\|u_{k}\right\|^{2}\,. Furthermore, the sum of a series of orthogonal vectors is independent of the order in which it is taken. Geometrically, the parallelogram identity asserts that AC^{2} + BD^{2} = 2(AB^{2} + AD^{2}). In words, the sum of the squares of the diagonals is twice the sum of the squares of any two adjacent sides. By definition, every Hilbert space is also a Banach space. Furthermore, in every Hilbert space the following parallelogram identity holds: ∥u+v∥2+∥u−v∥2=2(∥u∥2+∥v∥2) .\|u+v\|^{2}+\|u-v\|^{2}=2{\bigl (}\|u\|^{2}+\|v\|^{2}{\bigr )}\,. Conversely, every Banach space in which the parallelogram identity holds is a Hilbert space, and the inner product is uniquely determined by the norm by the polarization identity. For real Hilbert spaces, the polarization identity is ⟨u,v⟩=14(∥u+v∥2−∥u−v∥2) .\langle u,v\rangle ={\tfrac {1}{4}}{\bigl (}\|u+v\|^{2}-\|u-v\|^{2}{\bigr )}\,. For complex Hilbert spaces, it is ⟨u,v⟩=14(∥u+v∥2−∥u−v∥2+i∥u+iv∥2−i∥u−iv∥2) .\langle u,v\rangle ={\tfrac {1}{4}}{\bigl (}\|u+v\|^{2}-\|u-v\|^{2}+i\|u+iv\|^{2}-i\|u-iv\|^{2}{\bigr )}\,. The parallelogram law implies that any Hilbert space is a uniformly convex Banach space. This subsection employs the Hilbert projection theorem. If C is a non-empty closed convex subset of a Hilbert space H and x a point in H, there exists a unique point y ∈ C that minimizes the distance between x and points in C, y∈C ,∥x−y∥=dist⁡(x,C)=min⁡{∥x−z∥∣z∈C} .y\in C\,,\quad \|x-y\|=\operatorname {dist} (x,C)=\min {\bigl \{}\|x-z\|\mathrel {\big |} z\in C{\bigr \}}\,. This is equivalent to saying that there is a point with minimal norm in the translated convex set D = C − x. The proof consists in showing that every minimizing sequence (d_{n}) ⊂ D is Cauchy (using the parallelogram identity) hence converges (using completeness) to a point in D that has minimal norm. More generally, this holds in any uniformly convex Banach space. When this result is applied to a closed subspace F of H, it can be shown that the point y ∈ F closest to x is characterized by y∈F ,x−y⊥F .y\in F\,,\quad x-y\perp F\,. This point y is the orthogonal projection of x onto F, and the mapping P_{F} : x → y is linear (see § Orthogonal complements and projections). This result is especially significant in applied mathematics, especially numerical analysis, where it forms the basis of least squares methods. In particular, when F is not equal to H, one can find a nonzero vector v orthogonal to F (select x ∉ F and v = x − y). A very useful criterion is obtained by applying this observation to the closed subspace F generated by a subset S of H. A subset S of H spans a dense vector subspace if (and only if) the vector 0 is the sole vector v ∈ H orthogonal to S. The dual spaceH* is the space of all continuous linear functions from the space H into the base field. It carries a natural norm, defined by ∥φ∥=sup⁡∥x∥=1,x∈H∣φ(x)∣ .\|\varphi \|=\sup _{\|x\|=1,x\in H}|\varphi (x)|\,. This norm satisfies the parallelogram law, and so the dual space is also an inner product space where this inner product can be defined in terms of this dual norm by using the polarization identity. The dual space is also complete so it is a Hilbert space in its own right. If e_{•} = (e_{i})_{i ∈ I} is a complete orthonormal basis for H then the inner product on the dual space of any two f,g∈H∗f,g\in H^{*} is ⟨f,g⟩H∗=∑i∈If(ei)g(ei)‾\langle f,g\rangle _{H^{*}}=\sum _{i\in I}f(e_{i}){\overline {g(e_{i})}} where all but countably many of the terms in this series are zero. The Riesz representation theorem affords a convenient description of the dual space. To every element u of H, there is a unique element φ_{u} of H*, defined by φu(x)=⟨x,u⟩\varphi _{u}(x)=\langle x,u\rangle where moreover, ∥φu∥=∥u∥.\left\|\varphi _{u}\right\|=\left\|u\right\|. The Riesz representation theorem states that the map from H to H* defined by u ↦ φ_{u} is surjective, which makes this map an isometricantilinear isomorphism. So to every element φ of the dual H* there exists one and only one u_{φ} in H such that ⟨x,uφ⟩=φ(x)\langle x,u_{\varphi }\rangle =\varphi (x) for all x ∈ H. The inner product on the dual space H* satisfies ⟨φ,ψ⟩=⟨uψ,uφ⟩ .\langle \varphi ,\psi \rangle =\langle u_{\psi },u_{\varphi }\rangle \,. The reversal of order on the right-hand side restores linearity in φ from the antilinearity of u_{φ}. In the real case, the antilinear isomorphism from H to its dual is actually an isomorphism, and so real Hilbert spaces are naturally isomorphic to their own duals. The representing vector u_{φ} is obtained in the following way. When φ ≠ 0, the kernelF = Ker(φ) is a closed vector subspace of H, not equal to H, hence there exists a nonzero vector v orthogonal to F. The vector u is a suitable scalar multiple λv of v. The requirement that φ(v) = ⟨v, u⟩ yields u=⟨v,v⟩−1 φ(v)‾ v .u=\langle v,v\rangle ^{-1}\,{\overline {\varphi (v)}}\,v\,. This correspondence φ ↔ u is exploited by the bra–ket notation popular in physics. It is common in physics to assume that the inner product, denoted by ⟨x|y⟩, is linear on the right, ⟨x∣y⟩=⟨y,x⟩ .\langle x|y\rangle =\langle y,x\rangle \,. The result ⟨x|y⟩ can be seen as the action of the linear functional ⟨x| (the bra) on the vector |y⟩ (the ket). The Riesz representation theorem relies fundamentally not just on the presence of an inner product, but also on the completeness of the space. In fact, the theorem implies that the topological dual of any inner product space can be identified with its completion. An immediate consequence of the Riesz representation theorem is also that a Hilbert space H is reflexive, meaning that the natural map from H into its double dual space is an isomorphism. In a Hilbert space H, a sequence {x_{n}} is weakly convergent to a vector x ∈ H when lim⁡n⟨xn,v⟩=⟨x,v⟩\lim _{n}\langle x_{n},v\rangle =\langle x,v\rangle for every v ∈ H. For example, any orthonormal sequence {f_{n}} converges weakly to 0, as a consequence of Bessel's inequality. Every weakly convergent sequence {x_{n}} is bounded, by the uniform boundedness principle. Conversely, every bounded sequence in a Hilbert space admits weakly convergent subsequences (Alaoglu's theorem). This fact may be used to prove minimization results for continuous convex functionals, in the same way that the Bolzano–Weierstrass theorem is used for continuous functions on R^{d}. Among several variants, one simple statement is as follows: If f : H → R is a convex continuous function such that f(x) tends to +∞ when ‖x‖ tends to ∞, then f admits a minimum at some point x_{0} ∈ H. This fact (and its various generalizations) are fundamental for direct methods in the calculus of variations. Minimization results for convex functionals are also a direct consequence of the slightly more abstract fact that closed bounded convex subsets in a Hilbert space H are weakly compact, since H is reflexive. The existence of weakly convergent subsequences is a special case of the Eberlein–Šmulian theorem. Any general property of Banach spaces continues to hold for Hilbert spaces. The open mapping theorem states that a continuoussurjective linear transformation from one Banach space to another is an open mapping meaning that it sends open sets to open sets. A corollary is the bounded inverse theorem, that a continuous and bijective linear function from one Banach space to another is an isomorphism (that is, a continuous linear map whose inverse is also continuous). This theorem is considerably simpler to prove in the case of Hilbert spaces than in general Banach spaces. The open mapping theorem is equivalent to the closed graph theorem, which asserts that a linear function from one Banach space to another is continuous if and only if its graph is a closed set. In the case of Hilbert spaces, this is basic in the study of unbounded operators (see Closed operator). The (geometrical) Hahn–Banach theorem asserts that a closed convex set can be separated from any point outside it by means of a hyperplane of the Hilbert space. This is an immediate consequence of the best approximation property: if y is the element of a closed convex set F closest to x, then the separating hyperplane is the plane perpendicular to the segment xy passing through its midpoint. - ^Reed & Simon 1980, Theorem 12.6 - ^Reed & Simon 1980, p. 38 - ^Young 1988, p. 23 - ^Clarkson 1936 - ^Rudin 1987, Theorem 4.10 - ^Dunford & Schwartz 1958, II.4.29 - ^Rudin 1987, Theorem 4.11 - ^Blanchet, Gérard; Charbit, Maurice (2014), Digital Signal and Image Processing Using MATLAB, vol. 1 (Second ed.), New Jersey: Wiley, pp. 349–360, ISBN978-1848216402 - ^Weidmann 1980, Theorem 4.8 - ^Peres 1993, pp. 77–78 - ^Weidmann (1980, Exercise 4.11) - ^Weidmann 1980, §4.5 - ^Buttazzo, Giaquinta & Hildebrandt 1998, Theorem 5.17 - ^Halmos 1982, Problem 52, 58 - ^Rudin 1973 - ^Trèves 1967, Chapter 18
Any general property of Banach spaces continues to hold for Hilbert spaces. The open mapping theorem states that a continuoussurjective linear transformation from one Banach space to another is an open mapping meaning that it sends open sets to open sets. A corollary is the bounded inverse theorem, that a continuous and bijective linear function from one Banach space to another is an isomorphism (that is, a continuous linear map whose inverse is also continuous). This theorem is considerably simpler to prove in the case of Hilbert spaces than in general Banach spaces. The open mapping theorem is equivalent to the closed graph theorem, which asserts that a linear function from one Banach space to another is continuous if and only if its graph is a closed set. In the case of Hilbert spaces, this is basic in the study of unbounded operators (see Closed operator). The (geometrical) Hahn–Banach theorem asserts that a closed convex set can be separated from any point outside it by means of a hyperplane of the Hilbert space. This is an immediate consequence of the best approximation property: if y is the element of a closed convex set F closest to x, then the separating hyperplane is the plane perpendicular to the segment xy passing through its midpoint. - ^Halmos 1982, Problem 52, 58 - ^Rudin 1973 - ^Trèves 1967, Chapter 18