Orthonormal Bases

Last Update: May 10, 2005

Orthonormal Bases

Vectors are orthogonal (the mathematical term for perpendicular) if their dot or inner product is zero.  An orthogonal set of vectors consists of vectors that are orthogonal to each other (mutually orthogonal).  An orthonormal set of vectors consists of orthogonal vectors each of which has length one.

If S={u1,u2,...,un} is an orthonormal basis for a vector space and v is any vector in the vector space then v= <v,u1> u1 + <v,u2> u2 +...+ <v,un> un.

Example

The vectors v1={1/Sqrt[2],1/Sqrt[2]} and v2={-1/Sqrt[2],1/Sqrt[2]} form an orthonormal basis for 2-space as you can see by executing the next 4 lines.

v1={1/Sqrt[2],1/Sqrt[2]};v2={-1/Sqrt[2],1/Sqrt[2]};

v1.v1

v1.v2

v2.v2

The result shown above can be used to find the coordinates of {2,3} with respect to this basis as follows.  First the coordinate of {2,3} with respect to the first basis vector is found.

c1=Simplify[v1.{2,3}]

Next, the coordinate of {2,3} with respect to the second basis vector is calculated.

c2=Simplify[v2.{2,3}]

Then {2,3}=c1 v1 + c2 v2.  This can be verified by having Mathematica do the computation.

Simplify[c1 v1 + c2 v2]

Projections

When an inner product is given as denoted by <u,v> the projection of vector u onto vector v is given by (<u,v>/<v,v>)v.  The projection of vec1 onto vec 2 can be carried out by using the following function.

proj[vec1_,vec2_,inner_:Dot]:=
(inner[vec1,vec2]/inner[vec2,vec2])vec2

Example

Find the projection of vector {1,2,3,4} onto vector {3,1,1,-1} in four space.

u={1,2,3,4};v={3,1,1,-1};

proj[u,v]

Notice that the result is (1/3)v.  This means that the projection of u onto v runs in the same direction as v.

To show that {1,2,3,4} minus the projection of vector {1,2,3,4} onto vector {3,1,1,-1} is orthogonal to {3,1,1,-1} simply show that the dot product of these two vectors is zero.

(u-proj[u,v]).v

The definition of the dot product is changed as needed in certain situations, and the projection of a vector onto another vector will be changed by making the corresponding change in the definition of projection.  In general, when using the inner product <u,v>, the projection of u onto v is given by (<u,v>/<v,v>)v.

The Gram-Schmidt Process

Every nonzero finite dimensional inner product space has an orthonormal basis.  This can be shown by constructing such a basis using the Gram-Schmidt process as developed by Jorgen Gram and Erhard Schmidt.  This procedure is based on finding the projection of a vector onto another vector.

It is a constructive process which begins with any set of basis vectors for the subspace.  The first basis vector becomes the first basis vector in the orthogonal set.  The second basis vector minus its projection onto the first basis vector is taken as the second orthogonal basis vector.  The third basis vector minus its projections onto the first and second orthogonal basis vectors becomes the third orthogonal basis vector.  This process is repeated for all of the original basis vectors.  In this way, a set of orthogonal basis vectors spanning the same subspace as the original basis vectors is reached.  To complete the procedure, each of the orthogonal basis vectors is divided by its length to arrive at the orthonormal set of basis vectors.

The following procedure implements the steps of the last paragraph.  The starting set of vectors is called B.  B is a list of vectors.  In the line following the Block command, the first orthogonal vector is set equal to the first vector in set B.  The first 'Do' loop constructs the rest of the orthogonal vectors by subtracting projections as outlined above, and the second 'Do' statement makes the lengths of each of these orthogonal vectors equal to one.  When this procedure is run, matrix V will have as its rows the orthonormal basis vectors.

gramschmidt[B_,inner_:Dot]:=
     Block[{i,k},
          V=Table[1,{i,1,Length[B]}];
          V[[1]]=B[[1]];
          Do[V[[i]]=B[[i]]-Sum[proj[B[[i]],V[[k]],inner],
		{k,1,i-1}],{i,2,Length[B]}];
          Do[V[[i]]=V[[i]]/Sqrt[V[[i]].V[[i]]],
               {i,1,Length[V]}];
          V ]

Example

The next statement produces an orthonormal basis corresponding to the vectors {1,0,-1},{0,1,2}, and {2,1,-1}.  To apply the gramschmidt command to this set of basis vectors, they must be made the rows of a matrix.  This is accomplished in Mathematica by enclosing the vectors in another set of curly brackets as shown on the next line.  When this line is executed, you will see an orthonormal basis corresponding to the given vectors.

orthonormalbasis=gramschmidt[{{1,0,-1},{0,1,2},{2,1,-1}}]

To pick out the first vector of this orthonormal basis use the Mathematica notation for picking out the first row of a matrix, M[[1]] where M is the name of the matrix.  Applied to this example, you have

orthonormalbasis[[1]]

To show that this vector has length 1 use

orthonormalbasis[[1]].orthonormalbasis[[1]]

To show that the first two vectors in this orthonormal basis are orthogonal compute execute

orthonormalbasis[[1]].orthonormalbasis[[2]]

Exercise 1

Show that vectors {1/Sqrt[2],1/Sqrt[2]} and {-1/Sqrt[2],1/Sqrt[2]} form an orthonormal basis for 2-space.

Exercise 2

Find the projection  of vector {3,4} onto the subspace defined by the basis vectors  {1/Sqrt[2],1/Sqrt[2]} and {-1/Sqrt[2],1/Sqrt[2]}.

Exercise 3

(a) Find the projection of the polynomial 1+3x-4x^2 onto the polynomial -1+x-3x^2 in the space of all polynomials of degree 2 or less.  If u=a0 + a1 x + a2 x^2 and 
v=b0 + b1 x + b2 x^2 define <u,v> as {a0,a1,a2}.{b0,b1,b2}.
(b) Find a vector orthogonal to the vector that you found in part (a) with the property that when added to the vector found in part (a), you get vector 1+3x-4x^2

Exercise 4

Find the orthonormal basis spanned by vectors {1,2,3},{0,1,1},{0,0,-1} in 3-space, and find the coordinates of {4,5,6} with respect to each vector in this orthonormal basis.  Hint: Once you have an orthonormal basis, the coordinate of any vector, w, with respect to an orthonormal basis vector, v, is given by the dot product of w with v, that is w.v.  Second Hint: See the last example before exercise 1 to pick out elements of this orthonormal basis.

Exercise 5

Find an orthonormal basis for the plane x+y+z+w=0.  Find the vector in this plane that lies closest to the vector {1,2,3,4}.

The following vectors form a basis for x+y+z+w=0 but they are not orthogonal (nor are they of length 1).

v1 = {1, 0, 0, -1}

v2 = {0, 1, 0, -1}

v3 = {0, 0, 1, -1}

To find an orthonormal basis start with any vector, say v1 and rename it w1.  Then subtract the projection of v2 onto w1 from v2 and call the result w2.

w1 = v1

w2 = v2 - proj[v2, w1]

Next subtract from v3 its projections onto w1 and w2.  Call the result w3.

w3 = v3 - proj[v3, w1] - proj[v3, w2]

Now the three vectors w1, w2, and w3 are mutually orthogonal.

w3 . w1

w3 . w2

In order to complete the process of creating an orthonormal basis you need to divide w1, w2, and w3 by their lengths.  This is done by the gramschmidt function with the rows of the resulting matrix giving the orthonormal basis.

gs = gramschmidt[{v1, v2, v3}]

Now find the projection of the following w onto the plane x+y+z+w=0 byt finding the projection of w onto each of the orthonormal basis vectors and adding.

w = {1, 2, 3, 4}

projectionofwontoplane = 
      proj[w, gs[[1]]] + proj[w, gs[[2]]] + 
        proj[w, gs[[3]]]

The sum of these vectors should be zero since they lie on the plane.  This is shown in the next two statements.

projectionofwontoplane[[1]] + projectionofwontoplane[[2]] + 
      projectionofwontoplane[[3]] + 
      projectionofwontoplane[[4]]

Simplify[%]