Numerical Analysis 4

Spring 2005

Questions

Big O

Definitions

Definition 1.9: The function f(h) is said to be big Oh of g(h), denoted by f(h)=O(g(h)), if there exist constants C and c such that |f(h))≤C|g(h)| whenever h≥c

Definition 1.10: Let {(Subscript[x, n])}_(n = 1)^∞ and {(Subscript[y, n])}_(n = 1)^∞ be two sequences.  The sequence {(Subscript[x, n])} is said to be of order big Oh of {(Subscript[y, n])}, denoted by (Subscript[x, n])=O((Subscript[y, n])) if there exist constants C and N such that |(Subscript[x, n])|≤C|(Subscript[y, n])| when n≥N.

Definition 1.11: Assume that the function f(h) is approximated by a function p(h) and there is a constant M>0 and a positive integer n so that \(TraditionalForm\`\(\(|\)\(f(h) - p(h)\)\(|\)\)\/\(\(|\)\(h\^n\)\(|\)\)\)≤ M for h sufficiently close to 0.  Then p(h) is said to approximate f(h) with order of approximation O((h^n)).  This can be written as f(h)=p(h)+O((h^n))

Definition 1.12: Let {(Subscript[x, n])}_(n = 1)^∞ and {(Subscript[r, n])}_(n = 1)^∞ be two sequences.  Suppose (Underscript[limit, n -> ∞]) (Subscript[r, n])=0.  We say the sequence {(Subscript[x, n])} converges to x with order of convergence O((Subscript[r, n])) if there is a constant K>0 such that \(TraditionalForm\`\(\(|\)\(x\_n - x\)\(|\)\)\/\(\(|\)\(r\_n\)\(|\)\)\)≤ K for n sufficiently large (i.e. there is some N such that the inequality is true for n≥N).  This can be written (Subscript[x, n])= x + O((Subscript[r, n])).

Definition 1.13: Suppose ϵ is the original error in a computation and ϵ(n) represents the error after n steps of the computation.  If |ϵ(n)|≈nϵ, the growth of the error is said to be linear.  If |ϵ(n)|≈(K^n)ϵ , the growth of the error is said to be exponential.  If K>1, the error grows without bounds as n->∞, while if 0<K<1, the error->0 as n->∞.

Link to Wikipedia Page on Big O Notation

Some Examples

As noted in the above link, there are two situations in which the Big O Notation is used: (1) situations in which a sequence or function is converging as the argument approaches infinity; (2) situations in which a sequence or function is converging as the argument approaches some finite value.

As an example of (1) consider, from the Wikipedia link, 4(n^2)- 2n + 2 which is said to be O((n^2)) as n->∞.  This means there is some N such that |4(n^2)- 2n + 2|<(n^2) for all n>N.  Consider the graphs of |4(n^2)- 2n + 2| and 8 (n^2) (in red) for n from 0 to 10.  It looks like the red graph remains above the graph of the function, certainly for all n>4.

g1 = 
      Plot[Abs[4  (n^2) - 2  n + 2], {n, 0, 10}, 
        DisplayFunction -> Identity]; 
    g2 = Plot[Abs[8  (n^2)], {n, 0, 10}, PlotStyle -> RGBColor[1, 0, 0], 
        DisplayFunction -> Identity]; 
    Show[{g1, g2}, DisplayFunction -> $DisplayFunction];

Next, as an example of the 2nd use of the Big O notation, look at the infinitesimal asymptotics example from the Wikipedia link.  Here you want to show that the absolute value of the difference between (e^x) and 1+x+((x^2)/2)is less than some constant times the absolute value of (x^3)for x sufficiently close to 0 (i.e. for all |x|< δ for some positive δ).  The graphs in the following suggest this--there is an easy proof that to show that your conclusion from the graph is correct.

g1 = 
      Plot[Abs[Exp[x] - (1 + x + x^2/2)], {x, -1, 1}, 
        DisplayFunction -> Identity]; 
    g2 = Plot[Abs[x^3], {x, -1, 1}, PlotStyle -> RGBColor[1, 0, 0], 
        DisplayFunction -> Identity]; 
    Show[{g1, g2}, DisplayFunction -> $DisplayFunction];

Series[Exp[x], {x, 0, 2}]

Big O Notation is handled properly in Mathematica

Error Propagation

In most cases computations are not done in one step.  Therefore, knowing something about the way in which errors propagate in successive computations is important.

Assume that p = \(TraditionalForm\`p\&^\) + (Subscript[ϵ, p]) and q = \(TraditionalForm\`q\&^\) + (Subscript[ϵ, q]).  The relative errors are 
\(TraditionalForm\`\(\(|\)\(p - \(\(p\)\(|\)\)\&^\)\)\/\(\(|\)\(p\)\(|\)\)\)= \(TraditionalForm\`\(\(|\)\(ϵ\_p\)\(|\)\)\/\(\(|\)\(p\)\(|\)\)\)and \(TraditionalForm\`\(\(|\)\(q - \(\(q\)\(|\)\)\&^\)\)\/\(\(|\)\(q\)\(|\)\)\)= \(TraditionalForm\`\(\(|\)\(ϵ\_q\)\(|\)\)\/\(\(|\)\(q\)\(|\)\)\)

Then p + q = \(TraditionalForm\`p\&^\) + (Subscript[ϵ, p]) +  \(TraditionalForm\`q\&^\) + (Subscript[ϵ, q]) = \(TraditionalForm\`p\&^\) + \(TraditionalForm\`q\&^\) + (Subscript[ϵ, p]) + (Subscript[ϵ, q]).  So, in summation operations, the errors are also summed.

pq = (\(TraditionalForm\`p\&^\) + (Subscript[ϵ, p]))( \(TraditionalForm\`q\&^\) + (Subscript[ϵ, q])) = \(TraditionalForm\`p\&^\)\(TraditionalForm\`q\&^\) + (Subscript[ϵ, p])\(TraditionalForm\`q\&^\) + (Subscript[ϵ, q])\(TraditionalForm\`p\&^\) + (Subscript[ϵ, p])(Subscript[ϵ, q]).  In multiplication, then the errors are multiplied (last term) plus the two middle terms are added to the computation.  These middle terms have the effect of increasing errors considerably if \(TraditionalForm\`p\&^\) and \(TraditionalForm\`q\&^\) are bigger than 1 in absolute value.
The relative error of pq, \(TraditionalForm\`\(pq - \(p\&^\) q\&^\)\/pq\)=(((Subscript[ϵ, p])\(q\&^\)+(Subscript[ϵ, q])\(p\&^\)+(Subscript[ϵ, p])(Subscript[ϵ, q]))/pq)= (((Subscript[ϵ, p])\(q\&^\))/pq)+ (((Subscript[ϵ, q])\(p\&^\))/pq)+ (((Subscript[ϵ, p])(Subscript[ϵ, q]))/pq).  If you can assume that \(TraditionalForm\`p\&^\)≈p and \(TraditionalForm\`q\&^\)≈q , then \(TraditionalForm\`p\&^\/p\)≈1 and \(TraditionalForm\`q\&^\/q\)≈1, so the last equation above becomes \(TraditionalForm\`\(pq - \(p\&^\) q\&^\)\/pq\)=((Subscript[ϵ, p])/p)+ (((Subscript[ϵ, q]))/q)+ (((Subscript[ϵ, p])(Subscript[ϵ, q]))/pq)= ((Subscript[ϵ, p])/p)+ (((Subscript[ϵ, q]))/q)  (the (((Subscript[ϵ, p])(Subscript[ϵ, q]))/pq) term is missing because of the assumption that \(TraditionalForm\`p\&^\)≈p and \(TraditionalForm\`q\&^\)≈q, in which case (Subscript[ϵ, p])≈0 and (Subscript[ϵ, q])≈0)

Example: Problem 2 on page 39.

r[0] = 0.994
r[n_] := 1/2  r[n - 1]

Table[{n, r[n]}, {n, 1, 10}]

p[0] = 1; p[1] = 0.497
p[n_] := (3/2) p[n - 1] - (1/2) p[n - 2]

Table[{n, p[n]}, {n, 2, 10}]

q[0] = 1; q[1] = 0.497
q[n_] := (5/2) q[n - 1] - q[n - 2]

Table[{n, q[n]}, {n, 2, 10}]

TableForm[Table[{N[1/2^n], r[n], p[n], q[n]}, {n, 2, 20}]]

TableForm[
      errors = Table[{n, N[1/2^n], N[1/2^n] - r[n], N[1/2^n] - p[n], 
            N[1/2^n] - q[n]}, {n, 2, 24}]]

tsperrors = Transpose[errors]

ListPlot[Transpose[{tsperrors[[1]], tsperrors[[3]]}], 
        AxesOrigin -> {0, 0}, 
        PlotRange -> {{0, 24}, {-0.01, 0.01}}, 
        PlotStyle -> PointSize[0.02]];

ListPlot[Transpose[{tsperrors[[1]], tsperrors[[4]]}], 
        AxesOrigin -> {0, 0}, 
        PlotRange -> {{0, 24}, {-0.01, 0.01}}, 
        PlotStyle -> PointSize[0.02]];

ListPlot[Transpose[{tsperrors[[1]], tsperrors[[5]]}], 
        AxesOrigin -> {0, 0}, PlotRange -> {{0, 24}, {0, 1000}}, 
        PlotStyle -> PointSize[0.02]];

Beginning of Chapter 2: Solving x=g(x)

Iteration

You can find a solution of the equation  x = g (x)   by starting with  (Subscript[p, 0])  and iterating   (Subscript[p, n + 1])  =  g ((Subscript[p, n])).   Terminate the iteration process when the relative error estimate,   2|(((Subscript[p, n + 1]) -  (Subscript[p, n]))/(small +  ∣ 
                (Subscript[p, n + 1]) ∣ ))|,   in consecutive iterates  (Subscript[p, 0])  and  (Subscript[p, n + 1])  differs by less than the preassigned value  tolerance.  The next Mathematica function is a modification by me of the function formulated by the author of your textbook.  For the function FixedPointIteration[g0_,X0_,tolerance_,max_,digits_,{leftx_,rightx_},{lowery_,uppery_}], the following are the definitions of the arguments
g0_ is the function g(x),
X0_ is the starting value for the iterations,
tolerance_  is a the preassigned value tolerance,
max_  is the maximum number of iterations allowed,
digits_ is the number of digits displayed in the output.
leftx_ is the leftmost x-value displayed on the graph
rightx_  is the rightmost x-value displayed on the graph
lowery_ is the minimum y-value displayed on the graph
uppery_ is the maximum y-value displayed on the graph

Off[General::"spell1"];
Clear[FixedPointIteration, g0, X0, tolerance, max, x];
\(\(FixedPointIteration[g0_, X0_, tolerance_, max_, 
          digits_, {leftx_, rightx_}, {lowery_, uppery_}] := 
	
        Module[{g, k, x0, x1, 
            relerr}, 
 (*Initial\ Values\ for\ Variables*) 
	
          Set @@ {g[x_], g0}; 
		x0 = 
            N[X0, digits]; 
	\ \ plist = {X0}; 
		k = 
            0; 
		err = 1. ; 
		relerr = 1; 
		small = 
            N[1\/10\^20, digits]; 
		Print["p"\_0, " = ", 
            N[x0, digits]]; 
 (*Iteration*) 
		While[
            k < max && 
              tolerance ≤ 
                relerr, 
			Module[{}, 
				x1 = 
                g[x0]; 
				k = k + 1; 
				err = 
                Abs[x1 - x0]; 
				Print["p"\_k, " = ", 
                N[x1, digits], ",   |Δp| ≤ ", 
                N[err, digits]]; 
				relerr = \(2\ Abs[x1 - x0]\)\/\(Abs[x1] + small\); 
				x0 = x1; 
              plist = Append[plist, x1]]]; 
		p = 
            x0; 
		Print[" "]; 
 (*Collect\ approximations\ into\ a\ list\ to\ form\ points\ for\ graphing*) 
	
          doubleplist = 
            Flatten[Transpose[{plist, plist}]]; 
	
          pts = Transpose[{Drop[doubleplist, {Length[doubleplist]}], 
                Prepend[Drop[doubleplist, 2], 
                  0]}]; 
 (*Print\ Results*) 
		Print["The function is g[x] = ", g[x]]; 
          Print["A fixed point of g[x] is:"]; 
          Print["  p  = ", N[p, digits]]; 
          Print["g[p] = ", N[g[p], digits]]; 
          Print["|Δp| ≤ ", 
            N[err, digits]]; 
 (*Graph\ Results*) 
	
          gr1\  = Plot[g[x], {x, leftx, rightx}, 
              PlotRange -> {{leftx, rightx}, {lowery, uppery}}, 
              PlotStyle -> RGBColor[1, 0, 0], 
              DisplayFunction -> Identity]; 
		gr2\  = 
            Plot[x, {x, leftx, rightx}, 
              PlotRange -> {{leftx, rightx}, {lowery, uppery}}, 
              PlotStyle -> RGBColor[0, 1, 0], 
              DisplayFunction -> Identity]; 
	
          gr3\  = ListPlot[pts, 
              PlotRange -> {{leftx, rightx}, {lowery, uppery}}, 
              PlotStyle -> {PointSize[0.01]}, 
              DisplayFunction -> Identity]; 
	
          gr4 = Show[Graphics[Line[pts]], 
              DisplayFunction -> Identity]; 
		Show[{gr1, gr2, gr3, 
              gr4}, DisplayFunction -> $DisplayFunction];];\)\)
On[General::"spell1"];

FixedPointIteration[Exp[-x], 0.5, 0.1, 30, 
        12, {0, 1}, {0, 1}];

Clearly there is a point of intersection, a fixed point.

Fixed Points

Graphical Interpretation

Error Considerations