Numerical Analysis I

AMSC 666, Fall 2003

Basic Information

Course Description

1. Ten steps of Approximation Theory

   1. General overview
   • On the choice of norm: L² vs. L^¥ 
   • Weierstrass' density theorem
   • Bernstein polynomials
   
   
   2. Least Squares Approximations I. A general overview
   • Gramm mass matrix
   • ill-conditioning of monomials in L²
   
   
   3. Least Squares Approximations II. (Generalized) Fourier expansions
   • Bessel, Parseval, ...
   • Orthogonal polynomials: Legendre, Chebyshev, Sturm's sequence

   Assignment #1 [ pdf file ]

   4. Least Squares Approximations III. Discrete expansions
   • Examples of discrete least squares.
   • From discrete least-squares to interpolation

   Lecture notes: Spectral Expansions [ pdf file ]

   5. Interpolation I. Lagrange and Newton interpolants
   • Divided differences
   • Equi-distant points
   • Synthetic calculus
   • Forward backward and centered formulae
   
   
   6. Interpolation II. Error estimates.
   • Runge effect
   • region of analyticity

   Polynomial vs. spline interpolation [ demonstration ]

   7. Interpolation III. Interpolation with derivatives
   • Hermite interpolation
   • piecewise interpolation
   • Splines

   Lecture notes: Interpolation Error [ pdf file ]

   Assignment #2 [ pdf file ]

   8. Interplolation IV. Trigonometric interpolation
   • FFT
   • truncation + aliasing
   • error estimates
   • elliptic solvers
   • fast summations ( - discrete convolution)
   
   
   9. MiniMax Approximation
   • Chebyshev interpolation
   10. Modern aspects of approximation
   • Error Estimates - Jackson, Bernstein and Chebyshev
   • smoothness and regularity spaces
   
   
   

2. Numerical Differentiation and Numerical Integration

1. Numerical differentiation
   • Polynomial and spline interpolants - Error estimates
   • Equidistant points: synthetic calculus; Richardson extrapolation
   • Spline and trigonometric interpolation
   
   

   Assignment #3  [ pdf file ] and a solution to Jackson's theorem [ pdf file]

2. Gauss Quadratures



Lecture notes: A Very Short Guide to Jacobi Polynomials [ pdf file]

3. Numerical integration with equidistant points
• Newton-Cotes formulae
• Composite Simpson's rule
• Romberg & adaptive integration



Assignment #4 [ pdf file ]

MID-TERM #1 (Q & A's -- an example) [ pdf file ]

MID-TERM #2 (Q & A's -- the real one) [ pdf file ]





3. Solution of Linear System of Equations - Iterative Methods

   1. Introduction
   2. Fixed point iterations

   Lecture notes: Matrices -- norms, eigenvalues and powers [ pdf file]

   3. The basic algorithms:
      • Jacobi, Gauss-Seidel and SOR methods
      
      
   4. Steepest descent; Conjugate gradient method
   5. ADI and dimensional splitting methods
   6. Multigrid
   7. Preconditioners
   
   

   Assignment #5 [ pdf file ]

   
   

4. Eigen-solvers

1. Introduction
2. Similarity transformations, Rayleigh quotations
3. Power and inverse power method
4. Householder transformations
5. Hessenberg form and the QR method

Assignment #6 [ pdf file ]

6. Singular Value decomposition
7. Preconditioners
8. The divide and conquer method



FINAL (+ answers) [ pdf file ]

---

Epilogue - have you been paying attention in your numerical analysis course?

---




References

GENERAL TEXTBOOKS

K. Atkinson, An INTRODUCTION to NUMERICAL ANALYSIS, Wiley, 1987

S. Conte & C. deBoor, ELEMENTARY NUMERICAL ANALYSIS, McGraw-Hill
User friendly; Shows how 'it' works; Proofs, exercises and notes

G. Dahlquist & A. Bjorck, NUMERICAL METHODS, Prentice-Hall,
User friendly; Shows how 'it' works; Exercises

E. Isaacson & H. Keller, ANALYSIS of NUMERICAL METHODS, Wiley
The 'First'; Proofs; out-dated in certain aspects; Encrypted message in Preface

A. Ralston & P. Rabinowitz, FIRST COURSE in NUMERICAL ANALYSIS, 2nd ed., McGraw-hill,
Detailed; Scholarly written; Comprehensive; Proofs exercises and notes

J. Stoer & R. Bulrisch, INTRODUCTION TO NUMERICAL ANALYSIS, Springer-Verlag
detailed account on approximation, linear solvers & eigen-solvers, ODE solvers,..

B. Wendroff, THEORETICAL NUMERICAL ANALYSIS, Academic Press, 1966
Only the 'Proofs'; elegant presentation

APPROXIMATION THEORY

E. W. Cheney, INTRODUCTION TO APPROXIMATION THEORY
Classical

P. Davis, INTERPOLATION & APPROXIMATION, Dover
Very readable

T. Rivlin, AN INTRODUCTION to the APPROXIMATION of FUNCTIONS
Classical

R. DeVore & G. Lorentz, CONSTRUCTIVE APPROXIMATION, Springer
A detailed account from classical theory to the modern theory; everything; Proofs exercises and notes

NUMERICAL INTEGRATION

F. Davis & P. Rabinowitz, NUMERICAL INTEGRATION,
Everything...

(mainly) ITERATIVE SOLUTION OF LINEAR SYSTEMS

A. Householder, THE THEORY OF MATRICES IN NUMERICAL ANALYSIS
The theoretical part by one of the grand masters; Outdated in some aspects

G. H. Golub & Van Loan, MATRIX COMPUTATIONS,
The basic modern reference

Y. Saad, ITERATIVE METHODS for SPARSE LINEAR SYSTEMS,
PWS Publishing, 1996. (Available on line at http://www-users.cs.umn.edu/~saad/books.html)

R. Varga, MATRIX ITERATIVE ANALYSIS,
Classical reference for the theory of iterations

J. H. Wilkinson HANDBOOK for AUTOMATIC COMPUTATIONS, 1971
Modern theory started there with the grand master...

D. Young, ITERATIVE SOLUTION OF LARGE linear SYSTEMS, Academic Press, 1971
Excellent detailed account

(mainly) EIGEN-SOLVERS

B. Parllett, THE SYMMETRIC EIGENVALUE PROBLEM
Recommended

J. H. Wilkinson The ALGEBRAIC EIGEVALUE PROBLEM, 1965
The classical reference



---

Eitan Tadmor