Numerical Analysis I

AMSC 666, Fall 2013


Course Information

LectureRoom 4122 CSIC Bldg. #406 TuTh 2-3:15pm
Note special place: Math Bldg. Rm. 0403 on Tue 10/15
Note special place: Math Bldg. Rm. 0401 on Thu 10/17
InstructorProfessor Eitan Tadmor
Contacttel.: x5-0648   e-mail:
Office HoursBy appointment (e-mail: )
CSCAMM 4122 CSIC Bldg. #406
Teaching Assistant Ming Zhong e-mail:
TA Office Hours4122 CSIC Bldg. #406 MWF 10-12am
Midterm Thu 10/17 2-3:15pm 0401 Math building
NOTE: use of a calculator and your notes is allowed
Final Thu 12/19 10:30-12:30pm 4122 CSIC Bldg. #406
Grading25% Homework, 25% Mid-Term, 50% Final


Course Description

  1. Ten steps of Approximation Theory

    1. General overview
      • On the choice of norm: least-squares vs. the uniform norm
      • Weierstrass' density theorem
      • Bernstein polynomials

    2. Least Squares Approximations I. A general overview
      • Gramm mass matrix
      • Ill-conditioning of monomials in L2

    3. Least Squares Approximations II. (Generalized) Fourier expansions
      • Bessel, Parseval, ...
      • Orthogonal polynomials: Legendre, Chebyshev, Sturm's sequence
      Lecture notes: Spectral Expansions [ pdf file ]
      Assignment #1: [ pdf file ] ... with answers [ pdf file]
    4. Least Squares Approximations III. Discrete expansions
      • Examples of discrete least squares.
      • From discrete least-squares to interpolation

    5. Interpolation I. Lagrange and Newton interpolants
      • Divided differences
      • Equi-distant points
      • Synthetic calculus
      • Forward backward and centered formulae
      Assignment #2 [ pdf file ]
    6. Interpolation II. Error estimates.
      • Runge effect
      • region of analyticity
      Lecture notes: Polynomial interpolation and error estimates [ pdf file ]
    7. Interplolation III. Trigonometric interpolation
      • FFT
      • truncation + aliasing
      • error estimates
      • elliptic solvers
      • fast summations ( - discrete convolution)
      Additional lecture notes on FFT:
      • J. Colley, P. Lewis, J. Welch, The fast Fourier Transform and its applications, IEEE on Education, 1969, vol. 12 (1), [pdf file]

      • C. DeBoor, FFT as nested multiplication with a twist, SISC 1980 vol. 1(1) [pdf file]

      Assignment #3 [ pdf file ] ... with answers [ pdf file]
    8. Min-Max approximations
      • The alternation theorem
      • Chebyshev interpolant and its distance from the min-max polynomial
      • Economization
      Additional reading:
      • M. J. D. Powell, On the maximum errors of polynomial approximations defined by interpolation and by least squares criteria, Computer J., 1967, vol. 9, pp. 404-407, [pdf file]

      • The sin(x) subroutine in MATLAB [pdf file]

    9. Modern aspects of approximation
      • Error Estimates - Jackson, Bernstein and Chebyshev
      • smoothness and regularity spaces
      Lecture notes: Jackson's theorem [ pdf file]
    10. Approximation with derivatives and rational approximations
      • Hermite interpolation
      • piecewise interpolation: Splines
      • Rational approximations: Padè
      Assignment #4 [ pdf file ]
      Polynomial vs. spline interpolation [ demonstration ]
    MID-TERM [ pdf file ] ... and its answers: [ pdf file ]

  2. Numerical Differentiation and Numerical Integration

    1. Numerical differentiation
      • Polynomial and spline interpolants - Error estimates
      • Equidistant points: synthetic calculus; Richardson extrapolation
      • Spline and trigonometric interpolation
      Assignment #5 [ pdf file ] ... with answers [ pdf file]
    2. Gauss Quadratures
      3. Lecture notes: A Very Short Guide to Jacobi Polynomials [ pdf file]
    3. Numerical integration with equidistant points
      • Newton-Cotes formulae
      • Composite Simpson's rule
      • Romberg & adaptive integration

      On Newton-Cotes quadrature - Isaacson & Keller [ pdf file]
      Lecture notes: Numerical integration and Euler-Macluarin formula [ pdf file]
      Assignment #6 [ pdf file ] ... with answers [ pdf file]

  3. Solution of Nonlinear System of Equations - Iterative Methods

    1. Introduction
    2. Fixed point iterations
      3. Lecture notes:
    3. Low-order methods
    4. Newton method and modified Newton's methods
    5. Rates of convergence: low-order vs. high-order methods
      Additional reading:
      • other iterative methods: [Illinois method] [Pegasos method]
      Assignment #7 [ pdf file ]
    6. Accelrations
    7. Iterative solution - systems of equations
    8. Steepest descent; Conjugate gradient method
    9. Polynomial equations: local vs. global methods
      10. Assignment #8 [ pdf file ]

  4. Numerical Optimization

    1. Introduction
    2. Nonlinear least squares methods
    3. Steepest descent and CG methods
      4. Lecture notes: Gradient and conjugate gradient iterations [ pdf file]
    4. Newton's and quasi-Newton methods
    5. Rates of converegence
      6. Final Assignment [ pdf file ] ... with (selected) answers [ pdf file ]

    Epilogue


    References

    GENERAL TEXTBOOKS

    K. Atkinson, An INTRODUCTION to NUMERICAL ANALYSIS, Wiley, 1987

    S. Conte & C. deBoor, ELEMENTARY NUMERICAL ANALYSIS, McGraw-Hill
    User friendly; Shows how 'it' works; Proofs, exercises and notes

    G. Dahlquist & A. Bjorck, NUMERICAL METHODS, Prentice-Hall,
    User friendly; Shows how 'it' works; Exercises

    E. Isaacson & H. Keller, ANALYSIS of NUMERICAL METHODS, Wiley
    The 'First'; Proofs; out-dated in certain aspects; Encrypted message in Preface

    A. Ralston & P. Rabinowitz, FIRST COURSE in NUMERICAL ANALYSIS, 2nd ed., McGraw-hill,
    Detailed; Scholarly written; Comprehensive; Proofs exercises and notes

    J. Stoer & R. Bulrisch, INTRODUCTION TO NUMERICAL ANALYSIS, 2nd ed., Springer
    detailed account on approximation, linear solvers & eigen-solvers, ODE solvers,..

    B. Wendroff, THEORETICAL NUMERICAL ANALYSIS, Academic Press, 1966
    Only the 'Proofs'; elegant presentation

    APPROXIMATION THEORY

    E. W. Cheney, INTRODUCTION TO APPROXIMATION THEORY
    Classical

    P. Davis, INTERPOLATION & APPROXIMATION, Dover
    Very readable

    T. Rivlin, AN INTRODUCTION to the APPROXIMATION of FUNCTIONS
    Classical

    R. DeVore & G. Lorentz, CONSTRUCTIVE APPROXIMATION, Springer
    A detailed account from classical theory to the modern theory; everything; Proofs exercises and notes

    NUMERICAL INTEGRATION

    F. Davis & P. Rabinowitz, NUMERICAL INTEGRATION,
    Everything...

    Eitan Tadmor