About This Course

Numerical analysis is the study of algorithms that compute with real numbers when only finitely many digits and finitely many steps are available. Its subject is error: where it comes from, how it grows, how to bound it before a computation is run, and how to estimate it afterwards from the numbers the computation produces. This book is written for a reader working alone, an undergraduate or beginning graduate student in mathematics, science, or engineering who wants the theory at the level of proof and the algorithms at the level where every printed digit can be reproduced and explained.

Four things are assumed. From Calculus: derivatives, integrals, and Taylor’s theorem with remainder. From Linear Algebra: linear systems, elimination, norms, eigenvalues, and orthogonal projection. From Differential Equations: what an initial-value problem is and how linear equations with constant coefficients behave. From a first proof course: comfort with quantifiers, induction, and the limit of a sequence as it is treated in Real Analysis. When a theorem owned by one of those courses is used, it is stated with the hypotheses the argument needs and marked once with its library identifier; it is not proved again here.

Five abilities mark a reader who has finished the book, each exercised on a computation and none requiring the text at hand. Predict how many digits a floating-point computation can deliver, and separate the sensitivity of a problem from the instability of an algorithm. Solve a scalar equation with a method whose convergence you can prove and whose stopping test you can defend. Approximate a function, an integral, or a data set and state an error bound with its hypotheses. Solve a linear system or an eigenvalue problem by a direct or an iterative method and say what the residual does and does not certify. Integrate an initial-value or boundary-value problem, choose the step from accuracy and stability requirements, and produce convergence evidence that another person can check.

Notation is conventional. The symbol \(u\) always denotes the unit roundoff of the arithmetic in use, and \(\operatorname{fl}(\cdot)\) denotes the result of a computation carried out in floating-point arithmetic. For a vector \(x\) , \(\lVert x\rVert_2\) , \(\lVert x\rVert_1\) , and \(\lVert x\rVert_\infty\) are the Euclidean, sum, and maximum norms; the same subscripts on a matrix denote the induced matrix norms. For a function on an interval, \(\lVert f\rVert_\infty\) is the maximum of \(\lvert f\rvert\) over that interval. We write \(C^k[a,b]\) for the functions with \(k\) continuous derivatives on \([a,b]\) , and \(e_k\) for the error of the \(k\) -th iterate of whatever iteration is under discussion.

Every worked example is a computation, and every number printed in one was produced by running the method in IEEE double-precision arithmetic. You are meant to reproduce them. A dozen lines in any language with double-precision floats suffice for almost every example, and the short code listings in the text show the form such a program takes. Where a printed digit is dominated by rounding noise and may differ between machines, the text says so. Such digits depend also on the exact order in which an expression is evaluated: in the examples a power such as \(x^3\) is always formed by repeated multiplication, \(x\cdot x\cdot x\) , which IEEE arithmetic reproduces identically on every machine, whereas a library power routine need not. Each chapter closes with twenty exercises that run from recall through computation to proof, and several ask you to find the flaw in a plausible computation. In the solutions companion about nine exercises per chapter are worked in full, and every other one has either a graduated hint or a final answer to check against, so no exercise is left without support. The cumulative review after Chapter 10 does not say which chapter each of its problems draws on, because recognising which tool a problem calls for is part of numerical analysis.

Two neighbouring subjects are deliberately left to their owners. Whether a point is a minimizer, and what certifies it, is the business of Continuous Optimization; this book treats only the numerical behaviour of the iterations involved. What a fitted coefficient says about the process that generated the data is the business of Mathematical Statistics; this book treats the fit as a computation.