About This Course

Probability starts from a known law and asks what data it will produce. Statistics runs the other way: the data are on the table, the law that produced them is not, and the task is to say something reliable about that law and to say how reliable. This book develops the mathematics of that reverse step. It is written for a reader working alone, an advanced undergraduate or beginning graduate student who wants each procedure derived from an explicit model, each guarantee stated with the conditions under which it holds, and each number in an example reproducible by hand or with a few lines of code.

Three things are assumed. From Probability Theory: probability spaces and the rules for computing with them (math.probability-theory.probability-axioms), random variables and their distributions, expectation and variance, conditioning, the standard families (binomial, Poisson, normal, exponential, gamma, beta, uniform), and the two classical limit theorems. From Multivariable Calculus: partial derivatives, maximization of a function of several variables, and change of variables in an integral. From Linear Algebra: matrices, rank, orthogonality, and projection onto a subspace. When a result from one of those courses is used, it is stated with the hypotheses that the argument needs and marked once with its library identifier. It is not proved again here, because the proof belongs to the course that owns it.

The book has thirteen chapters. Chapters 1 to 3 set up the objects: statistical models, the sampling distributions of statistics, and the reduction of data by sufficiency. Chapters 4 to 6 treat point estimation, by risk, by likelihood, and by the theory of best unbiased estimators. Chapters 7 and 8 construct confidence sets and tests and show that the two are the same construction read in two directions. Chapter 9 supplies the large-sample theory that justifies approximate versions of all of them. Chapter 10 applies the whole apparatus to the Gaussian linear model, which is regression. The last three chapters cover the parts of inference that the classical core leaves out and that practice depends on: Bayesian inference and decision in Chapter 11, randomized experiments and their design in Chapter 12, and the evaluation of forecasts in Chapter 13.

A reader who finishes the book and its exercises can do six things with a blank sheet of paper and a table of distributions. Write down a statistical model for a described data-collection procedure and decide which features of it the data can and cannot determine. Derive the sampling distribution of a statistic, exactly where that is possible and approximately otherwise, and use it to build a confidence set with a stated coverage. Find a maximum likelihood estimator, including on a boundary, and compare estimators by bias, variance, and risk. Construct a test with a prescribed level, compute its power, and report a p-value without misreading it. Fit and criticize a linear model. Decide when a Bayesian analysis, a randomized design, or a proper scoring rule is the right tool, and carry out each.

Notation is fixed throughout. Random variables are capital letters and their observed values are the matching lower-case letters. A sample is \(X_1,\dots,X_n\) , with sample mean \(\bar X=\frac1n\sum_{i=1}^n X_i\) and sample variance \(S^2=\frac{1}{n-1}\sum_{i=1}^n (X_i-\bar X)^2\) . A model is a family \(\{P_\theta:\theta\in\Theta\}\) , and \(E_\theta\) , \(\operatorname{Var}_\theta\) , and \(P_\theta\) denote expectation, variance, and probability computed under the member with index \(\theta\) . A density or probability mass function is written \(f(x;\theta)\) . The logarithm \(\log\) is always the natural logarithm. Subscripted critical values are upper points: \(z_\alpha\) is the number with \(P(Z>z_\alpha)=\alpha\) for a standard normal \(Z\) , and \(t_{\nu,\alpha}\) , \(\chi^2_{\nu,\alpha}\) , and \(F_{a,b,\alpha}\) are defined in the same way for the \(t\) , chi-square, and \(F\) distributions. In words, however, “the \(q\) quantile” or “the quantile of order \(q\) ” of a distribution with distribution function \(G\) always means a point \(d\) with \(G(d)=q\) , so that the \(0.975\) quantile of the standard normal distribution is \(z_{0.025}\) . The exponential distribution \(\operatorname{Exp}(\lambda)\) has rate \(\lambda\) and mean \(1/\lambda\) , and \(\operatorname{Gamma}(a,b)\) has shape \(a\) , rate \(b\) , and mean \(a/b\) . The indicator of an event \(A\) is \(\mathbf 1\{A\}\) .

The data sets in the examples are small and were constructed for this book, so that every calculation can be followed by hand; they illustrate methods and are not evidence about any real process. A few of them recur: an inspector’s count of defective items, the calls arriving at a switchboard, and nine measurements of a reference length are each analysed in several chapters by different methods, and comparing the answers is part of the instruction.

Every chapter ends with exercises that run from recall to proof. Do them with pencil and, where numbers are involved, check them with software afterwards. The companion solutions give complete arguments for some, graduated hints for others, and final answers for a few. Several exercises present a plausible argument that is wrong and ask you to find the error; statistical practice is full of such arguments, and recognising them is a separate skill from producing correct ones. The cumulative review at the end mixes the chapters without saying which one each problem belongs to, because deciding which tool applies is part of the problem.