Linear Models Theory for Data Science

This book provides a holistic view of linear models as the foundation for many statistical and machine learning methods. It covers the theory of the standard linear model, then explores extensions of that model and the use of linear predictors in a broad range of more computationally demanding modeling techniques. Optimization procedures that underlie this evolutionary tree of models are also covered, while highlighting the central role of the linear model in statistics, machine learning, and data science. The early chapters cover linear model formulation, matrix algebra, distribution theory, and estimation and inference for the standard linear model. Following an overview of optimization techniques, the later chapters cover models for correlated responses, regularization, models for non-normal responses, predictive models, and models for inherently nonlinear relationships. Throughout, the emphasis is on the interrelated ideas connecting these many methods that collectively address a wide range of modeling situations.

While the book emphasizes theoretical foundations, it also recognizes that implementation is essential for understanding why and how the methods work. Numerous data analysis examples illustrate modeling principles and the corresponding theory. Computing code and datasets are available from the book's GitHub repository and permanent Zenodo archive. Exercises provide hands-on training in applying the methods.

Designed as a graduate-level textbook for students in statistics, data science, and related fields, the book provides a concise and engaging treatment of linear models and their extensions. After studying this book, readers will have a comprehensive understanding of linear models and their connections to a broad range of methods employing linear predictors, providing a solid foundation for advanced study and research in data science.