The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition

There is no other book I know of in this space with the same combination of thorough detailed math, intuition, application to real-world data, and excellent graphics. It's also very well-written. Their notation can be a bit weird, but whatever. Maybe I'm weird for finding their notation weird. Enough praise. Just buy it and study it. I personally like it better than the comparable books by Barber, Bishop, Murphy, and others, but to each their own. These three are excellent books in their own right, and maybe some would prefer them, especially if one does a lot of Bayesian modeling. But usually, one doesn't. And if you're a beginner in machine learning, my opinion is that studying Bayesian inference as a default can be confusing. Reading advice, if you're not a mathematician (if you are, you don't need my advice): I highly recommend going through a book on standard statistical inference first, else you might be a bit lost, and subtle points that Hastie et al make might be missed (I often pick up details on a second reading - lots of "aha" moments to be had). Not to mention the fact that some of their derivations will seem impenetrable; that one for bias and variance of the linear model in chapter 2 nonplussed me for a while. Luckily there are the accompanying notes by Weatherwax et al (google it), which are seriously helpful. Good options for background are Casella & Berger (the standard), the book "Statistical rethinking from scratch" by Edge (such a good book!), the book "Probability and mathematical statistics" by Meyer (this looks excellent but I don't know it well), and many others (the number of books written on statistical inference asymptotically approaches infinity). Some people like the book by Wasserman but I find it so "skeletal" (as one reviewer said) that one has to go elsewhere for the details anyway. So why not just read a less skeletal book? Anyway, back to ESL. Reading this has made me a less dumb person, even though I've only read in detail the first 3 chapters. I hope it will do the same for you.

I have been using The Elements of Statistical Learning for years, so it is finally time to try and review it. The Elements of Statistical Learning is a comprehensive mathematical treatment of machine learning from a statistical perspective. This means you get good derivations of popular methods such as support vector machines, random forests, and graphical models; but each is developed only after the appropriate (and wrongly considered less sexy) statistical framework has already been derived (linear models, kernel smoothing, ensembles, and so on). In addition to having excellent and correct mathematical derivations of important algorithms The Elements of Statistical Learning is fairly unique in that it actually uses the math to accomplish big things. My favorite examples come from Chapter 3 "Linear Methods for Regression." The standard treatments of these methods depend heavily on respectful memorization of regurgitation of original iterative procedure definitions of the various regression methods. In such a standard formulation two regression methods are different if they have superficially different steps or if different citation/priority histories. The Elements of Statistical Learning instead derives the stopping conditions of each method and considers methods the same if they generate the same solution (regardless of how they claim they do it) and compares consequences and results of different methods. This hard use of isomorphism allows amazing results such as Figure 3.15 (which shows how Least Angle Regression differs from Lasso regression, not just in algorithm description or history: but by picking different models from the same data) and section 3.5.2 (which can separate Partial Least Squares' design CLAIM of fixing the x-dominance found in principle components analysis from how effective it actually is as fixing such problems). The biggest issue is who is the book for? This is a mathy book emphasizing deep understanding over mere implementation. Unlike some lesser machine learning books the math is not there for appearances or mere intimidating typesetting: it is there to allow the authors to organize many methods into a smaller number of consistent themes. So I would say the book is for researchers and machine algorithm developers. If you have a specific issue that is making inference difficult you may find the solution in this book. This is good for researchers but probably off-putting for tinkers (as this book likely has methods superior to their current favorite new idea). The interested student will also benefit from this book, the derivations are done well so you learn a lot by working through them. Finally- don't buy the kindle version, but the print book. This book is satisfying deep reading and you will want the advantages of the printed page (and Amazon's issues in conversion are certainly not the authors' fault).

This review is written from the perspective of a programmer who has sometimes had the chance to choose, hire, and work with algorithms and the mathematician/statisticians that love them in order to get things done for startup companies. I don't know if this review will be as helpful to professional mathematicians, statisticians, or computer scientists. The good news is, this is pretty much the most important book you are going to read in the space. It will tie everything together for you in a way that I haven't seen any other book attempt. The bad news is you're going to have to work for it. If you just need to use a tool for a single task this book won't be worth it; think of it as a way to train yourself in the fundamentals of the space, but don't expect a recipe book. Get something in the "using R" series for that. When it came out in 2001 my sense of machine learning was of a jumbled set of recipes that tended to work in some cases. This book showed me how the statistical concepts of bias, variance, smoothing and complexity cut across both fields of traditional statistics and inference and the machine learning algorithms made possible by cheaper cpus. Chapters 2-5 are worth the price of the book by themselves for their overview of learning, linear methods, and how those methods can be adopted for non-linear basis functions. The hard parts: First, don't bother reading this book if you aren't willing to learn at least the basics of linear algebra first. Skim the second and third chapters to get a sense for how rusty your linear algebra is and then come back when you're ready. Second, you really really want to use the SQRRR technique with this book. Having that glimpse of where you are going really helps guide you're understanding when you dig in for real. Third, I wish I had known of R when I first read this; I recommend using it along with some sample data sets to follow along with the text so the concepts become skills not just abstract relationships to forget. It would probably be worth the extra time, and I wish I had known to do that then. Fourth, if you are reading this on your own time while making a living, don't expect to finish the book in a month or two.

This is a book for excelling undergraduate mathematicians or graduate-level mathematicians. Truthfully I'm not confident that I would have been able to truly grasp a lot of the material as an undergraduate Statistics major (maybe in my senior year). With that being said. If you're a mathematician then this book will give you a phenomenal grasp of the material in a time when everyone is getting into machine learning but no one actually knows any of the math. Is it important to know the math? Maybe not. You can become a successful analyst with only the computational experience. But I have a passion for the field and I enjoy knowing what it is I'm doing. Anyways, I especially recommend this book because it covers unsupervised learning. This is one of the most overlooked facets of machine learning and it's becoming an ever-growing field. I believe some of the next big statistical discoveries will be in unsupervised learning. Anyways, if you're not strong in mathematics then I recommend Introduction to Statistical Learning with Applications in R. That'll get you started in the field quite well! Good luck to all students and passionate observers!

Math books, at least data science texts, can usually be divided into those which are easy to read but contain little technical rigor and those which are written with a scientific approach to methodology but are so equation dense that it’s hard to imagine them being read outside an advanced academic setting. Fortunately, The Elements of Statistical Learning proves the exception. The text is full with the equations necessary to root the methodology without engaging the reader with long proofs that would tax those of us employing these techniques in the business world. The visual aspects of the text seem to have been written with John Tukey or Edward Tufte in mind. Though their frequent use makes the book some seven hundred pages long, reading and comprehension is made much easier. And, though it’s been almost ten years since the book was published, the techniques described remain, for the most part, at the cutting edge of data science. I was told by some other analysts I know that this was their bible for data science. I was somewhat skeptical of this kind of hyperbole but was pleasantly surprised that the book matched these high expectations. If you have an undergraduate degree in a mathematically related discipline, The Elements of Statistical Learning will prove to be an invaluable reference to understand the rapidly advancing avalanche of data mining techniques.

"The Elements of Statistical Learning: Data Mining, Inference and Prediction," 2nd edition by Trevor Hastie, Robert Tibshirani and Jerome Friedman is the classic reference for the recent developments in machine learning statistical methods that have been developed at Stanford and other leading edge universities. Their book covers a broad range of topics and is filled with applications. Much new material has been added since the first edition was published in 2001. Since most of these procedures have been implemented in the open-source program R, this book provides a basic and needed reference for their application. Important estimation procedures discussed include MARS, GAM, Projection Pursuit, Exploratory Projection Pursuit, Random Forest, General Linear Models, Ridge Models and Lasso Models etc. There is an discussion of bagging and boosting and how these techniques can be used. There is an extensive index and the many of the datasets discussed are available from the web page of the book or from other sources on the web. Each chapter has a number of problems that test mastery of the material. I have used material from this book in a number of graduate classes at the University of Illinois in Chicago and have implemented a number of the techniques in my software system B34S. While the 1969 book by Box and Jenkins set the stage for time series analysis using ARIMA and Transfer Function Models, Hastie, Tibshirani and Friedman have produced the classic reference for a wide range of new and important techniques in the area of Machine Learning. For anyone interested in Data Mining this is a must own book.

I like this book but with some reservations. For an intermediate or advanced student, this is a great book for expanding your toolkit -- it discusses and explores many techniques with which you are probably already familiar, and plenty more with which you are probably not. This book has a smorgasbord of in-depth explorations of a wide array of useful techniques. So it's great as a reference, and a great read for any practitioner looking to add more tools to their toolbox (and aren't we all looking for that)? As noted, though, I have a few reservations. First, I wouldn't recommend this for the beginner -- many of the derivations skip some steps that are obvious if you've seen this problem before, but not so much if you're seeing it for the first time, and the importance and area of application of each technique isn't always made clear. This is a good book to read, but it shouldn't be your first. Second, some of the material is a little...dated. The material on neural networks is so dated it's basically not useful -- doesn't discuss any of the more recent advances (dropout, batch normalization, LSTMs etc), and same goes for any material in here related to image recognition. (Nearest neighbors with handcrafted features for image recognition? seriously? What about CNNs?) Of course that's probably just a function of when this was written. Keeping those reservations in mind, though, this book will give you a thorough grounding in a wide array of powerful techniques for analyzing your data. You'll want this on your shelf as a reference at the very least.

This is a quite interesting, and extremely useful book, but it is wearing to read in large chunks. The problem, if you want to call it that, is that it is essentially a 700 page catalogue of clever hacks in statistical learning. From a technical point of view it is well-ehough structured, but there is not the slightest trace of an overarching philosophy. And if you don't actually have a philosophical perspective in place before you start, the read you face might well be an even harder grind. Be warned. Some of the reviews here complain that there is too much math. I don't think that is an issue. If you have decent intuitions in geometry, linear algebra, probability and information theory, then you should be able to cruise through and/or browse in a fairly relaxed way. If you don't have those intuitions, then you are attempting to read the wrong book. There were a couple of things that I expected (things I happen to know a bit about), but that were missing. On the unsupervised learning side, the discussion of Gaussian mixture clustering was, I thought, a bit short and superficial, and did not bring out the combination of theoretical and practical power that the method offers. On the supervised learning side, I was surprised that a book that dedicates so much time to linear regression finds no room for a discussion of Gaussian process regression as far as I could see (the nearest point of approach is the use of Gaussian radial basis functions [oops: having written that, I immediately came across a brief discussion (S5.8.1) of, essentially, GP regression - though with no reference to standard literature]).

Great looking book

Livre parfait pour les personnes avec un bon background statistique, sinon je vous recommande pattern recognition and machine learning de Christopher M. Bishop qui repars de la base mais tend à suggérer une pensée plus bayesienne. Lire les deux vous donnera une vision Clair d'un peut prêt tout sur le machine learning hors réseau de neurones.

Questo volume è fondamentale per chiunque voglia approfondire le proprie basi (teoriche...) sull'apprendimento statistico. Scritto dai titani del campo, è un libro omnicomprensivo che, partendo dalle basi (nei primi capitoli, probabilmente per introdurli in maniera strumentale alla trattazione sviluppata, vengono descritte le tecniche base di regressione e classificazione) arriva a descrivere concetti molto più complessi e avanzati, come le varie tecniche di regolarizzazione (Ridge, LASSO), il metodo di Benjamini-Hochberg, le SVM etc.

Great book! Yet you wont learn much purely by reading it if you are newbie / student in statistics and or data science! Authors are the most revered alive researchers in the field of statistics!

Recebi bem antes do previsto, sou Estatístico e não precisei estudar o livro inteiro para saber que é uma obra de excelência, as explicações são muito boas e as figuras coloridas e auto explicativas. Realmente vale a pena a compra. Antes de ler seria bom visitar novamente os livros de cálculo e álgebra linear, e as técnicas estatísticas como regressão linear e não linear.