The Numerati

[[ASIN:156726217Imagine that you are hiking through the woods, as you move through the trail you leave behind a series of clues that a tracker could follow and interpret. If the tracker was skillful enough they would be able to accurately estimate you weight, height, speed, and perhaps even determine where you are heading. Back at your home, you leave a similar sort of trail daily as you go about your normal routine: buying groceries, shopping online, reading blogs, surfing the internet. This is your data trail and there are modern day trackers who are following it. Stephen Baker calls them the Numerati and it is these people who are the subject of his book

The Numerati are a diverse group of mathematicians, scientists, and entrepreneurs who are tracking you as you leave your imprint in the data world in pursuit of a pot of gold - you. Or not really you, but a model of you. A model that allows the Numerati to not only understand where you've been, but predict how you will act. The goal of these models is a complete understanding of our most basic wants, desires, and fears and how they guide our behavior. It is in the predictive ability of these models where in lies the pot of gold. Governments, businesses, political parties are all willing to pay a great deal of money to have access to these models and each of these parties wants a particular model of your behavior.

It is these specific models of us as consumers, lovers, voters etc. that provide an allow Mr. Baker to provide interesting insights into the inner world of the Numerati. structures his book using various personas or archetypes Lover, Voter, Consumer etc., so further investigate the inner workings of the Numerati. Attempting to model complete individuals is too complex, but providing smaller, more discrete models of particular aspects of our behavior are much more predictive and therefore valuable to those who are interested in manipulating or persuading us.

So is this good or bad? Mr. Baker is suitably neutral in his treatment as the models in themselves have no moral value, but only predictive value. But he does not ignore the moral dimension as he does provide examples of how some of his subjects express reservations about how their research is being applied. But Mr. Baker takes a much more pragmatic view of the industry, it exists and is not likely to disappear, it has too much value to the powers that be, so it is better to shine a light on it so that we can better understand its implications. And if this was his goal, Mr. Baker has succeeded.



1 Project Decisions: The Art and Science]]

The science of data-mining is gaining in sophistication, but don't look to this book for any real understanding. Baker has written a book containing very little actual information content. He does not even attempt to convey how these techniques work or what their limitations are. Instead he paints a picture of a sinister and not-too-human "Numerati" that is handling our data while spurning basic social skills. It's a comic book plot that takes the place of any actual factual information. All you come away with is the idea that Baker is scared of what mathematicians are doing. 90% of the book is fluff.

IBM, Google, Accenture, Carnegie Mellon, Intel, Mayo Clinic used mathematical models to do data mining on consumer patterns. The book is an easy read. You do not need any mathematical or quantitative background.

Yes, data modeling and data mining existed for many years. Modeling human behavior to find the niche in marketing, remain to be the research processes that these companies are working on.

For years, marketing is being creative, trying to design the best ad that sells. With quants marching in the room, marketing is very different today. This book will be better if more data or analysis can be presented.

This book was not only boring, I also didn't learn anything at all. I really struggled to get through this book, and thought many times about just giving up. I wish I had just given up and stopped reading it after the first few pages. It is also written in such a pretentious style - whoever uses the work "confrere" these days? Do yourself a favor and buy something else.

It may be that you have a "shopper's card" at your local grocery; you hand it to the teller as you check out, and the computer registers, besides what the total is and how the store's inventory will need to be restocked, just what the purchases were for you as a specific individual shopper. Maybe it will mail you some coupons on items it can tell you will be interested in, based on what you have already bought. Not too interesting, not too challenging for the computer, not too intrusive. But what will happen when you get a smart cart at the store? That's one that will welcome your insertion of your shopper's card, and then tell you what your shopping list usually looks like so you don't forget anything, where today's bargains are (in other words, what the store manager is trying to offload), and the fastest route through the aisles so you can get everything you need. If this sounds like it could be a useful tool for you, and also sounds a little creepy because of all the information the store (and the cart) knows about you, it's just the beginning. You may well want to see what else those who are mining your personal information are up to by reading _The Numerati_ (Houghton Mifflin) by Stephen Baker. Baker is a business journalist who wants to let us know about a new reach of mathematics into our lives. There are no equations here, just stories of the mathematicians and computer geeks that use them to find and exploit patterns of our day-to-day existence. Baker has cast some light onto many facets of an arcane realm of number crunchers, and has written a book that is entertaining and often disconcerting.

You can decide that you do not want to have a shopper's card. You can also decide that you do not want a cell phone, you never want to purchase anything on a credit card, or you do not wish to use an internet search engine. If you do volunteer for such activities, the Numerati have you. You cannot help but leave a digital trail. Most of Baker's chapters involve his looking into a particular realm of number crunching, interviewing the geeks and mathematicians who are involved, describing what has been done so far, and explaining the prospects for the not-too-distant future. Perhaps the brightest prospects for data mining are medical. Patients will do nothing extra to deliver information; it will just be monitored passively. Imagine a bed equipped with sensors that would tell how many hours we are actually spending in it, or how much tossing or turning we do, or how many times we get up for a bathroom break and how much fluid is lost on each such trip. Maybe there will be magic carpet on the floor of an elderly patient's house; it could register weight gain, or a new peculiarity in gait, or a fall, or even if the patient has stopped moving around the house during the day.

Privacy concerns are valid; it remains to be seen how much each of us will have to re-think what privacy actually means. There could also be moral questions involved; if you could make a mathematical model of a pedophile, and your church or school screens job applicants using such a model, and the screen says a candidate is an 85% fit, what is the right thing to do (and, an entirely separate question, what will be the thing to do to minimize legal liability)? And that percentage fit - it's going to be what any Numerati have to put up with, because any prediction or pattern can only indicate not reality, not truth, but mere probability. Several of the boffins interviewed here say that as complicated as are the mathematical algorithms to turn people into data, the math is the easy part; it's the humans that are hard to figure out. It is surprising, too, how simple tasks are actually monumental; terrorist watch lists of mere names present a nightmare, as any non-terrorist traveler who has a similar name will tell you. Internationalizing such data is a horrendous task; the Chinese alone, for instance, spell Osama Bin Laden eleven different ways. Baker's brightly-written and enthusiastic book presents pleasing pictures of how our numbers will come up in the future, and emphasizes those without neglecting to mention the darker issues of data misuse. He even did his own little experiment that verified something information techs have known since the most primitive of electronic computers. He and his wife filled out questionnaires at a dating site, and were dismayed that the computer did not point them in each other's direction as potential matches. It turns out that Baker had mistakenly excluded women of his wife's age. The verification: garbage in, garbage out.