Weapons of Math Destruction
The book ‘Weapons of Math Destruction’, or WMD for short, by Cathy O'neil has been praised quite thoroughly, so today I wanted to take a peek at what makes this book so special, and why it might, and most definitely is, be in any econometrician’s or data scientist’s interest to read it. In addition, it is a great book to recommend to family and friends who might not like math very much, but would be extremely engaged by the social topics and ethics handled in the book.
The book follows a relatively simple format, where we are first introduced to a general definition of a model, and how this can be anything from a piece of code, a complex mathematical formula, to a simple recipe, to a diet plan. Each and every model depends on input data, a certain step-plan to analyze this data, and give an outcome in a desired form, where a good model manages to incorporate a corrective feedback loop in this, i.e. any time the model does not perform well, it can adjust to perform better in the future. The first examples of WMDs are discussed here, where a questionnaire on a convict's past on seemingly unrelated items, such as whether drugs or alcohol were consumed during the activity, can very quickly lead to a powerful proxy for race.
This introduction to the topic is followed by madam O’neil’s own journey of disillusionment, where her jobs in finance and e-commerce showed that from the largest to the smallest players in the industry, everyone was using data, and almost everyone was also building Weapons of Math Destruction. The rest of the book handles the different applications of data in everyday lives, how these function, and ultimately shows that old biases, such as racism and xenophobia, end up being encoded in mathematical and statistical models, which are expected to provide some kind of unwavering truth. The WMDs show up almost everywhere, from teachers’ receiving bogus performance scores, to colleges’ tendencies to focus far more on performance statistics than actual learning, to for profit colleges’ setting up propaganda campagnes to trap people in debt for mediocre degrees, to police models suffering from severe overfitting, to how ‘e-scores’ constructed from all manner of data can affect one’s insurance. The degree to which these WMDs can affect one’s life is astounding, and tend to either have a small upside but an extremely large downside, or only an extremely large downside. The mathematical algorithms used to divide us up into smaller and smaller groups to market everything from colleges to car insurance to us codify beliefs not of today, or of the future, but of the past and the past only, with all the nasty connotations involved.
Fortunately, there is a light at the tunnel, as each of these algorithms, and many more, have a single thing in common: they can identify the vulnerable in our society. Then, when one’s needs and wounds are exposed through the data, they can be helped back on their feet, and not be left to the devices of institutions focused on profit. A potent example of this is the work of a Harvard PhD, who created a model which can scan vast networks of supply chains, and find signs of forced labour in them, a great example of being able to find the weak and exploited and help them.
This change will not happen quickly however, and certainly not without regulation. Models and algorithms of any kind should be able to be held to account, and should not remain a black box for all but those with vested interests in keeping it as such.
This book took me quite flat footed. Initially, I thought it was simply about the tendency of certain models to create blowups, or maybe even as a how to guide on how to be an evil data scientist, which it could be interpreted as, but it turned out to be far more interesting than that. This book takes the doubts and oddities of modelling, and clearly shows how even the most innocuous of models can lead to extremely divisive and unethical outcomes. This is the book for any data scientist who wants to see the practical ethics of their field, and for anyone else with an interest in data science and how it impacts our society. In summary, I can only describe this book as the one that any econometrician should have read in their ethics courses. Weapons of Math Destruction provides concise and exciting material both in practicality and ethical dilemmas on the use of data and complex models, where I consider it of the essence to instill its main message in every freshly minted econometricians, data scientists, and OR specialists: ‘Am I creating a WMD?’
Our well known Henk Tijms was mentioned in a newlsetter for math teachers, about the articles he wrote for the SECTOR.
Interested? Read it in this newsletter.
