11.1 Understanding machine learning

There are many differences between machine learning and traditional methods of statistical inference. Below are the takeaway points worth revisiting in group discussion.

First, considering machine learning and statistical analysis as antipodes is wrong. The two methods overlap quite a lot. They use similar concepts and ideas, albeit as we have learned in Chapter 11, the terminology varies.

Second, the most significant difference between the two is theoretical. While statistical inference is by and large about understanding the population based on a sample, machine learning aims to understand the data, to learn meaningful patterns and association from it, and to devise models with the highest possible predictive accuracy. These aims are not necessarily in direct contradiction with statistical inference, but it is true that learning algorithms and classical statistical models treat data in fundamentally different ways.

Third, it can be challenging to come up with examples of research better suited for machine learning. The most obvious example is when our theoretical understanding of a given area of research is limited and we expect to benefit from insights derived from the data. Another obvious example is when predictive accuracy matters more than theory-testing. It is common in social policy and practice-oriented business research. Machine learning is also efficient when it comes to uncovering complex, non-linear relationships (see Exercise 11.3). If the researcher suspects that a linear model is conceptually inadequate, she has a good reason to turn to machine learning. By contrast, theory-driven research with a clear set of hypotheses is better suited for traditional methods of statistical analysis. Machine learning is prone to unpicking theoretically irrelevant relationships, an outcome not desirable in theory-driven studies. Having said that, in every piece of research there is room for combining both methods so long as it is done logically.

11.2 Why are black-box algorithms problematic?

Two main problems arise from black-box machine learning algorithms.

First, black-box algorithms are agnostic about causality. An algorithm will rank a predictor high if it helps to produce accurate predictions regardless of whether there is a genuine causal effect on the outcome. When fitting regression models, researchers would normally give a thorough consideration to underlying causal relationships.

Second, many learning algorithms, including random forest, do not quantify effect size. Nor do they show whether the effect of a predictor on an outcome is positive or negative. This is in contrast to regression analysis, where one can get a sense of the direction and magnitude of the relationship between independent and dependent variables by simply looking at regression coefficients. In regression analysis, there are many ways by which researchers can quantify the effect of one variable on another. In logistic regression, odds ratios are easily interpretable measurements of effect size.

11.3 Interpreting machine learning algorithms

Three conclusions stem from the partial dependence plot below.

First, seems to have a positive effect on turnover. That is, the older an employee the more likely she is to quit.

Second, while the above conclusion is broadly correct, the effect of age on turnover is a little more nuanced. The graph shows an inverted U-shaped relationship where age increases the likelihood of labour turnover but only to a point, between 20 and 40 years old. 40 years old seems to be an inflection point beyond which as workers get older, they are less likely to quit.

Third, the x-axis indicates that most of the data is between 27-28 and 45 years old (see the small bars originating from the x-axis). That’s something one has to bear in mind as model’s predictions are only reliable within the boundaries of the data. Fewer observations in a particular group (e.g. those approaching retirement or those younger than 25) would normally mean that we should exercise caution when looking at predictions in such areas.