Machine Learning Model Interpretability

|Building a ML Pipeline : Part 5

Because sometimes the evaluation metrics wont be enough..

Until unless inherently interpretable models such as Linear Regression, Logistic Regression, GLM, GAM, Decision Tree etc. are used, interpreting a Black Box Model becomes a necessity for the built model to go into production.

This is the Fifth and the last article in the series Building a ML Pipeline, in continuation to below mentioned articles:

Part 1 : Building Blocks of a ML Pipeline

Part 2 : Reading the Data to Feature Engineering

Part 3 : Feature Selection

Part 4 : Machine Learning Pipeline

Wouldn’t it be nice if we could have some set of tools that can unsheathe and shed some light to the complexities of a Black Box Model which themselves are inherently non-interpretable. The tools described in this article help to understand anatomy of a Black Box Model.

1) Intro

Being the fifth and final article in the Building a ML Pipeline series, i would highly recommend you to go through the prior articles in the series, you can find the links above.

“..The Information Commissioner’s Office (ICO) is putting forward a regulation that businesses and other organizations are required to explain decisions made by artificial intelligence (AI) or face multimillion-dollar fines if unable to…” — techhq.com, Dec 2019

As rise of Machine Learning/AI has propagated it to penetrate into various aspects of the public life it has also amplified the need for regulations put forth by the governments on the same..

“..In February 2019 Polish government added an amendment to a banking law that gives a customer a right to receive an explanation in case of a negative credit decision…” — kdnuggets.com

And this has lead to various advances in Model Interpretability , a much under appreciated realm of the Machine learning universe. From as basic as Feature Importance to LIME, SHAP etc. Google launched Explainable AI in late 2019.

Difference between Interpretable and Black Box model.

This article deals with aspect of model interpretability i.e given a Black Box Model, which has been selected as a solution for a specific problem how do you go about making sense of the intricacies that go inside it?

Are you happy with the predictions that your model made, or do you want to know the Why the predictions were made.

FUN FACT : Nerium oleander, the sweetly scented killer. Some of Napoleon’s soldiers died in Spain when they used oleander sticks to roast meat..Boy!(Photographer)

If you have gone through the prior articles in this series, then the model that we selected to predict the price at auction an was RandomForestRegressor. A tree based ensemble Bagging model.

If the models were to be utilised by the bidder or the auctioneer to predict the price of a particular model at which it will be auctioned, then that becomes a high stake problem both for bidders and for the auctioneer as it involves money. With high stakes stems out the insect of human curiosity and start asking “Which features does this model find the most useful?” or “Why did it price that model at $500,000 when almost similar model was auctioned for $100,000??”.

There are two different set of model explainability tools at your disposal,

1. Model Dependent Explanation : As the name suggests, those explainability techniques which are available for a certain algorithm/model alone.
2. Model Agnostic Explanation : Techniques which can be leveraged on any Machine Learning model, irrespective of the model that has been used.

which explain three different type of Explainability :

1. Global : Explaining the entire model as a whole.
2. Local : Explaining individual predictions.
3. Per Feature : How does a feature behave w.r.t to the model predictions or the actual target variable.

Various methods of Model Explainability and corresponding type and the scope w.r.t each of them. The last method is not for Model Interpretability, but capture the interactions between the actual target variable and the features. Although i wont be touching on the last method in detail in this article, but to have an understanding how to implement them you can go through the Part 3 of this series.

Since this is in continuation to the prior article, assume that the trained RandomForestRegressor is present against the variablemodel in the environment.

Method 1 : Correlation @ Per Feature

Correlation is a measure of how two variable are associated with each other (Positive or Negative Corrrelation) and by how much (between -1 →+1). It is a Bi-Variate technique to measure the strength between two variables.

e.g Does the Advertising effect the Retail Sales? And if so by how much?

Correlations just tries to answer the question that wether a target variable Retail Sales is dependent on a feature Advertising or not and if it is then by how much and in what direction i.e Positively or Negatively?

There are different types of Correlations that can be employed to measure the association of target variable to a features, i have covered this topic in another article.

Method 2 : PDP & ICE Plots @ Per Feature

PDP : Partial Dependence Plot shows the marginal effect one or two features have on the predicted outcome of a machine learning model.

ICE : Individual Conditional Expectation Plot gauges on how the predictions will change with the change in the features value.

Given a dataset D with n observations and p predictor variables and y as response variable.

For the variable pᵢ of interest ,

• STEP 1 : For each instance in the dataset D, keeping other features constant, permute through all the values of pᵢ for a single instance and make predictions[So if pᵢ has 100 unique values, and there are 1000 datapoints, the new dataset becomes 100*1000 in rows].
• STEP 2 : Now, for predictions group-by the unique values of pᵢ(X axis in PDP Plot) and take the mean per group of unique values of pᵢ(Y axis in PDP Plot).

Negate the STEP 2 in pdp calculation, and then the plot generated would be representing each instance in stead of the averaged effect — Individual Conditional Expectation (ICE) Plot.

This article here has a much simpler explanation of how to interpret PDP and ICE plots.

PDP & ICE plot : Since this is rather computationally expensive to plot PDP and ICE plot, i have restricted this example to 1000 training samples.

PDP(Yellow Line) & ICE(Blue Lines) plot : To interpret this graph, Each Blue Line represent a particular instance in the dataset, how will the predictions change if all the other features other than Coupler_System are kept as it is and the Coupler_System variable is varied through its distribution. The Yellow Line is the average of the blue lines showing how the predictions are being affected wrt Coupler_System on an average and in what direction.

Method 3: LIME @ Local

Local Interpretable Model-agnostic Explanations : In Layman terms, what LIME does is given any model, be a complex one or a fairly simple one, it will try an answer the question - Why did this model predict a particular ŷ for a particular X?.

(Source)

Suppose we wanted to know why the highlighted red dot prediction was made. So LIME will sample data-points near that highlighted red data-point and fit a fairly simple model for eg Linear Regression or as in this case Logistic Regression and then see the weights across the feature space and the feature weights of this locally trained model will become our interpretation!

For LIME — We first need to build the Explainer, here its an Tabular Explainer since we are dealing with Tabular Data. Then use that explainer to explain a particular instance, there is a parameter num_features which basically tells the Explainer how many features to use to explain this instance.

LIME Explanations : Since we gave the parameter num_features = 5 in the model so here what we have are the top 5 features and how are they affecting this particular instance.

Method 4: ELI5 @ Global & Local

“Explain Like I’m 5” -is a package that incorporates various methods and techniques around model explainability.

ELI5 can be leveraged to find explanation for the entire model (Global) or for each prediction (Local). ELI5 also provides a neat feature for calculating PermutationImportance , I have discussed about that in Method 7.

ELI5 Global Interpretability : Explaining the weights of the trained model.

ELI5 Global Interpretability : Feature Importance for the model with Standard Deviation for each prediction.

ELI5 Local Interpretability : Calculating which feature were important for predicting the 899th index in the Test Data.

ELI5 Local Interpretability : Each measure reflects the magnitude and the direction in which each feature motivated 899th prediction. <BIAS> term that we see is nothing but the initial starting point for the regressor and then as we tread down the table we see how much each feature is affecting this prediction.

ELI5 Global Interpretability : Permutation Importance.

ELI5 Global Interpretability : Permutation Importance, a different form of Feature Importance with Standard Deviations. Permutation Importance is MUCH BETTER & PREFERRED over default Feature Importance.

Method 5: SHAP @ Global & Local

SHapely Additive exPlanations Values, motivated by the work done by Lloyd Shapley(1953) in Game Theory. The Shapley value is the average marginal contribution of a feature value across all possible coalitions.

The goal of SHAP is to explain the prediction of an instance x by computing the contribution of each feature to the prediction. The SHAP explanation method computes Shapley values from coalitional game theory.

SHAP Global Interpretation : First we need to calculate the Shapely Values for each of the instance in the dataset. Then plot a summary plot for the same…

SHAP Global Interpretation : From the summary plot, analysing the product_age column : lower the feature value (Blue) markings higher the impact of the feature to push the prediction in positive direction i.e lesser the product_age more the resale price, makes sense.

SHAP Local Interpretation : From the already calculated shapely values this will show effect of each feature value in predicting the target.

SHAP Local Interpretation : This plot tries to gauge each feature and by how much did it effect the prediction for eg Since the product_age was 6 years(relatively low for product_age) product_age feature pushed this prediction to the higher side by the width of that pink bar.

Method 6: Tree Visualisation @ Global

Since random forest is an ensemble method i.e it consists of multiple estimators (Decision Trees) which then at last contribute to the final prediction. We can not see all the trees at once but what we can do randomly pick from those trees and see how the tree is being built which can give some intuition about which features are coming out to be important and what are the good dissection value among each.

Code for Tree Visualisation

One of the estimators in RandomForestRegressor — A Decision Tree.

Instead of just visualising just the split points at each node and the number of samples traversing through, it would be more helpful to see the data traversing through that leaf and the split point at which each further split would be made.. dtreeviz is just the library for that (Authors Talk).

Visualising a single Decision Tree wrt to values at each node. Beware of the size of XTrain and YTrain as rendering for higher size can be computationally expensive also the depth for this decision tree singleEstimator is kept at 5 for ease of interpretation.

One of the estimators in RandomForestRegressor — A Decision Tree.

Method 7: Feature Importance@ Global

As the coefficients for Linear Regression model shed light on how important a feature might be for the target variable. Similarly RandomForestRegressor has something called as Feature Importance for the trained model.

There are multiple ways in which you can calculate Feature Importance of an Ensemble Based Tree model :

Of these above three, sklearn implements the default method which is although the fastest but is rather flawed as argued in this article. Drop-Column although most computationally expensive than the rest but it is the “Ground Truth” of Feature Importance.

You can find a detailed explanations of these three methods in this article here.

To understand how default RF Feature Importance are calculated internally, look at this SO Conversation for a better understanding.

Default Feature Importance

Feature Importance

“..It’s also worth pointing out that feature importances should only be trusted with a strong model. If your model does not generalize accurately, feature importances are worthless. If your model is weak, you will notice that the feature importances fluctuate dramatically from run to run..” — Article

FUN FACT : Did you know that we have better maps for Mars than we have for our ocean bed?(Photographer)

Phew! Quite frankly writing this was more intensive than i thought it would be.

As always, I welcome any feedback, you can put that in comments below or reach out to me on LinkedIn, would love to connect.

See you next time..!Au Revoir!🙋🏽‍♂️