A Tale of Three Models: SHAP and LIME Side by Side

Last Friday night, I was staring at a SHAP summary plot for an XGBoost model in my office near La Jolla. The pattern of dots on the screen was different from what I had expected, and by the time I finished tracing why the discrepancy existed, it was past 2 AM. That's when I remembered the explanation I'd gotten from running LIME on the same model. The two methods were telling entirely different stories about the same prediction.

So I decided to apply both SHAP and LIME to three models I had actually worked on recently and document what each one revealed and what each one missed.

Patient Readmission Prediction Model

This was a project I did with a healthcare startup in San Diego. It was a gradient boosting model with around 40 features. When I extracted SHAP's global feature importance, the interval between discharge and the first outpatient visit had the strongest impact. It made intuitive sense. But when I used LIME to explain individual patient cases, this feature often didn't rank near the top. Instead, LIME picked up on the presence or absence of specific drug combinations as more important. When I looked into it later, I found that those drug combinations were a strong local signal appearing only in certain patient subgroups, something that got diluted into the average under SHAP's global perspective and was easy to overlook.

Ingredient Pairing Experiment

This one was entirely a personal project. While exploring the intersection of cooking and machine learning (a topic I love), I built a small model that predicts pairing compatibility using ingredient chemical composition data. I used a random forest, and with few features (12) and a small dataset, SHAP and LIME told nearly the same story. The number of shared flavor compounds was overwhelmingly important, and both methods showed this clearly. This was a case where I experienced firsthand that when the model is simple and the feature space is small, the difference between the two tools shrinks noticeably.

There was one interesting wrinkle on the LIME side, though. For certain pairings (unexpected combinations like cilantro and chocolate), LIME's explanations were unstable. The ranking of feature weights changed every time I ran the same instance. This comes down to the randomness of the perturbed samples LIME generates for local approximation. The closer a data point sits to a decision boundary, the worse this variability gets.

Customer Churn Prediction

The third was a B2B SaaS project using a neural network with over 80 features. This is where SHAP (specifically DeepSHAP) really shone. It could decompose feature interaction effects using SHAP interaction values. SHAP showed that the interaction between login frequency and the number of support tickets was more important for predicting churn than login frequency alone, an insight that proved useful when explaining things to the client. LIME couldn't capture this kind of interaction structure. On the other hand, even with this complex model, LIME generated explanations for individual predictions quickly, and when it came to showing non-technical stakeholders "why this customer was classified as at risk of churning," LIME's simple linear approximation was actually better suited for communication.

So When Asked Which One to Use

I use both. Truly. After working through these three projects, here's the rough framework I've settled on. When I want to dig into global feature importance and interactions, and when a SHAP implementation exists for the model architecture, I reach for SHAP first. When I need to explain individual predictions quickly, especially when the audience is non-technical, LIME gives more intuitive results.

Whichever one you use, though, don't take the results at face value. Make a habit of cross-checking whether the explanations align with your domain knowledge. That late night in La Jolla, when the SHAP plot looked strange and I didn't stop there but compared it with LIME, that was the starting point that led me to uncover the hidden local patterns in the patient readmission model.