Performance with Transparency – successful credit risk modelling
Part 3 of the Modern Model Management Series
In anticipation of the upcoming 19th Credit Scoring and Credit Control conference organised by the Credit Research Centre of the University of Edinburgh Business School, the first article in this series looked at how credit risk modelling has evolved over the years and what the future might hold. I concluded that success in modern credit risk management requires balancing;
(i) automation with human oversight
(ii) innovation with governance, and
(iii) performance with transparency
Article 2 addressed points (i) and (ii) above, now let’s dive some more into ‘performance with transparency’.
Just because we can doesn’t mean we should
It has become relatively easy to develop complex M.L. models such as RF and XGB models (for example, in Paragon’s Modeller software tool), and there is much hype about the power of such models. But are they always needed?
For some, there's an ongoing temptation to equate complexity with sophistication and performance. More layers, more algorithms, more data - the assumption is that these always lead to better models. But what if the pursuit of complexity is actually undermining the very purpose of our models?
Transparency is not a checkbox to be ticked. It's a fundamental requirement that goes to the heart of how organisations make decisions, manage risk, and maintain trust with all stakeholders - from regulators to customers.
The hidden costs of complexity
Every layer of complexity in a model comes with a hidden cost. It's not just about the computational resources or the development time. The true costs are spread across the whole model lifecycle:
Development becomes more challenging. More complex models require more specialized skills, longer development cycles, and more intricate validation processes. What might have been a straightforward development becomes a months-long project requiring multiple experts.
Deployment introduces additional friction. More complex models often require more sophisticated infrastructure, specialised hosting environments, and more complex integration processes. The path from model conception to operational use becomes increasingly difficult.
Maintenance becomes a constant challenge. Each layer of complexity introduces more potential points of failure, more potential for drift, more requirements for monitoring and adjustment. What was once a manageable model becomes a resource-intensive system demanding continuous attention.
Understanding becomes exponentially more difficult. As models grow more complex, the ability of stakeholders to understand the decision-making process reduces. A model that cannot be understood is a model that cannot be trusted.
Simplicity as a strategic advantage
The most effective models are often those that strike a delicate balance between predictive power and interpretability. This doesn't mean sacrificing performance, but rather being intentional about where and how complexity is introduced. Consider the parallels in other fields. In engineering, the most elegant solutions are those that solve complex problems with seemingly simple approaches. As my Maths teacher from school used to say, “KISS (keep it simple, stupid)”.
Building explainability into the model, not after the fact
Arguably, explainability is not a post-hoc exercise to be addressed after a model is developed. It should be a fundamental constraint in the model development process itself. This means:
Choosing algorithms that inherently provide clear decision paths
Designing model architectures with transparency as a primary consideration
Implementing constraints that force the model to provide clear rationales for its decisions
Some machine learning approaches are inherently more interpretable. Decision trees, for instance, provide a clear decision path that can be easily traced. Linear models offer straightforward coefficient interpretations. These should not be dismissed in favour of more complex approaches without compelling reasons.
Before introducing additional complexity, organisations must conduct a comprehensive assessment of the benefits. What tangible performance improvement does the additional complexity provide?
Is it a marginal gain of 1-2% in predictive accuracy?
Does it truly represent a meaningful business improvement?
What are the full lifecycle costs?
Development time and expertise required
Deployment infrastructure
Maintenance and re-builds
Training and education for stakeholders
Potential regulatory scrutiny
Reduced model interpretability
Regulatory and ethical considerations
Regulators are increasingly demanding not just model performance, but model understanding. A complex model that cannot be explained is becoming a regulatory liability. In credit risk, where decisions directly impact individuals' financial opportunities, the ability to provide clear, understandable rationales is not optional - it's essential.
Moreover, there are ethical dimensions. A model that cannot be explained is a model that cannot be assessed for potential biases or unfair practices. Complexity can become a shield that masks potential discriminatory decision-making. The next article in this series will explore fairness and potential bias in more detail.
A pragmatic approach to model complexity
By using modelling tools such as Paragon’s Modeller it is possible to develop very simple through to very complex models. This enables the following pragmatic approach to determining the appropriate level of complexity required for a particular model build and model use.
Start with the simplest model that meets performance requirements
Only introduce complexity where there is a clear, measurable benefit
Continuously reassess the model's complexity against its performance and explanatory power
Prioritise stakeholder understanding at every stage
One of our Modeller clients, a UK Bank, recently conducted some research on the complexity of their Application scorecards for one of their portfolios. They compared both Random Forest and XGBoost models to their current WoE logistic regression scorecards. Although the Tree ensemble modelling approaches showed some uplift, it provided the evidence that their current scorecards are almost as strong whilst providing strong benefits in the areas of explainability and transparency, deployment, monitoring and maintenance.
Conclusion
Successful credit risk modelling is about creating models that are powerful, yet transparent and simple in their ability to communicate decisions. The future of credit risk modelling requires navigating the delicate balance between predictive power and human comprehension. Getting this balance right will lead to models that can most effectively communicate their reasoning, providing the essential transparency and trust.
This article is part of a series exploring modern approaches to credit risk management. For more insights on how these principles can be applied in your organisation, we'd be happy to discuss your specific challenges and objectives. Please get in touch.