The Limits of AI in Credit Decisions: A Practitioner’s Honest Assessment After Several Decades in the Data

AI is here. We all use it. We see its power. We also see its limits. This is not about theory. This is about practice. My decades in credit, working with thousands of firms, show me this. AI is a tool. A powerful one. But it is not the decision maker. We are.

We want certainty in credit. AI promises it. It sifts through more data than ever before. It finds patterns invisible to the human eye. Descriptive analytics tell us what happened. Diagnostic analytics explain why. This is where AI shines. It scales our understanding. It spots trends across thousands of companies. We can see where risk is building. We can see why past defaults occurred. This level of insight is transformative. It grounds our intuition. It refines our questions.

But certainty is not absolute. AI models are built on past data. The past is not always prologue. Economic shocks happen. Market dynamics shift. Our models may not capture black swan events. They may not predict shifts in consumer behavior. Predictive analytics help us forecast. They tell us what is likely to happen. They are our best guess. But they are still guesses. We must understand their confidence levels. We must know their error margins.

The “Black Box” and the Demand for Answers

Regulators are asking tough questions. We are too. As practitioners, we must explain our decisions. The CFPB is clear. There is no special exemption for AI. We cannot hide behind complex algorithms. We must provide specific reasons for adverse actions. This is where the “black box” problem bites. Some AI models are incredibly complex. Even their creators can’t fully explain every single decision. They are powerful, but opaque.

This opacity creates a clear tension with our job. We need to know why a credit application was denied. We need to articulate that reason to the applicant. We need to be able to defend it if challenged. Prescriptive analytics aim to tell us what to do. They offer recommendations. But if the recommendation is to deny credit, we need to be able to say why. An AI might say “credit score too low.” But why is it too low? What specific factors contributed? This requires a level of detail AI often struggles to provide without significant human effort.

This is not just a regulatory headache. It’s a business imperative. Building trust means transparency. Customers deserve to know. Our own internal risk teams need to understand the drivers. We must be able to audit our own decisions. The EU AI Act, classifying credit scoring as high-risk, reinforces this. Mandatory bias audits. Strict human oversight. These aren’t optional extras. They are structural requirements.

The Trade-off Between Model Complexity and Explainability

We see this trade-off daily. Simpler models are easier to explain. Linear regressions. Decision trees. We understand their logic. We can trace the inputs to the outputs. Advanced AI, like deep neural networks, can achieve higher accuracy. They can uncover subtle, non-linear relationships. But their internal workings are arcane. Even statistical measures of feature importance can be misleading in complex ensembles. This leaves us vulnerable. Vulnerable to regulatory scrutiny. Vulnerable to our own lack of understanding. We lead with data, yes. But we must also lead with clarity.

Human Oversight: Not a Button to Press

The idea that AI can automate all adverse action explanations is a myth. Human review is not a checkbox exercise. It is a fundamental part of responsible decision-making. We review exceptions. We interpret edge cases. We consider qualitative factors. AI has not yet mastered empathy. It cannot assess a borrower’s character. It cannot understand a genuine business pivot. These are human judgments. They require experience. They require context.

The Perils of “Alternative” Data and Embedded Bias

We are all exploring “alternative” data. It promises a broader view. It can show us creditworthiness where traditional data is thin. But this is where AI can amplify existing societal inequalities. Studies are showing this clearly. AI models using social media or browsing history can penalize low-income and minority borrowers. These variables can act as proxies. They can inadvertently discriminate. This is not malice. It is often an unintended consequence of correlation. The AI finds patterns. It doesn’t understand fairness inherently.

This digital redlining is a serious risk. It is a risk to our reputation. It is a risk to our compliance. It is a risk to the principle of fair credit access. We must be incredibly vigilant here. We need to understand the data we feed these models. We need to test them for bias rigorously. Descriptive analytics can help us see if there are disparities in outcomes. Diagnostic analytics can help us uncover why those disparities exist. But the mitigation often requires manual intervention. It requires us to question the data itself.

Unconscious Bias in Data Selection

We must ask: what data did we choose? Why did we choose it? Was it chosen because it was readily available, or because it was truly informative about credit risk? Are we unintentionally selecting data points that correlate with protected characteristics? Our training data matters. If the past was biased, our AI will likely learn those biases. This demands a proactive approach. It means actively seeking out and mitigating bias. It means challenging the data.

The Evolving Definition of “Fairness”

What does “fairness” mean in credit? Is it identical outcomes for all groups? Or is it identical opportunity? The legal and ethical landscape is evolving. AI models must be adaptable. They must be interpretable enough for us to intervene when fairness is compromised. This is not simple. It’s a continuous process of evaluation and adjustment. Leading here means setting clear standards for fairness.

The Supply Chain of Risk: Beyond the Financial Statement

Our world is interconnected. Credit risk extends far beyond the balance sheet. Supply chain intelligence is crucial. AI can help us see these linkages. It can track disruptions. It can identify concentrations. It can predict the ripple effects of a supplier failure. Predictive analytics here are vital. They can show us how a problem with one entity can impact another. This is where AI excels at processing vast, unstructured information. News feeds, shipment data, geopolitical events.

But AI cannot replace on-the-ground knowledge. It cannot fully grasp the nuances of a long-standing business relationship. It cannot intuitively assess the resilience of a management team under duress. These are human insights. They are vital for understanding complex, real-world risk.

Bridging the Gap Between Data and Relationship

We see a supplier failing. AI can flag the risk. It can quantify the potential exposure. But understanding why they are failing, and what mitigating steps can be taken with that specific supplier, often requires human interaction. A conversation with the supplier. A conversation with their key customers. This human element is irreplaceable. Our decisions are informed by data, but often cemented by relationships.

The Limits of Quantifying Qualitative Factors

How do you quantify leadership, or a company’s culture of compliance? AI is good at numbers. It’s less good at ethos. We need to combine AI-driven insights with our own experience. This is where we hold the tension between leading and collaborating. We lead with the data AI provides. We collaborate with our teams, and with our clients, to interpret it.

Decision Intelligence: AI as a Partner, Not a Replacement

The true value of AI in credit decisioning is not automation. It is augmentation. It is about enhancing our own decision-making capabilities. Decision intelligence is the framework. It combines data, analytics, and human judgment. AI provides the powerful analytical engine. Descriptive analytics show us the landscape. Diagnostic analytics reveal the forces at play. Predictive analytics forecast potential futures. Prescriptive analytics suggest actions.

But the ultimate decision rests with us. We are the decision makers. We are responsible. Our role is to ask the right questions. To interpret the answers. To inject human wisdom. We must guide the AI. We must ensure it serves our objectives.

The Art of Asking the Right Questions

AI can generate endless reports. It can surface countless correlations. But without intelligent questioning, this can be overwhelming. Our experience shapes our questions. We ask: “What could go wrong?” “What assumptions are we making?” “Does this still feel right, despite the numbers?” These are questions AI cannot formulate for itself.

Integrating AI Insights into a Holistic View

Consider a large commercial loan. AI can analyze financial statements, market trends, industry benchmarks. It can flag potential risks in the supply chain. But it cannot, on its own, assess the intangible value of a long-term customer contract or the impact of a new competitor entering the market. We take the AI’s output. We weave it into our broader understanding. We discuss it. We debate it. This iterative process creates a more robust decision.

The Enduring Role of Human Judgment and Expertise

After decades in this field, one truth remains constant. AI is a phenomenal tool. It expands our reach. It deepens our insight. It processes scale we could only dream of before. But it has not, and likely will not, replace human judgment. Our ability to reason, to empathize, to understand context, to uphold ethical standards—these are our superpowers.

We must guide AI. We must ensure it aligns with our values. We must be prepared to override it when necessary. The CFPB’s stance is clear. The EU AI Act is clear. Responsibility ultimately lies with us. This is not a burden. It is an opportunity. An opportunity to make better, fairer, more informed decisions. Our experience is not obsolete. It is essential. It is the bedrock upon which AI builds. We lead the charge. We integrate the intelligence. We make the final call. This is the reality of AI in credit, today and for the foreseeable future.