Model Risk Management: Quantitative and qualitative aspects
In recent years there has been a trend in financial institutions towards greater use of models in decision making, driven in part by regulation but manifest in all areas of management.
Model Risk Management
In this regard, a high proportion of bank decisions are automated through decision models (whether statistical algorithms or sets of rules).
First, the last few years have seen an increase in the use of automated electronic platforms that execute trade commands which have been pre-programmed by time, price or volume, and can start without manual intervention, a system known as algorithmic trading. As an example, an automated trade command that took place on May 6, 2010 resulted in a 4,100 million-dollar “flash crash” of the New York Stock Exchange, which fell more than 1,000 points and recovered to the same value in only 15 minutes.
Second, partly encouraged by Basel regulations, banks are increasingly using decision models (consisting of statistical algorithms and decision rules) in their origination, monitoring and credit recovery processes. Thus, whether or not a loan is viable is determined by estimating the probability of default (PD) of the client. Similarly, banks monitor customer accounts and anticipate credit deterioration using automatic alert models, pre-classify customers and determine their credit limits; and, in credit collections, they develop statistical profiles of delinquent customers in order to apply different recovery strategies.
In the commercial area, customers are able to select a product’s characteristics (loan amount, term and purpose, insurance coverage, etc.) and the system makes a real-time decision on viability and price. In many cases, the model asks the customer a number of questions and proactively makes the offer that best suits the customer (doing this manually would be a slow and complex process).
The use of valuation models for products and financial instruments has become widespread in financial institutions, in both the markets and the ALM business. Some classic examples are Black- Scholes, CAPM and Monte Carlo valuation models.
Another area where the use of models is more and more frequent is fraud and money laundering detection. Bank and regulators alike use models that identify fraudulent or money laundering-oriented transactions, which requires combining statistical customer profiling models (know your customer - KYC), transaction monitoring rules and black lists.
Also, customer onboarding, engagement and marketing campaign models have become more prevalent. These models are used to automatically establish customer loyalty and engagement actions both in the first stage of the relationship with the institution and at any time in the customer life cycle. Actions include the cross-selling of products and services that are customized to suit the client’s needs, within the framework of CRM.
Other examples include the calculation of capital charges for all exposures (credit, market, operational, ALM, etc.) through their individual components; the quantification of a bank’s current liquidity position, projected under different scenarios; the projection of the balance sheet and income statement and the use of stress testing models6; or the modeling of many key components in business planning and development, such as optimal bundle, customer and non-customer income or churn.
The use of models brings undoubted benefits, including:
- Automated decision-making, which in turns improves efficiency by reducing analysis and manual decision-related costs.
- Objective decision-making, ensuring that estimated results are the same in equal circumstances and that internal and external information is reused, thus leveraging historical experience.
- Ability to synthetize complex issues such as a bank’s aggregate risk.
However, using models also involves costs and risk, some of which are the following: - Direct resource costs (economic and human) and development and implementation time.
- The risk of trusting the results of an incorrect or misused model. There are specific and recent examples of this which have resulted in large losses.
Model risk may thus be defined as «the potential for adverse consequences based on incorrect or misused model output and reports».
Model error may include simplifications, approximations, wrong assumptions or an incorrect design process; while model misuse includes applying models outside the use for which they were designed9.
Model risk thus defined is potentially very significant and has captured the attention of regulators and institutions, whose approach ranges from mitigation via model validation to the establishment of a comprehensive framework for active model risk management.
In the more advanced cases, this active management has been formulated into a model risk management (MRM) framework that sets out the guidelines for the entire model design, development, implementation, validation, inventory and use process.
This is substantiated by the fact that regulators, particularly in the U.S., have started to require such frameworks – as stated in the guidelines issued by the Federal Reserve System (Fed) and the Office of the Comptroller of the Currency (OCC ) – which are serving as a starting point for the industry.
Regulations do not discuss model risk quantification aspects in detail, except in very specific cases relating to the valuation of certain products, in which they even require model risk to be estimated through valuation adjustments (model risk AVAs ) that may result in a larger capital requirement or in the possible use of a capital buffer for model risk as a mitigating factor in a broader sense, without its calculation being specified.
Against this background, this study aims to provide a comprehensive view of model risk management: its definition, nature and sources, related regulations and practical implications. With this in mind, the document is structured in three sections that address three goals:
- Introducing model risk by providing a definition, analyzing its sources and summarizing the most important regulations on the subject.
- Describing a desirable framework from which to approach model risk management in a practical way and based on examples seen in financial institutions.
- Advancing model risk quantification (and its potential practical application) through a quantitative exercise that will illustrate the impact of this risk.
For more information, click here to access the full document in pdf (also available in Spanish and Portuguêse).