Managing model risk

Elgin Chetsanga

The need for models.

Financial models have become an important part of day-to-day business in many financial institutions the world over.

The outputs of these models are used by various stakeholders including regulators, boards, senior management to make decisions that have a bearing on the survival of their organisation. Models types and complexity are increasing. The reliance on models has now given rise to a concern called model risk.

Essentially users of model outputs are questioning the reliability of their models and luckily risk management has an answer for this question.

The history of model risk

Model outputs are used to make many far-reaching decisions and model risk is a legitimate concern.

In 1998 a company called Long term Capital Management based in the United States used models that considerably under-estimated the risks of a profound economic crisis, due to an insufficient data-window and a lack of stress testing. Based on this model output, Long term Capital Management ended up requiring a recapitalisation of $3,65 billion.

Another instance where we see the power of models is in the 2008 financial global financial crisis. There was an over-reliance on a model called the Gaussian copula model, which eventually led to mispricing of the risk of collateralised debt obligations.

What is a model

Researchers define a model as a method, system or approach that applies statistical, economic, financial or mathematical theories, techniques and assumptions to process inputs into quantitative outputs.

The model processes data inputs into a quantitative-estimate type of output.

A model is expected to have three separate components which are the input, the processor and the results/output. Examples of models include Probability of Default models, Loss Given Default model etc.

However, not everything that has an in input-processor — output architecture qualifies as a model.

Models should be distinguished from End User Developed Applications (EUDAs). EUDAs are applications, which are developed and maintained by a user rather than a specialised unit with the relevant depth and expertise in the area. Some EUDAs may include simple applications that leverage code such as SQL, Python or ACL, spreadsheets, user databases etc.

What is model risk management?

A PWC Survey conducted in 2022 defines model risk as the potential loss an institution may incur, because of decisions that could be principally based on the output of internal models, due to errors in the development, implementation or use of such models.

There are various sources of model risk. The same survey also attributes sources of model risk to the incorrect identification, erroneous model implementation in a system; unreliable or incomplete data; uncertainties about statistical and mathematical methods in place; inaccurate calibrations; model misuse; incorrect interpretation of model results; inappropriate assumptions stemming from the use of upstream and downstream models; incomplete or inaccurate model inventory and more additional sources.

In order to manage the slew of model risk sources a model risk management framework should be created and implemented by an organisation.

The Model Risk Management (MRM) framework helps to have oversight over the entire model life cycle by deploying a governance framework with clear roles and responsibilities. An MRM is critical and also helps people to build awareness which is important in eventually mitigating model risk.

Model risk regulation

Before the 2008 financial crisis, most countries did not have regulation on MRM. Financial services industry MRM was driven by best practices rather than regulatory standards. However, the crisis regulators around the world began to issue guidance and regulations.

Ever since then, regulation around model risk has been growing in scope. In the United States, the Federal Reserve Board (FRB) and Office of the Comptroller of the Currency (OCC) issued joint guidance in 2011 on Model Risk Management targeting large financial institutions which was further expanded to cover all banks in 2017 with more than $1 billion in assets. Similar guidance has also been issued by regulators globally.

How is model risk management done?

Many MRM frameworks exists but they all seem to have the same core aspects which Model Risk Identification, Model Risk Measurement and Model Risk Mitigation.

These aspects of Model risk management need to be executed in the context of the Model Life Cycle.
The model life cycle starts off from the model origination stage which is when a need for model is identified by users.

Model development then follows and in this phase a model is designed and documented. Model testing follows model development and this phase includes testing the models to ensure that the model is performing the way it is expected.

A key phase after model testing is the independent review of the model by parties which are not involved in the model development. This allows an objective clear eye review of the model to occur before model is moved on to model approval stage.

The model life cycle then moves on to the model implementation This stage or phase involves deploying the model for use in real life environment. This means that the model outputs are now used for decision making.

The model will need to be monitored during this stage for performance. Models, which reach end of life cycle will then move to the last stage of life cycle which is model retirement.

Model risk is complemented through a governance structure which utilises tools such as policies and procedures which define the relevant risk management activities. Human resources will need to be allocated in line with the three line responsibilities set out in the policies.

In summary

While Model Risk Management is relatively a young discipline, it has become very central in this technological age.

The world is on a rapid digital expansion drive and the role of model risk management is undergoing significant change as the dynamics that surround it also evolve. Companies should proactively identify, measure and monitor the risks which are linked to models in this modern age.

Elgin Chetsanga is a head of risk at a local financial institution. He writes in his personal capacity. Elgin can be reached on [email protected]