4/28/17 - The Role of Internal Audit in Model Risk Management
Home > News & Insights > Insights & Publications

4/28/17 - The Role of Internal Audit in Model Risk Management

Financial services firms have begun expressing concern about supervisory expectations for the internal audit function’s role in model risk management. Supervisory expectations around MRM continue to rise — including those for model audit — as part of a broader regulatory effort to achieve effective control of model risk throughout the financial system.

Supervisory expectations vary depending on firm size, complexity, institution type, and regulatory jurisdiction. Similarly, internal auditors may design the model-audit function in different ways depending on firm culture, variation among MRM frameworks, or the decisions made by representatives of the firm in carrying out their roles. Although a range of model-audit practices can be observed throughout the industry, many firms have arrived at common questions regarding leading practices and their implementation.

At a Promontory roundtable event, industry and regulatory attendees expressed their mutual desire to maintain a dialogue to ensure that the views of financial institutions and their supervisors are well understood. The best window into supervisory expectations for the role of internal audit in MRM is interaction with financial regulators. Additionally, model risk managers and their internal-audit counterparts should be familiar with the various regulatory documents pertaining to both MRM and internal audit.

In North America, the key documents to consider include the following:

  1. The Federal Deposit Insurance Corp.’s Supervisory Insights on model governance
  2. The Federal Reserve Board and Office of the Comptroller of the Currency’s “Supervisory Guidance on Model Risk Management
  3. The Federal Housing Finance Agency’s advisory bulletin on MRM
  4. Canada’s Office of the Superintendent of Financial Institutions’ “Enterprise-Wide Model Risk Management for Deposit-Taking Institutions
  5. The “Federal Reserve Guidance on the Supervisory Assessment of Capital Planning and Positions for LISCC Firms and Large and Complex Firms
  6. The Federal Reserve’s “Supplemental Policy Statement on the Internal Audit Function and Its Outsourcing

A major theme in these documents is that model validators must be independent of model developers, and that internal audit must be independent of both. In practice, banks typically employ a “three lines of defense” framework for the implementation and oversight of MRM activities. In this framework, model owners, developers, and users are the first line of defense, model validation is the second, and internal audit is the third.

Due to the level of independence required of both model risk managers and model auditors, aspects of their roles can sometimes be confused. Model owners, meanwhile, are the key stakeholders throughout a model’s development and provide an initial level of assurance that a model is sound. Model risk managers investigate a model’s fitness for use through an evaluation of both conceptual and technical soundness, help maintain the model inventory, and guide the evolution of model-related policies and procedures. Model auditors are expected to evaluate the effectiveness of the MRM framework and ensure that all model-related policies and procedures are followed. Model auditors must be granted sufficient access to model validation, whether it is conducted by internal or external resources, and may be viewed as even more critical for firms that outsource a significant amount of their model-validation work. In some cases, model auditors may replicate model-development or -validation activities, but model auditors are not expected to replicate complete model validations or participate in the model-validation process.

Although an effective model-audit function should not duplicate the second line of defense, auditors do need to review a model’s development and validation documentation; therefore, a certain degree of technical proficiency is required. Model-audit units are often challenged to find staff with strong business knowledge and the requisite balance of quantitative and qualitative skills. In order to overcome this staffing issue, many banks cultivate talent internally through the sponsorship of employee training and rotational programs. Banks also frequently team model auditors with business-line auditors to promote collaboration and the exchange of knowledge and skills. Additionally, external consultants are often hired as a source of stopgap staffing or to provide specialized expertise. In order to retain high-performing model-audit team members, it is important that staff be empowered with sufficient authority to effectively do their jobs.

Model auditors frequently face the additional challenge of determining the breadth and depth of the audit plan. A complex financial institution with over $250 billion in assets may have more than 1,000 models in its inventory. A similarly sized noncomplex firm may use approximately 500 models. Although smaller firms generally rely upon fewer models, most firms throughout the industry have developed a significant reliance on modeling. Considering the number of models in place at most financial institutions, it is not practical for internal audit to monitor the enforcement and effectiveness of controls through every stage of each model’s life cycle. Instead, many financial institutions take a sample-based approach to their model-audit reviews. A sample-based approach should be risk-sensitive, placing greater emphasis on the review and testing of higher-risk models.

Individual model audits should monitor adherence to MRM policies and procedures and inform an aggregate view of the MRM framework’s effectiveness. Although there is no single measure for assessing model risk, internal audit should independently present a summary of MRM-related observations to senior management and the board. As part of its reviews, internal audit should evaluate the metrics used by the MRM group for management reporting and provide feedback regarding their adequacy, if appropriate. MRM metrics should be simple and straightforward, so that they are not a source of model risk themselves. Regulators expect that both second- and third-line functions evaluate and report on model risk across the firm.

Internal audit did not always play a significant role in MRM. In fact, the concept of MRM only became popular after the financial crisis of 2008-9. However, in the current environment, model risk is expected to be treated like any other risk: identified, measured, and controlled. Firms with internal audit departments that inadequately cover model risk have received unwanted regulatory attention. In the event that internal audit is unable to provide adequate model-related coverage, MRM controls should be augmented until a long-term solution is in place.

Given the increasing reliance of financial institutions on models, MRM’s growth in importance and stature is no surprise. Through the growth of this discipline, the role of internal audit in MRM has grown as well. Audit departments have strained to hire sufficient resources for model audits. In line with standard audit processes, many banks have implemented sample-based audit testing. Audit reports have long been a source of board information, and nowadays high-quality audit reports include MRM data as well. Audit reports are also a favorite source of information for financial-industry supervisors and a common starting point for regulatory exams.

How Promontory Can Help

Promontory helps all types of financial institutions develop, implement, and evaluate their MRM frameworks and programs. Our staff of experts includes former regulators responsible for developing the supervisory guidance on MRM, as well as former industry professionals from MRM and audit departments. We specialize in the development of solutions that consider both industry and supervisory perspectives. Our track record proves that we bring business value while maintaining a focus on regulatory compliance.

Contact Us

Nicholas Kiritz
Director and Lead Expert for Model Risk Management
nkiritz@promontory.com
+1 202 370 0401

Erik Larson
Managing Director and Global Head for Quantitative Methodologies and Analytics
elarson@promontory.com
+1 202 384 1200

Mark Levonian
Managing Director and Global Head for Enterprise Economics and Risk Analysis
mlevonian@promontory.com
+1 415 986 4160

Miles Ravitz
Principal
mravitz@promontory.com
+1 212 542 6787