What Is model risk management and how is it supported by enterprise MLOps?

Model risk management is about reducing bad consequences of decisions caused by trusting incorrect or misused model outputs. An enterprise starts by using a framework to formalize its processes and procedures, which gets increasingly difficult as data science programs grow. Systematically enabling model development and production deployment at scale entails use of an Enterprise MLOps platform, which addresses the full lifecycle including Model Risk Management. With a framework and enterprise MLOps, organizations can manage data science at scale and realize the benefits of model risk management that are received by a wide range of industry verticals.

What Is model risk management?

The genesis of model risk management was in the banking industry. It was first defined by the US Federal Reserve and Office of the Comptroller of the Currency (SR 11-7) in April 2011. The early definition now accepted as standard in the industry is: “The use of models invariably presents model risk, which is the potential for adverse consequences from decisions based on incorrect or misused model outputs and reports.” The guidance warns: “Model risk can lead to financial loss, poor business and strategic decision making, or damage to a bank’s reputation.”

In this Guidance on Model Risk Management, the regulators stressed the importance of using a systematic process, which would address all error points in a model starting with its design, through implementation, and ongoing maintenance. Financial institution decision makers were advised to be wary of the potential for limitations of models – especially when used differently from their original intent.

Since those earlier days, the surging use of models has produced significant AI applications that are disrupting major industries beyond banking and finance. Examples include healthcare, retail and e-commerce, food tech, logistics and transportation, travel, real estate, entertainment, and gaming. As a business becomes model driven, model risk management can help limit risks for sustainable growth in almost any industry vertical.

Types of model risk

The complexity of models, and general limitations of expertise with data science among business leaders, creates an environment ripe for risk. To manage model risk, it’s important to understand where the use of models can fall short. Three typical types of model risk include:

Wrong model

The excitement of climbing the ladder of a new business opportunity quickly sours when one discovers the ladder is leaning against the wrong wall. Correct alignment of a particular model with a corresponding business challenge is clearly essential. Failure to understand how that model applies to the challenge – or specifying an incorrect model – may doom an opportunity before it gets off the lower rungs.

Model implementation

The process of doing data science is about learning from experimentation failures, but inadvertent errors can create enormous risks in model implementation. Many moving parts of a model are subject to programming errors, technical errors, or by using inaccurate numerical approximations. Any one of these risks can become a material weakness in model implementation.

Model usage

Deciding to deploy a poor model is a risky error of usage. The “fuel” powering models is data, and issues there will also cause usage risk. Data may be the wrong type, or be too limited in scope, or be accidentally skewed by bias. Algorithms may execute in unintended ways due to calibration errors. Also beware using a model outside of the environment for which it was designed. Outputs for a new problem might not be consistent with the model’s original intent for a different problem without careful research and testing.

With a clear understanding of types of risks for your models, the next step is to institutionally manage these risks with a model risk management framework.

Model risk management framework

Model risk management is about systematically minimizing potential risks to computational models used for analyzing business strategies, informing business decisions, identifying and measuring risks, and many other business activities that could trigger unintended consequences from model development, inputs, or outputs. Model risk management uses a framework of activities, policies, and procedures to systematically formalize those processes for implementation by a large enterprise.

Elements of a model risk management framework

A model risk management framework combines sound governance principles with end-to-end documentation in the design, development, validation, and deployment of new models in the business. It is an essential enabler of the model development process, particularly in financial and life sciences institutions where the cost of failure in meeting regulatory standards is severe. As mentioned, a framework can also help model-driven organizations in many other industry verticals to minimize model risk.

Key elements of a model risk management framework include:

• Model definition: Entails recording the problem statement and intended purpose of the model output, including all development decisions such as techniques and datasets used.
• Risk governance: Establishing a clear charter for assessing risk across each model including policies, procedures, and controls to be implemented. Note that governance is different from compliance, which is about conforming operational policies and procedures with statutes or regulatory requirements.
• Lifecycle management: Identifying dependencies and factors of importance for continued usage of the model through its lifecycle.
• Effective challenge: Independently assessing and verifying to ensure all assumptions and decisions made during development are appropriate.

Using a framework helps an organization to prepare for model risk management. Doing the practical aspects of implementation and ongoing monitoring of risk requires the use of a software solution called enterprise machine learning operations (MLOps).

Implementing a model risk management framework with enterprise MLOps

Implementing a model risk management framework entails many complex practices and processes. The surge in machine learning applications has spawned a new software field called enterprise MLOps – both a practice and technology platform for the full lifecycle of building models, testing data and algorithms, deploying models at scale, maintaining models for ongoing relevance, and providing an enterprise-class system of record to document how models work and why they can be trusted.

Think of MLOps as being akin to ITOps, DataOps, ModelOps, or DevOps. Wikipedia defines MLOps as being the “set of practices at the intersection of Machine Learning, DevOps, and Data Engineering.” MLOps is generally focused on the deployment and management of models into production. Enterprise MLOps has two primary goals: to operationalize and automate model development and deployment processes at scale, and to improve quality of production models. Both are mandatory for efficient and effective achievement of business and regulatory requirements. With enterprise MLOps, an organization can get control and full visibility over the lifecycle of managing and developing models, while also supporting the practical implementation of model risk management.

Core capabilities of enterprise MLOps

The enterprise MLOps platform automates systematic processes for ensuring that models are built and validated properly. While exploring options for enterprise MLOps, consider four key capabilities across the data science lifecycle:

• Manage. Scaling data science and resulting models requires significant project management capabilities. These help to avoid key-person risk, which can be an impediment to realizing model velocity when a data science expert suddenly leaves for a new job. Integrated project management enables collaboration and governance of complex research projects. And it facilitates audit and review processes for compliance and trust in models.
• Develop. The core work of data science is building models. For a large enterprise, scaling the work of model building faces enormous barriers. For efficiency, data scientists need instant access to a broad array of tools, languages, libraries, integrated development environments, and compute resources. Lack of an integrated platform results in a substantial loss of productivity. Keeping track of all related artifact versions with respective software elements at scale is unachievable without an MLOps platform.
• Deploy. A sweet spot of Enterprise MLOps is fast and flexible deployment of models at scale. Enterprise MLOps eliminates the need for data scientists to rely on IT or software developers to deploy each model. The platform ensures that all resources required for deployment are immediately available to stakeholders.
• Monitor. Enterprise MLOps integrates a strong model maintenance system. It includes model retraining and rebuilding with a full history and context of original modeling work; previous versions are kept intact and are easily consumable by any stakeholder in the enterprise.

For example, the Domino Data Science Platform for enterprise MLOps includes a variety of templates and processes that allow data scientists to become immediately productive in their work – and enable data science leaders to have full visibility and control of scaling models for enterprise production. Our white paper provides a project-by-project checklist for model risk management, including model definition template, model validation process, model production process, and model inventory.

Benefits of enterprise MLOps and model risk management

The benefits of model risk management are a powerful incentive to implement enterprise MLOps; enabling models at production scale with the ability to be a “model-driven” business. When first developed for the banking industry, model risk management was often perceived as a path for facilitating compliance with regulatory standards such as SR 11-7, CRD IV, and GDPR. As model risk management extends through a variety of other industries, compliance is still an important objective. There are, however, a variety of other benefits for every model-driven business.

• Capacity. Use of enterprise MLOps will significantly boost throughput and capabilities of data science teams across the enterprise. Enterprise MLOps does this by providing instant access to any dataset, language, tool, integrated development environment, compute resource, or other element used to do data science – all on demand and without the need to get help from IT for configuring a resource.
• Quality. Doing data science is all about experimentation … doing it again, and again, and again until a model is ready for deployment at scale. Enterprise MLOps facilitates easy reusability of any dataset or tool used in development. A platform will provide teams and leaders with comprehensive knowledge management and the ability to collaborate with any stakeholder.
• Governance. Organizations get automatic enforcement of activities, policies, and procedures for developing and deploying models at scale for any use case. For example, guidance of SR 11-7 recommends that governance focus on testing and analysis to promote model accuracy. Related benefits of transparency improve the promotion of ethics, controlling bias, or enabling explainability of a model’s algorithm to facilitate trust.
• Operations. Automation of processes by enterprise MLOps enables streamlined workflows across teams, better documentation standards, and consistency in the model development process. Teams get more verifiable and accurate model results through integrated peer review and analysis, and the ability to compare current results against previous results.
• Management. Enterprise MLOps allows an organization to align personnel, infrastructure costs, and project prioritization with business value. Model risk management lets data science leaders show executives and board members why the organization’s efforts in data science are “worth it.”

Conclusion

The use of model risk management is essential for avoiding unintended consequences from model development, inputs, or outputs. The concept, amplified by a model risk management framework and implemented with an enterprise MLOps platform, is the bedrock of a model-driven business. As concluded by McKinsey & Company, “The expanding scope of models and the increased use of models based on advanced analytics have amplified the strategic importance of model risk management.”

If your company is on the journey of being model-driven, we invite you to read some practical advice and checklists in our white paper, Model Risk Management in Domino Data Lab.