Machine learning governance built into the model development workflow

Q: Why does ML model governance review take so long in regulated industries?

Governance review is slow when evidence has to be reconstructed after models are trained. Ad-hoc experimentation scatters parameters, metrics, and artifacts across notebooks and local files with no shared structure. Cross-run comparison becomes manual, reproducibility depends on individual memory, and the reconstruction effort can rival the time spent building the original model.

Q: How does a configuration-driven ML pipeline support model reproducibility and governance?

A configuration file defines every aspect of a pipeline run — which tasks execute, in what order, with what parameters. Each run automatically captures the git commit hash, execution environment revision, and pipeline entry point, creating a complete record. This means any experiment can be reproduced precisely from three pieces of information, and governance reviewers receive a structured evidence package rather than manually assembled artifacts.

Q: How do you automate audit trail creation during ML model development?

Automating the audit trail requires building evidence capture into the pipeline itself. When each run automatically records its configuration, execution environment, and task outputs into a centralized run record, the governance package is complete by definition. Pushing that record to an experiment tracking system like MLflow means reviewers receive structured, comparable evidence without relying on the data scientist to compile it.

Reconstructing governance evidence after a model is built takes as long as building the model itself. Structured ML experiment pipelines eliminate that reconstruction step by capturing the evidence as each run executes.

Chris Porter

Data Science Advisor

Fannie Mae

Rabbani Mozahid

Director

Fannie Mae

What you'll take away from this session

Design the evidence package before training any models

Governance review gaps trace back to evidence that was never captured during experimentation. Defining required parameters, metrics, and artifacts before any models are trained ensures the needed analysis for the evidence package is captured.

Structured runs speed governance review

Standardized experiment output eliminates the reconstruction task at review time. Parameters and metrics are already aligned across runs, and the evidence package is essentially complete.

Configuration files keep pipelines stable during iteration

Every variation between experiments is visible and auditable when pipeline changes are made in configuration, not code. The stable parts of the workflow stay stable; the variable parts are visible and auditable in each run.

Three anchors make any run reproducible years later

Any experiment can be reconstructed precisely from three automatically captured values: the git commit hash at execution time, the environment revision, and the pipeline script file path.

True reusability happens after governance approval

The real test of a reusable workflow is how fast it can adapt after governance approval. Using the fnm-benchmark, an internal Python library for automating evidence capture, the team compressed a traditional four-month project into six weeks. Since the infrastructure was reusable, a post-approval revision took less than a day to complete.

Fannie Mae builds and maintains credit risk models for its Desktop Underwriting platform in one of the most heavily regulated lending environments in the U.S. The core problem their team faced was structural: ad-hoc experimentation scatters parameters, metrics, and artifacts across notebooks and local files, leaving each governance review as a reconstruction effort rather than a verification. Their solution was fnm-benchmark, an internally developed Python library that treats a configuration file as the single source of truth for every pipeline run and populates a centralized RunContext automatically as each task executes.

Machine learning governance built into the workflow means the evidence exists by definition when review begins, not because a data scientist remembered to compile it. The first production use of fnm-benchmark was a credit risk model for the Desktop Underwriting platform. A project that had historically taken four months was completed in six weeks, with a review-ready evidence package. When leadership requested one additional run after approval, the revision was complete in under a day.

The distinction that matters is between documentation that describes what was done and infrastructure that captures it as it happens.

FAQ

Why does ML model governance review take so long in regulated industries?

How does a configuration-driven ML pipeline support model reproducibility and governance?

How do you automate audit trail creation during ML model development?

Transform the work that matters most

See how Domino helps the world’s most regulated enterprises build, scale, and govern AI-powered applications.

Watch demo Talk to an expert