Agentic AI governance frameworks designed for predictive models cannot track the risk surface created by autonomous systems that select their own tools, call third-party agents, and act without human checkpoints. Regulated enterprises need a governance model that avoids concrete bad outcomes.
Standard governance controls assume testable, bounded systems. Agentic AI decides for itself which tools to call and in what order, making monitoring, intervention, and human-in-the-loop assumptions structurally unreliable.
Connecting your system to an external orchestrating agent means losing visibility into how that agent was built, tested, and governed. Most organizations are not treating this as the risk decision it is.
Organizations must define their “Ethical Nightmares,” concrete failures they want to avoid, instead of anchoring AI governance around abstract principles like fairness or transparency. Clearly identifying these bad outcomes gives development teams guardrails to build against and provides regulators with a transparent framework to evaluate.
Any level of an organization — project team, department, or board — can answer the same three questions and get complementary, actionable results: What are our AI ethical nightmares? What resources will we build to avoid them? How will we train people to use those resources?
Ethical Nightmare Challenge (ENC) teams are cross-functional groups trained on a shared method to surface risks early. They work at the site of complexity, building resources to avoid nightmares before deployment.
Reid Blackman shares how current AI governance practices are failing because values-based frameworks were designed for stability and cannot keep pace with agentic AI. As autonomous systems decide for themselves which tools to call and in what order, the risk surface expands faster than any policy document can track. For banks and other regulated enterprises navigating agentic AI governance, that gap between policy and deployment is where regulatory exposure lives.
The session introduces the Ethical Nightmare Challenge, where organizations build cross-functional teams trained on a shared method to identify and address the specific bad outcomes they need to avoid. Multi-agent architectures, particularly those involving third-party agents, are a governance problem that most organizations have not scoped correctly. The ENC model shifts model risk management from a review gate to an embedded problem-solving capability applicable at every level, from the project team to the board.
