Amy Vataksi
Forward Deployment Engineering Intern
Agentic AI, AWS AgentCore, Strands Agents, Amazon Nova, Domino LLM Gateway, MLflow tracing, observability
Data scientists, ML engineers, AI engineers, forward deployed engineers, FSI platform teams
The challenge:
Organizations adopting AI agents are increasingly running them on a cloud vendor's agent runtime. AWS has Bedrock AgentCore, Google has the Gemini Enterprise Agent Platform, and Azure has the Azure AI Foundry Agent Service. Standardizing on a managed runtime hands off the work of operating the orchestration layer.
However, that choice splits the stack. This split isn't unique to any one stack. Agent deployment areas sit isolated from the traditional MLOps toolchain, and teams whose data and infrastructure live in AWS often get pushed toward tools like AgentBuilder that don't connect back to where their models are governed. Domino's hybrid-cloud nature makes it the go-between that reconnects a fragmented toolchain. For teams whose models live in Domino, the split has a specific cost. Domino already provides governed model endpoints, detailed experiment tracking, reproducible runs tied to a user and a project, and an audit-ready Agent Performance tab. Those properties don't automatically follow the agent into the AgentCore runtime. Once the agent runs there, the record of what it did (every model call, tool invocation, and reasoning step) lands in AWS, disconnected from the governance layer the team already built in Domino. Domino stays where governance lives; the job is to extend it to the agent wherever it runs.
Teams in regulated industries feel this most directly. Frameworks like SR 26-2 and the NIST AI RMF call for model decisions to be traceable and auditable. Domino ties every inference back to a specific model version, a specific user, and a specific run. Without a bridge, you have execution in AWS and governance in Domino, and the two records never meet.
This disconnect isn't only about records. It also shows up in how credentials move between the two systems. Without a shared authentication boundary, a team running agents in AgentCore has to choose between handing AWS credentials directly to the runtime, duplicating Domino's access policies inside AWS, or exposing a Domino API key to code running outside Domino's network entirely. Each option spreads credentials across two clouds instead of keeping them inside the perimeter where they're governed. The audit trail problem and the credential problem share the same root cause: once the agent runs outside Domino, nothing about how it authenticates, or what it did, travels back automatically.
The solution:
Domino governs the full lifecycle of the model: where it is built, hosted, and monitored. It connects to whatever agent runtime you already run. You can swap runtimes without rebuilding your governance, and every run still reports back to the same audit trail in Domino. Credentials stay inside that same boundary: the agent authenticates to Domino with a single API key, so no separate model or provider credentials need to live inside the AgentCore runtime. This integration pairs Bedrock AgentCore with Domino and gives you a working, reproducible setup you can install and run:
Confirm the following before starting. The execution role setup (below) must be completed by an AWS administrator before the deploy script can run.
Domino project access
You need permissions to create apps, run jobs, publish model endpoints, and view the Agent Performance tab.
AWS account with an AgentCore execution role
Create an IAM execution role in your AWS account with these permissions:
bedrock-agentcore:* - allows the runtime to register and manage AgentCore agentsbedrock:InvokeModel - allows the runtime to call models on Bedrockiam:PassRole - required by agentcore configurelogs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents - allows the runtime to write to CloudWatchsecretsmanager:GetSecretValue scoped to the path domino/api-keyAWS CLI in your Domino compute environment
The deploy script calls aws sts get-caller-identity to verify credentials before deploying. Confirm the AWS CLI is available in your compute environment before running the deploy job. It ships by default in the Domino standard environment (v2.35.6). If it's missing in your environment, install AWS CLI v2 via your compute environment's Dockerfile. See AWS's install instructions.
Credentials as Domino project environment variables
Add DOMINO_API_KEY under Account -> Account Settings > Environment Variables. This key is personal to your account, so setting it here keeps it out of the project's shared configuration and avoids exposing it when the project is shared with collaborators. This key authenticates both calls to the Domino LLM Gateway and MLflow trace writes.
Add the following under Project Settings > Environment Variables, since these are shared credentials for the integration rather than personal to one user:
AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY — AWS credentials with permission to create AgentCore runtimes.The .env file configured
Copy .env.example to .env and fill in:
DOMINO_HOST — your Domino instance URL (e.g., https://your-org.dominodatalab.com)DOMINO_MODEL_URL — your Domino instance’s LLM Gateway base URL (e.g. https://your-org.dominodatalab.com/apps/llm_gateway/v1)AWS_REGION — the AWS region where AgentCore will runAGENT_RUNTIME_NAME — a name for the AgentCore runtime (e.g., agentcore-domino)DOMINO_PROJECT_NAME, DOMINO_PROJECT_OWNER, and DOMINO_APP_ID don't need to be set manually. Domino injects these automatically into every run, so the integration picks them up without any manual configuration. See the Default Domino Environment Variables Glossary for the full list, and Manage Environment Variables for how Domino resolves a variable across Compute Environment, Project, and User scopes.
Important: .env is committed to git by design. This file contains only non-sensitive configuration data such as your host URL, AWS region, and runtime ARN. All secrets and sensitive credentials must be stored as Domino project environment variables. Since the deploy job packages the agent from the committed git state, your .env file must be tracked; if gitignored, the deployment will fail.
Leave AGENTCORE_RUNTIME_ARN as the placeholder for now. You will fill this in after Step 3.
Run deploy.sh as a Domino Job. The script runs in two phases. First, agentcore configure generates .bedrock_agentcore.yaml, the AgentCore deployment manifest:
agentcore configure \ --entrypoint agent.py \ --name "$AGENT_RUNTIME_NAME" \ --requirements-file requirements.txt \ --deployment-type direct_code_deploy \ --runtime PYTHON_3_11 \ --disable-memoryThen agentcore deploy packages the agent and pushes it to AgentCore Runtime with these environment variables baked into the container:
DOMINO_API_KEY - Domino LLM Gateway auth and MLflow tracking tokenMLFLOW_TRACKING_URI - your Domino host, so the agent knows where spans are shippedDOMINO_PROJECT_NAME and DOMINO_PROJECT_OWNER - used to resolve MLflow experiment writesOTEL_SERVICE_NAME - the service label on OTel spans (set to AGENT_RUNTIME_NAME)MODEL_ID - only forwarded if set in .env; otherwise defaults to nova model aliasWhen the deploy completes, agentcore status prints the runtime ARN. Copy it for Step 3.
Important: deploy builds from committed git stateThe deploy job packages agent.py from the last committed git state, not your working copy. If you modified agent.py and did not commit and push before running the job, the old code will be deployed. The ARN will not change between deployments since AgentCore updates the runtime in place. An unchanged ARN after a redeploy is normal and not a sign that the deploy failed.
Note on toolkit version: deploy.sh installs bedrock-agentcore-starter-toolkit with no version pin, so it always pulls the latest release from PyPI at deploy time. Command names and flags reflect the toolkit version current as of this writing; if AWS changes the CLI, this step may need updating.
Take the ARN printed by agentcore status and paste it into .env as AGENTCORE_RUNTIME_ARN. Commit and push the updated .env before publishing the app - the app reads this value at startup.
In the Domino UI, publish the project as an Agent with app.sh as the launch command. Publishing as an Agent rather than a standard App gives users the same web frontend, but also wires the deployment into Domino's agent tracing, so its runs are captured in the Agent Performance tab without any extra setup. When the app starts, app_server.py reads AGENTCORE_RUNTIME_ARN from the environment and generates agent_config.yaml automatically. You do not need to create or edit this file by hand.
The app exposes two routes:
/ - the SPA frontend with the chat interface and step-by-step audit trail/api/invoke - the FastAPI endpoint that proxies prompts to AgentCore and replays OTel spans into MLflowOptional: smoke test from CLI
To call the deployed agent directly without the app (useful for confirming the agent is running before publishing):
set -a; . ./.env; set +apython invoke_agentcore.py "What is the capital of France?"This logs each OTel span as a separate run in the agentcore-traces experiment and prints span details to the terminal.
Open the published app and send a prompt. Confirm:
1. A response appears in the UI. The trace may take a moment to appear. If it isn't there right away, refresh the page.
2. The audit trail shows the expected steps - LLM, tool call, and agent span types, depending on what the agent did.
3. "Recorded in Domino· trace [ID]" appears at the bottom of the response.
4. The trace is visible in the Domino Agent Performance tab.
If the trace does not appear in the Performance tab:
The app runs in development mode by default, which logs to the agentcore-traces experiment in Experiment Manager. The Agent Performance tab reads from a different experiment. To switch to production mode, set these environment variables in the Domino App's environment:
DOMINO_AGENT_IS_PROD=trueDOMINO_APP_ID - the App ID shown in the Domino UI for this appIn production mode, the app logs to agent_experiment_{DOMINO_APP_ID}, which is the experiment the Performance tab reads. In development mode, check the agentcore-traces experiment in Experiment Manager instead.
If the app returns an error instead of a response:
Check the Domino Job log for the deployment. The most common cause is a stale deployment: your changes were committed after the job ran, so the job deployed the old code.
New capabilities are added to the agent as plain Python functions decorated with @tool. Strands reads the function's docstring to construct the schema the model sees. agent.py includes one working example, get_account_summary, which currently returns mock data; connecting it to a real Domino model endpoint means calling that endpoint from inside the function body, using get_domino_api_key() to resolve the credential and sending it as an X-Domino-Api-Key header, consistent with how DominoAuth handles authentication elsewhere in the file. Once written, a new tool is registered in the tools= parameter inside make_agent().
The system prompt is defined inside make_agent(), within the Agent(...) constructor, and as shipped is a placeholder intended only to demonstrate that the integration works end to end. For a production use case, it should be expanded to include a role definition (who the agent is and the domain it operates in, rather than a generic description), behavioral constraints (what the agent must never do, and how it should respond to errors or uncertainty from a tool call), and tool usage guidance (explicit direction on when each registered tool should be used, since the model only knows a tool exists from its docstring and doesn't know the intended policy for when to call it). The system prompt should be revisited each time a tool is added or changed.
When the Domino LLM Gateway blocks a request for tripping a guardrail, the error currently falls through to the agent's generic exception handler, and the result is a raw exception type, message, and stack trace surfaced directly in the UI, making a guardrail block indistinguishable from an application crash. The intended fix is to catch this specific error before it reaches the generic handler and return a clear, user-facing message in its place, while still recording the associated trace spans, since a blocked request is itself an event worth auditing and omitting its spans would leave the one request that was blocked as the one request with no record.
For the tracing architecture this integration uses, including how OpenTelemetry spans map to Domino runs and how the Domino Performance tab is populated, see the GenAI Tracing Tutorial blueprint.
For patterns on deploying and governing agentic workflows more broadly in Domino, see the Deploying Agentic AI Systems blueprint.
Forward Deployment Engineering Intern
Amy is a Forward Deployed Engineering intern, who builds agent platform integrations connecting Domino's AI platform to external AI orchestration frameworks. She's especially interested in the intersection of regulated-industry ML workflows and modern agentic tooling.
