Navigating the hurdles of AI costs and governance

Enterprises continue scaling their AI ambitions, but two barriers keep pulling them back: spiraling cost and persistent AI governance gaps. These challenges aren’t immediate and don’t always show themselves at the beginning of a company’s AI journey, but research shows these hurdles are on everyone’s horizon.

Research signals you can’t ignore:

• 59% of respondents say misaligned governance processes and priorities hold their AI programs back.
• 60% list rising operational costs as the chief limit on scale.
• 60% expect less than 50% return on investment (ROI) from AI in the next 12 months.
• 47% select data quality as the top success metric for governing data for AI.
• 45% of AI leaders cite high costs/budget limitations as their top obstacle for delivering AI strategy.

AI leaders as referenced above are companies that have already built and deployed a foundation for a successful AI practice. That foundation includes the following critical components:

• Security standards and compliance
• Data access/use polices
• Identifying AI leadership in the organization
• Legal considerations
• Project governance oversight
• AI program standards and polices
• Enterprise architecture requirements
  

Leaders represent just 20% of research respondents and while they have done work to build out a foundation for enterprise AI they are often the first to encounter cost and governance challenges.

Why governance is so important

As companies move faster with sophisticated AI strategies, new tools and solutions are generally needed to better manage aspects of AI governance. Governance is driven by process and software frameworks that deploy guardrails and speed innovation. Going too long without it creates significant issues.

Governance gaps can fall within several areas. Here are three to consider.

1. Model lifecycle management – Many teams deploy without independent validation, risk scoring, or sign-off of projects and models. That leaves blind spots in bias, model drift, and security.
2. Regulatory exposure – The EU AI Act banned “unacceptable-risk” systems in February 2025 and sets staged obligations for transparency and oversight over the next three years. Firms that do not tighten processes now will scramble later and risk financial penalties.
3. Data quality debt – Poor lineage and unclear data ownership make it hard to prove that training data meets internal or legal standards. BARC research shows almost half the market treats data quality as the number one governance success metric. The fix starts with a single end-to-end workflow for registration, validation, approval, monitoring, and audit of every model and their source(s).

Treat each of these areas as a gate that must pass documented checks before moving on. Automate the capture of metrics and evidence so you can answer auditors in minutes, not weeks.

Growing costs of AI

Clearly understanding AI costs translates to AI FinOps and most companies have yet to deploy this type of technology because their AI strategy hasn’t yet hit a complex level. Once it does, costs soar unless the company has the appropriate view, context and controls to impact it. Live FinOps dashboards help companies monitor and respond to workloads, project usage, compute and storage issues that can all be costly.

Cost overruns can stem from:

1. Idle or over-provisioned GPU clusters.
2. Unused development workspaces that stay live.
3. Inference endpoints left unattended.
4. Redundant copies of large training datasets.
5. Unchecked cloud resources attached to projects.
   

AI FinOps dashboards should be positioned directly inside your MLOps stack. They should stream near-real-time usage from your cloud or on-premises environments into a unified view.

Leaders vs. followers, BARC data shows that organizations already subject to strict regulation such as finance, health, regulated manufacturing are moving faster than others. These companies are early to bake in governance and cost controls, helping them scale with predictability. Followers often run pilots in isolation, discover the cost or compliance bill late, and slow rollouts to re-engineer foundational processes. That lag shows up in ROI while leaders plan for higher payback because they avoid re-work.

A few words of advice:

• Link governance and cost KPIs to business outcomes. Make them board-level metrics, not technical afterthoughts.
• Adopt a unified control plane. One system to register, validate, approve, monitor, and retire models. Document every hand-off. Build real-time cost observability.
• Connect to granular data on compute, storage, and traffic. Tie alerts to hard budget caps and departmental owners.
• Automate model monitoring. Track accuracy, drift, and cost per request. Trigger retraining or rollback automatically.
• Run cost post-mortems. After each major experiment, review spend against forecast, identify idle resources, and update provisioning rules.

The bottom line

AI innovation is no longer limited by clever ideas or tooling. The bottlenecks are the money you burn and the controls you follow. Tackle both with the same urgency and you clear the runway for the next wave of models and projects without the surprise invoices or last-minute compliance fire drills that stall everyone else.

Check out this eBook for full research insights on how enterprise leaders are keeping pace with AI innovation and transformation.

Sources:

Domino Data lab with BARC, From prototype to payoff, how enterprises are leading through AI transformation. April 2025.

BARC Research, Preparing and Delivering Data for AI, Adoption Trends, Requirements and Best Practices, March 2025.

BARC Research, Optimizing Your Architecture for AI Innovation, March 2024.

About BARC

BARC is a leading analyst firm for data & analytics and enterprise software with a reputation for unbiased and trusted advice. Our expert analysts deliver a wide range of research, events and advisory services for the data & analytics community. Our innovative research evaluates software and vendors rigorously and highlights market trends, delivering insights that enable our customers to innovate with data, analytics and AI. BARC’s 25 years of experience with data strategy & culture, data architecture, organization and software selection help clients transform into truly data-driven organizations.