Forrester TEI Study: Domino Delivers 542% ROI and Payback within 6 Months

Forrester recently surveyed¹ thousands of business and technology decision-makers to gain insight into their use of data and analytics, and the factors that contribute to their purchasing decisions. When asked what benefits their organization expected when using Artificial Intelligence (AI) technologies:

• 30% expected to “increase revenue growth”
• 30% expected to “improve operational efficiency and effectiveness”
• 30% expected to “increase automation of internal processes”
• 27% expected to “increase employee productivity”

Unfortunately, 15% of those surveyed also said that “no unified belief in a clear ROI to AI” was one of the biggest challenges holding their company back from adopting AI technologies. And 21% were most concerned about the “opportunity costs of their AI investment”. Clearly, organizations have high expectations for AI (as they should), but are having trouble quantifying the profound benefits that an investment in AI can have on their business. It’s no wonder then that 17% cited the “lack of executive support to drive the use of AI” as one of their organization’s biggest challenges.

Forrester has offered many suggestions on how organizations can realize the benefits of adopting AI and data science. For example, according to The Forrester Wave™: Notebook-Based Predictive Analytics And Machine Learning, Q3 2020, “Notebook-based PAML offerings take notebook-based data science to the next level by making individual data scientists dramatically more productive, especially by providing collaboration capabilities for teams.”

We have known for a long time that Domino makes data science teams more productive. We’ve documented many of their incredible stories, and in some cases have been able to measure the impressive benefits. For example, Lockheed Martin attributes $20 million in annual cost savings to their use of our platform thanks to reduced IT costs, increased onboarding efficiencies, and a 10x increase in data scientist productivity.

But some people are naturally skeptical when they see incredible numbers like this. Others are unsure of where to invest in data science and MLOps solutions given the state of confusion that exists in the market right now with many vendors pushing vastly different approaches that target different types of users and have different ambitions of scale. We feel it’s critical that buyers weigh the potential impact of enterprise MLOps against the investment they have to make in people, processes, and technology, and then measure the return on that investment.

Quantifying the Business Value of Domino

In order to clear up some of the skepticism and confusion, we commissioned an independent Forrester Consulting study: The Total Economic Impact™ (TEI) of the Domino Enterprise MLOps Platform. Forrester conducted detailed interviews with several Domino customers to understand how they’re using our platform and gain insight into the benefits they’re receiving. Based on these interviews they built a composite organization consisting of a team of 50 data scientists, with projected team growth that aligns with what many of our customers experience in their first three years of using Domino.

The results speak for themselves: Forrester projects that organizations that use Domino realize nearly $30 million in total economic benefits over a three-year period, with a return on investment of 542%. They also concluded that an investment in Domino pays for itself in less than six months.

“If we hadn’t invested in Domino, first, I wouldn’t have been able to set up a team at all, because you can’t hire a high-skilled data scientist without providing them with the state of the art working environment.” – Chief Analytics Officer, Insurance

The projected benefits of our platform as realized by the composite organization over a three-year period include:

Reduced time to configure compute resources of $9.8 million

According to customer interviews, Domino allows data scientists to begin working on new models almost instantaneously by providing automated access to infrastructure and managed environments, saving on average 70 hours per instance. Interviewees reported that the previous process to provision infrastructure, create the environment, and install/maintain individual packages often took days to weeks, depending on the complexity of the build. The net time savings for the composite organization was estimated at $9.8 million over three years.

Self-serve access to powerful compute resources also results in fewer support calls to engineering; Managers estimate that Engineers save about 30% of the previous hours spent responding to requests from data scientists.

“Before Domino, it could take two to three weeks to understand and spin up an infrastructure and then start the work. With Domino, that went from weeks to just a click of a button.” – Director of Data Platforms and Privacy, Pharmaceuticals

Increased operational efficiency of $8.5 million

Data science managers reported that with Domino, their data science team can make use of more tools, spend less time waiting for models and experiments to run, and utilize advanced collaboration and visualization capabilities to share their work with others in new ways.

Managers estimate savings of 200 hours per year per data scientist, which for the composite organization results in a three-year savings of $5.1 million.

Organizations also realized benefits related to faster onboarding of new team members, and more efficient model validation, deployment, and maintenance. For the composite organization, these savings added up to an additional $3.4 million.

“We certainly are implementing more accurate models or even models that we couldn’t have done before with more complicated workflows. And that’s probably an improvement of about 20% of where we might have been.” – Principal Consultant, Financial Services

Incremental profits of $5.2 million

Domino customers reported that being able to collaborate more effectively during the development cycle has led to data science solutions that are better aligned to business needs and can directly impact revenue. They’re able to produce more powerful models and also benefit from a higher success rate for model deployment, leading to a significant impact on the bottom line. For the composite organization used in this study, incremental profit was estimated to be close to $5.2 million dollars.

“Our use of Domino has had a strong impact on revenues in the millions of dollars, tens of millions of dollars in fact.” – Principal Consultant, Financial Services

Get the Details of the Forrester TEI Study

These numbers and quotes are just the beginning of the details you’ll find in The Total Economic Impact™ (TEI) of the Domino Enterprise MLOps Platform. Check out the study to learn about the challenges interviewees were facing before they implemented Domino, as well as get greater insight into the benefits that Domino is providing for them.

June 7, 2021 update: I just moderated a webinar featuring Dr. Kjell Carlsson, Principal Analyst, Forrester, along with the author of the TEI, Mary Barton. Kjell described some of the key challenges that organizations face when scaling data science, and we had a lively discussion about the TEI results themselves along with practical examples from their interviews and Domino case studies. You can watch the replay of the webinar now.

¹ Base: 2,456 Data and Analytics decision-makers in the US, Canada, UK, France and Germany; Source: Forrester Analytics Business Technographics Data and Analytics Survey, 2020.