Survey reveals issues companies face when building AI models

The first ever State of AI Infrastructure Survey has revealed challenges faced by companies building and deploying AI models

Although companies have been investing heavily into AI models, fewer than half of them make it to production. That’s according to the first ever State of AI Infrastructure Survey, commissioned by Run:AI

The research conducted by Global Surveys questioned more than 200 scientists, AI and IT practitioners, and system architects across companies with over 5,000 employees. 77% of respondents said that most of their AI models never come into use. For a fifth of those who responded, that number drops even lower, with only 10% of their models reaching production. 

Just 17% of AI companies are able to achieve high utilisation of their expensive AI resources. 22% of those building AI solutions say that their infrastructure mostly sits idle, as more than a third of respondents have to manually request access to GPU resources and are stuck with static allocations of hardware accelerators. 


Challenges when it comes to using AI with data and the role of the cloud 

The biggest challenge identified by AI workers in the survey was on the data side, with 61% saying that issues like data collection, data cleansing and governance caused problems. 42% highlighted challenges with their companies' AI infrastructure and compute capacity. 

These two extremes of the AI development process – data and infrastructure – were cited more often than issues related to model development and training time, which just 24% of respondents mentioned as a challenge.

The survey also highlighted the role of the cloud in AI, with 53% saying that their AI applications and infrastructure are in the cloud, and 34% planning to move to the cloud in the next few years. Containers have become a standard infrastructure choice for running AI workloads, with 80% of respondents saying they use containers for some AI workloads, and 49% saying most or all of their AI work was run in containers. For orchestration, Kubernetes leads the way with 42% saying they used the popular solution and another 16% planning to adopt it. Next up was RedHat Openshift, with 13% using it and 6% interested.

“Companies are committing to AI, investing millions into infrastructure and likely millions more in highly-trained staff. But if most AI models never make it into production, the promise of AI is not being realised. Our survey revealed that infrastructure challenges are causing resources to sit idle, data scientists are requesting manual access to GPUs and the journey to the cloud is ongoing. Companies that handle these challenges the most effectively will bring models to market and win the AI race,” said Omri Geller, CEO and co-founder of Run:AI.


Featured Articles

Need for responsible AI in some of the world’s largest banks

Research shows one-third of North America and Europe’s largest banks lack transparency and are not publicly reporting on their AI development

Lenovo: Employees prefer mix of AI and human IT support

New Lenovo survey shows 91% of employees believe they would be more productive when their IT issues at work are resolved quickly and effectively

Kyndryl’s Data and AI Console to simplify data management

Data-driven solution expands and increases observability and insights, while enhanced data governance helps identify irregularities and threats

Deep neural networks still struggling to match human vision

Data & Analytics

Metaverse destined to become an impossible, dangerous place


Clever coders lead the way as Microsoft launches 365 Copilot

AI Applications