Survey reveals issues companies face when building AI models
Although companies have been investing heavily into AI models, fewer than half of them make it to production. That’s according to the first ever State of AI Infrastructure Survey, commissioned by Run:AI.
The research conducted by Global Surveys questioned more than 200 scientists, AI and IT practitioners, and system architects across companies with over 5,000 employees. 77% of respondents said that most of their AI models never come into use. For a fifth of those who responded, that number drops even lower, with only 10% of their models reaching production.
Just 17% of AI companies are able to achieve high utilisation of their expensive AI resources. 22% of those building AI solutions say that their infrastructure mostly sits idle, as more than a third of respondents have to manually request access to GPU resources and are stuck with static allocations of hardware accelerators.
Challenges when it comes to using AI with data and the role of the cloud
The biggest challenge identified by AI workers in the survey was on the data side, with 61% saying that issues like data collection, data cleansing and governance caused problems. 42% highlighted challenges with their companies' AI infrastructure and compute capacity.
These two extremes of the AI development process – data and infrastructure – were cited more often than issues related to model development and training time, which just 24% of respondents mentioned as a challenge.
The survey also highlighted the role of the cloud in AI, with 53% saying that their AI applications and infrastructure are in the cloud, and 34% planning to move to the cloud in the next few years. Containers have become a standard infrastructure choice for running AI workloads, with 80% of respondents saying they use containers for some AI workloads, and 49% saying most or all of their AI work was run in containers. For orchestration, Kubernetes leads the way with 42% saying they used the popular solution and another 16% planning to adopt it. Next up was RedHat Openshift, with 13% using it and 6% interested.
“Companies are committing to AI, investing millions into infrastructure and likely millions more in highly-trained staff. But if most AI models never make it into production, the promise of AI is not being realised. Our survey revealed that infrastructure challenges are causing resources to sit idle, data scientists are requesting manual access to GPUs and the journey to the cloud is ongoing. Companies that handle these challenges the most effectively will bring models to market and win the AI race,” said Omri Geller, CEO and co-founder of Run:AI.
