AI: The engine accelerating the vehicle of Data Analytics

AI is taking off. According to Accenture's report, 84% of business executives believe they need to use AI to achieve their growth objectives. And the first and foremost benefit of the AI and Big Data Analytics fusion, is that it helps organisations to achieve significantly enhanced customer service.

The meeting of AI in Data and Analytics is able to generate insights, automate processes, deliver predictions and drive actions that lead to better business outcomes. By powering data and analytics with AI, organisations can gain a far more comprehensive view of their operations, their customers, their competitors and the market as a whole.

This overarching view enables organisations to gain insight into customer behaviour, identify trends in user activity and to make potent data-driven decisions. This is a fast-emerging field, and one that offers capabilities that traditional data analysts simply could not achieve in terms of speed, scale and granularity.

And if you want to know about enterprise data management and analytics, then you need to speak to a Chief Technology Officer that has both academic, and boots-on-the-ground experience in the field.

Chris Royles is the EMEA Field CTO at Cloudera. He helps organisations innovate through the use of data, working across industries that are regulated and organisations where data privacy is critical.

“My focus is on the development of skills and methods for migration to the Enterprise Data Cloud,” he says. “When you think of EMEA, it is a diverse set of regions and industries; so I have the fortune of speaking to a lot of organisations that use data and AI in many different areas of their business.”                   

Royles also holds a Ph.D in Artificial Intelligence and says that recently, “it is wonderful to see AI becoming far more mainstream.”

Effectively applying AI & ML to analyse large volumes of data: Derive actionable businesses insights

At Cloudera, they have always believed data is a foundational resource to derive actionable insights. Services such as ChatGPT have raised awareness and fuelled conversations about AI and its potential business benefits. “For organisations to truly benefit from the innovations in Generative AI and Large Language Models (LLMs), they should look to use existing Open Source foundational models to augment how they converse with their trusted and private Enterprise data,” says Royles. “We refer to this as Enterprise AI, and how an Open Data Lakehouse can support Enterprise AI that organisations and their customers can trust.”

To achieve AI at scale and drive digital business transformation, Royles advises that organisations must be able to secure and govern all of their data, irrespective of where it resides under management. “By building a corpus of trusted information, organisations can use it to underpin engaging user interactions. To build trust the services also have to be current, aware of the latest information and able to respond to changes in real-time.”

Key challenges in implementing AI-driven data analytics projects - and strategies to address these challenges

Implementing AI-driven data analytics projects poses several challenges that organisations must address strategically.

Royles thinks that getting research and prototypes out of the lab is the biggest barrier to success. He says that it’s important to make this easy by having clear access to the raw materials, as well as the build processes for an AI service, “and engaging with stakeholders on what this really means in respect to business impact. For example, at Cloudera we package our research and patterns into Applied Machine Learning Prototypes (AMPS), which organisations can deploy against their own systems with just a few clicks.”

He says that being able to scale inference, fine-tuning and training across the parallel compute and GPU resources required, must be inherently available to your practitioners, and organisations should have the choice as to where and how this is provisioned - on both public and private cloud.

He says, “ensuring trust in your data through strong governance and high durability is vital, necessitating investment in data governance practices and robust data collection.

“Through enhanced interpretability, businesses can employ explainable AI techniques that provide transparency and insights into model decisions. Also comprehensive auditing of how humans interact with models can help understand and fine tune the models themselves and drive continuous improvement.  This is often referred to as building a data flywheel.

“Finally, driving adoption requires effective change management efforts, including comprehensive employee training and fostering a data-driven culture. By proactively addressing these challenges, businesses can successfully implement AI-driven data analytics projects, gaining valuable insights and a competitive edge in their decision-making processes.”

How AI and analytics can help organisations in making data-driven decisions: Best practices for integrating AI into the decision-making processes

According to Royles, AI and data analytics provide organisations with the ability to make data-driven decisions. He says, “to effectively integrate AI into the decision-making processes, several best practices should be followed.” He says the first is essential and that’s to define clear objectives for the decision-making processes and identify specific questions or problems that AI and analytics can address. Organisations must be clear on what they want the outcome to be.

Once this has been established and agreed, they must ensure data quality and accessibility. This involves implementing data governance practices, validating data sources, and establishing robust data integration processes to ensure access to relevant, accurate, and up-to-date data.

Promoting a collaborative approach is important too. If organisations can foster collaboration between the teams involved, they can ensure that AI models and analytics align with the business need and objectives rather than being unaligned. This interdisciplinary collaboration will help in developing insights that are reasonable, meaningful, and actionable. This can be achieved via small, focused AI projects and iterating based on feedback and insights too.

After doing this, organisations can gradually scale up by incorporating additional data sources and expanding the scope of analytics applications. By following these practices, organisations can effectively integrate AI into their decision-making processes, leading to data-driven insights and informed strategic decisions that will drive business success and competitive edge.
Royles ends by saying: “As a final cautionary note, if an AI service is too slow, inaccurate, or responds in ways that don’t add value, users will quickly lose trust, and it will be very hard to recover.”