The AI Interview: Walter Sun, SVP & Global Head of AI, SAP

Walter Sun arrived at SAP in 2023 from Microsoft with a specific brief: solve the application chaos that forces enterprise employees to navigate dozens of systems just to complete basic tasks. Now, as SVP and Global Head of AI at SAP, he oversees the company’s attempt to convert that complexity into a single conversational interface.

The challenge is familiar to anyone who has worked in a large organisation. Booking travel requires logging into Concur. Hiring staff means opening SuccessFactors. Checking customer data involves yet another system. SAP’s answer is Joule, an AI assistant launched in September 2023 that aims to handle tasks across the company's entire application portfolio.

“The biggest value driver for business AI for SAP is that we actually have products in all different lines of businesses,” Walter explains during an interview at SAP Sapphire 2025. “So if you’re a business, you could potentially have SAP applications from your ERP to your Human Capital Management and SuccessFactors, you use Concur for travel. And the fact that we have all the different systems and AI underneath all of it is that you can actually connect the dots to all of them.”

Walter leads SAP’s AI product engineering organisation, which builds reusable technologies for its Business Technology Platform. Document extraction capabilities built once can power expense processing in Concur, invoice handling in supply chain systems and financial ledger analysis.

Joule now covers 80% of common scenarios across the company’s business applications: substantial progress towards SAP’s goal of embedding over 130 out-of-the-box AI capabilities across its cloud portfolio by 2025. The assistant operates through what SAP calls “omnipresence” – maintaining conversational context while executing tasks across multiple systems that would traditionally require separate logins and navigation paths.

"Because Joule is omnipresent and it also has history, you can actually ask a question and chat with it like you chat with a human assistant," Walter describes. "Say, look, I need to go to Sapphire. Book me a trip to Orlando, it goes to Concur and does that. And then a few minutes later say, oh, I need to hire a new employee for my team. Can you please open a requisition? And it goes into SuccessFactors to do that and you say, oh, I need to figure out where our biggest customers are and how much they're spending. It can go into our CX tool and find solutions there."

The 80% coverage milestone required substantial engineering effort. Walter notes that customers wanted Joule functionality across all business lines, forcing SAP to prioritise integration work. The company reached this threshold by the end of 2024, then expanded availability across its customer base.

Early this year, SAP introduced Joule Studio, allowing customers to extend the AI assistant to their own applications. Companies can connect internal tools like organisational management systems or holiday approval workflows to the SAP ecosystem, enabling Joule to access these systems when needed.

“So if you have a company and you have your own, let’s say, organisational tool or maybe your own holiday approval tool, you can actually build that feature as an extensibility to the SAP ecosystem and Joule can go to that when necessary as well,” Walter explains.

Building agent-to-agent communication standards

SAP announced a partnership with Google in April to develop standards for agent-to-agent communication, enabling AI assistants from different providers to work together directly. The collaboration aims to prevent enterprise AI from fragmenting into incompatible vendor systems.

“We know that there are other providers of agents,” Walter explains. “And so we announced together with Google that we were working with them on the agent to agent interoperability, which was a higher layer of interoperability than within the sub-agent layer of technology.”

The partnership extends beyond Google to include Microsoft and AWS. Walter describes the approach as creating “orchestration and connectivity at a higher level” rather than traditional API integrations, allowing SAP’s agents to access capabilities from Google's Agentspace platform and vice versa.

SAP has built an agent builder capability within Joule that creates agents across the company’s business applications. The platform includes pre-built agents for common processes, while Joule Studio enables custom agent development for specific customer requirements.

Walter suggests this approach addresses customer concerns about vendor lock-in. “By having agent to agent as an open standard and having other partners join, we actually at this conference also mentioned that Microsoft and AWS are working with us as well. We can actually make it easier for our customers to have a wider ecosystem without anything being closed.”

Addressing AI performance challenges

SAP supports more than 30 large language models through its Generative AI Hub, creating an optimisation challenge that the company addresses through partnerships with Not Diamond and AI search startup Perplexity. The company has also announced partnerships with Palantir Technologies for data connectivity and Adobe for demand forecasting solutions. The Not Diamond collaboration focuses on prompt optimisation across different AI models.

“Each large language model is like a human. They’re trained to have human-like characteristics, and by the same token, each LLM has its own preference of how it’s spoken to,” Walter explains.

The technical issue stems from how different AI models respond to identical prompts. Instructions optimised for OpenAI’s models may produce suboptimal results when used with Google’s or Anthropic's systems, not because of model quality differences but due to different training approaches.

“If you have a prompt that’s optimised for one language model and then you say, hey, I have a new language model, let me see how well it works. I use the same prompt. The answer might be suboptimal, but it’s not necessarily that the new LLM is not as good, but it’s actually been tuned differently," Walter notes.

Not Diamond's technology translates prompts between language models to ensure performance comparisons reflect actual capabilities rather than prompt compatibility. The partnership will help SAP determine which models perform better for specific business scenarios.

The Perplexity partnership, meanwhile, addresses extending Joule’s knowledge beyond internal company information. While SAP's AI can access corporate documents and policies, many business decisions require external context that enterprise systems cannot provide.

“That’s where Perplexity comes in. You do a search online, you realise SAP Sapphire is a conference that happens in Orlando and this year in Madrid in Europe and the dates, and then it comes back and says, okay, now you can actually figure out if it’s within policy and then go to Concur and book the trip,” Walter explains.

Walter addresses AI hallucination concerns by highlighting SAP’s visibility across customer business processes. The company’s presence in enterprise resource planning, human capital management and customer relationship management provides context that can ground AI responses in actual business data.

“One of the differentiators SAP has is because we have multiple lines of businesses, of products. If a company uses all of our different products, we have a good view of their business processes,” Walter explains. “And so when you use Joule and you access large language models, we can actually ground it into your system.”

SAP structures this approach around three principles: relevant, reliable and responsible AI. The company implements content moderation filters and fact-checking mechanisms to verify information accuracy before presenting results to users.

All of this forms part of SAP’s strategy for Business AI. The company says its innovations, announced at Sapphire, could drive productivity increases of up to 30% for enterprise customers.

“Christian [Klein, SAP’s CEO] mentioned there’s a 30% goal of improvements or efficiencies,” Walter says. “So we’re hoping that with all these capabilities we talked about, we can actually help people get more done at work.”