A new international study from Genesys lays bare a growing paradox at the heart of customer experience: consumers want AI to deliver faster, more personalised service, but they also expect strong safeguards—governance that many organisations still lack.

Surveying over 4,000 consumers and 1,600 customer experience and IT leaders across more than ten countries, including the Netherlands, the report finds four in five consumers want clear, organisation-wide rules for how AI systems interact with them. Yet only a third of business leaders say their organisation has a formal AI governance policy in place. At stake is the deployment of agentic AI—systems designed to think, decide and act with minimal human input—which is moving rapidly from experimental to mainstream in customer-facing operations.

Chief among consumer concerns are data transparency and the reliability of AI outputs. Around 37 percent worry about “hallucinations”—when AI systems generate false or misleading information. The concern is shared by 59 percent of customer experience leaders, who view hallucinations as a threat to brand loyalty and trust. While 91 percent agree that oversight protects brand reputation, more than a third admit their organisations are flying blind, with no formal oversight policies in place.

The trust gap is even starker around data handling. Over 80 percent of business leaders say they trust AI with sensitive customer data, but just 36 percent of consumers feel the same. Appetite for AI-powered speed and efficiency remains strong—91 percent of leaders expect agentic AI to make services more effective—but without transparency, these gains risk fuelling consumer scepticism and regulatory pushback.

Industry analysts see both risk and reward. Capgemini forecasts that agentic AI could deliver up to $450 billion in value by 2028, but warns that few organisations are ready to scale. The consultancy reports broad agreement—75 percent of leaders—that human oversight enhances AI’s benefits, and 93 percent see no cost penalty for keeping humans in the loop. The real value, Capgemini argues, lies in transparency and collaboration, not unchecked automation.

Some in the sector are already moving to close the governance gap. A Capgemini–NVIDIA partnership launched earlier this year offers enterprise-ready frameworks for agentic AI adoption, with data governance and compliance embedded into the tooling. Early use cases, such as Norway’s sovereign AI cloud via the Telenor AI Factory, showcase how localised, regulated AI infrastructure can balance innovation with accountability.

Deloitte’s consumer research echoes these findings. While enthusiasm for Gen AI is widespread in homes and workplaces, concerns about privacy and control remain front of mind. Consumers want transparent data practices and tools that let them manage how their information is used. Deloitte argues that trust must be earned through privacy-by-design and user-friendly controls.

The International Association of Privacy Professionals (IAPP) also points to broader regulatory momentum. Drawing on findings from Pew and KPMG, it notes persistent public anxiety about privacy in the AI age, warning that corporate transparency and regulatory clarity are essential if businesses want to scale responsibly. For UK firms, the message is clear: governance is no longer optional. The nation’s AI and data privacy ambitions depend on building systems that are not only powerful, but trusted. That means investing in human–AI collaboration, embedding privacy into design, and communicating clearly with customers about how AI is used and protected.

To lead responsibly, UK organisations should act on four fronts: – Implement clear, organisation-wide AI governance, defining decision rights, data use and accountability. – Preserve human oversight in high-risk areas such as identity verification, financial transactions and sensitive service touchpoints. – Adopt privacy-by-design with controls that allow customers to manage their data and understand AI decisions. – Communicate proactively, explaining the role of AI, the safeguards in place, and the benefits to service quality.

Consumers are not asking for less innovation—they are demanding more responsibility. With the right frameworks, the UK can meet this moment, turning governance from a compliance exercise into a competitive edge.

Created by Amplify: AI-augmented, human-curated content.