Artificial intelligence is transforming critical workflows across sectors—from diagnosing illnesses to piloting autonomous vehicles—but its rapid adoption is also expanding the attack surface for cybercriminals. As AI systems become more capable, attackers are exploiting the very qualities that make them powerful.
A growing class of threats targets AI’s inputs and training data. Prompt injection attacks, for example, involve feeding AI systems misleading instructions to manipulate outputs. In high-stakes scenarios such as healthcare or autonomous transport, the consequences could be dangerous. Even subtle data manipulations—such as altering financial risk models—can undermine trust and open the door to fraud.
Beyond inputs, attackers are increasingly targeting AI infrastructure. Data poisoning corrupts training datasets, leading to faulty predictions and systemic risks. Model extraction, where adversaries replicate proprietary AI models through repeated queries, enables intellectual property theft and circumvention of costly development.
The rise of generative AI has supercharged these risks. According to Microsoft, state-backed actors from Russia, China, Iran and North Korea are using AI to automate phishing, clone government officials using deepfakes and craft sophisticated malware. Phishing attempts have surged by over 1,200 percent since generative AI’s emergence, with more than 200 AI-generated fake content incidents recorded in July 2025 alone.
The attacks are also becoming more personalised. Cybercriminals scrape social media and corporate websites to tailor phishing messages, increasing their success rate. North Korean IT operatives have reportedly used deepfakes to deceive recruiters and influence internal systems, while adaptive malware—capable of mutating to evade detection—marks a new phase in cybercrime.
Emerging offensive tools such as HexStrike-AI, which pairs large language models with hacking utilities, have amplified these risks. HexStrike-AI automates reconnaissance and attack execution, enabling faster and more efficient exploitation of vulnerabilities like those in Citrix systems. Fortinet reports that automated cyber scans now exceed 36,000 per second, contributing to a sharp rise in stolen credentials.
Leading AI firms are responding. Anthropic recently revealed it had blocked attempts to misuse its Claude AI model to generate phishing content and malicious code. The company’s transparency highlights the escalating arms race in AI security and the need to embed safeguards early in development.
For organisations, the message is clear: AI offers vast potential but demands robust, adaptive security. Monitoring for anomalous inputs and outputs, securing training data and proprietary models, and educating employees on AI-driven scams are all vital. So too is the deployment of threat detection systems that are AI-aware.
While the cyber threat landscape is intensifying, it need not curtail innovation. Instead, it presents an opportunity to embed security as a foundation of AI development. With proactive strategies, the UK and others can maintain trust, protect assets and remain global leaders in responsible AI advancement.
Created by Amplify: AI-augmented, human-curated content.