AI is increasingly being used in attacks such as phishing and social engineering, but it is not always recognised in incident response. That’s according to ISACA's 2026 State of Cyber report, which found that 71% of organisations have not run any AI incident response exercises, while only 3% have mature, formal runbooks for AI-specific incidents – and 30% have not begun to address their response at all.

It comes as almost half of CISOs say they have experienced a deepfake related incident in the last 12 months.

How does AI change incident response and what should companies do about it?

Unprepared Businesses

Many firms are using AI, but they aren’t aware of the risks the technology poses.

Tom Lovell, head of security services at Infinity Group describes how businesses are embracing AI tools and investing in the technology’s capabilities. “But far fewer have updated their incident response plans to reflect the risks AI creates.”

Most security teams have established playbooks for ransomware, phishing and data breaches, but AI-specific scenarios such as deepfake-enabled fraud, misuse of generative AI, prompt injection attacks or sensitive data exposure through AI tools are often not formally covered, says Lovell. As a result, many organisations are “more exposed than they realise,” he says.

Across the board, organisations “aren’t prepared for all that AI has made possible or is capable of,” says Crystal Morin, senior cybersecurity strategist at Sysdig. “They are adopting AI faster than they're preparing to respond when AI turns into the thing attacking them, and that gap can be damaging.”

Morin cites the example of JADEPUFFER. “An attacker pointed an agent at a vulnerable target and stepped away from the keyboard. The agent exploited the flaw, moved laterally, harvested and reused credentials, destroyed a database, and even crafted the ransom note. None of the individual techniques were new, but AI helped connect and run them more efficiently. Current incident response teams aren’t built for attacks that plan, adapt, and execute at machine speed with no human in the loop to slow it down.”

Most incident response plans were written for a world where the question was “which system is compromised?” says James John, incident response manager at Bridewell.  

AI-enabled attacks increasingly ask a different question, he explains: “Which person, or conversation, can we trust?”

“Very few playbooks cover what to do when a director's voice is used to authorise a payment, when the helpdesk resets credentials for someone who sounded exactly like a colleague, or when an attacker is using your own AI tools to map your environment.”

Biggest Gaps

The biggest gap is that many organisations are still treating AI as “a technology project rather than a security and governance challenge,” says Lovell. “They may have policies around AI use, but they haven't tested how they would detect or respond when something goes wrong.”

Lovell describes gaps around incident response runbooks, executive and employee training, ownership of AI-related risks, visibility of shadow AI usage, comprehensive vulnerability management and identity verification processes. “Many organisations still rely on processes that assume what people see or hear can be trusted.”

Meanwhile, many assume their existing playbook already covers AI-driven threats. It doesn't, says Morin. “A standard phishing response assumes a human clicked on a bad link. It says nothing about verifying whether the voice on a call is real, or about an AI agent that was manipulated into taking an action it shouldn't have. Most incident response plans were built around static attack patterns that moved at human speed. AI-enabled attacks break both of those assumptions.”

There’s also a culture gap that teams should not ignore, says Morin. “For years, security teams and annual enterprise training sessions have told employees to spot the fake – bad grammar, the wrong logo, or an email address that’s slightly off. AI removes most of those tells, so today’s employees need to work on different skills.”

AI Incident Response

The gaps are clear, so where should firms start when considering incident response in the AI age?

CISOs need to start by “looking honestly at which AI-enabled attacks could affect their organisation and asking whether their existing incident response plans would actually cope,” says Tracey Hannan-Jones consulting director in information security, UBDS Digital.

Focus on modernising detection capabilities and strengthening identity protocols, while also preparing for rapid containment, advises Andrew Krug, head of security advocacy at Datadog. “That includes enforcing multi-factor authentication, monitoring changes to admin accounts and protecting and testing backups. Also rehearse AI-related scenarios, including phishing or deepfake impersonation that leads to compromised credentials.”

Start with the basics, “because they still matter most,” says John. He believes training should move away from “spot the deepfake” and towards “verify, regardless of how convincing it seems.”

John advises building AI into your existing playbooks rather than writing a separate one “that sits on a shelf.” He recommends adding injects to your exercises, such as a cloned voice on a call to finance, an attacker interrogating your AI assistant with a stolen account, or an agent acting outside its permissions. “Exercising is where you discover the decisions nobody has authority to make, before an attacker forces the issue.”

Hannan-Jones thinks staff training needs to catch up with the current AI-driven threat landscape. “Teaching people to identify phishing through bad grammar or suspicious wording is increasingly outdated. The focus now needs to be on verification. If something involves money, sensitive data or privileged access, verify it through another trusted channel, however convincing the original message, call or video might appear.”

Most importantly, organisations should assume AI will feature in future incidents, advises Lovell. “The goal isn't to stop using AI; it's to ensure governance, controls and response capabilities evolve at the same pace as adoption. The organisations that do this now will be significantly more resilient than those that wait until an AI-related incident occurs."

Kate O'Flaherty Cybersecurity and privacy journalist