AI as Both Attacker and Defender — Why I Think This Is a Reality Financial Services Shouldn't Ignore

George Ralph from RFA explores how AI is now both attacker and defender—creating faster, more sophisticated threats while forcing financial services to fight back with AI-driven security they can’t afford to ignore.

In 2026, it is no longer a secret that threat actors are weaponizing AI to launch attacks more sophisticated than anything previously seen in the history of computing.

From deepfake-driven KYC fraud and synthetic identity scams to hyper-personalized spear-phishing targeting C-suite executives, malicious actors are operating at a speed and scale that legacy defenses simply cannot match.

Keeping pace requires that firms fight AI with AI. Forward-thinking financial services firms are already deploying AI-powered security operations capable of analyzing billions of log events daily, detecting anomalous transaction patterns, and flagging insider threat signals in near real-time.

This trend has fascinated me for the last few years, which is why I wrote this piece. I begin by exploring the ways attackers are leveraging AI to craft more complex and sophisticated threats — then turn to how defenders can harness the same technology to fight back.

Key Takeaways

• AI has lowered the barrier for attackers. Sophisticated fraud is now accessible to virtually anyone.

• AI-powered attacks move faster than any human team can respond to with legacy tools.

• Fighting AI with AI is a baseline requirement for financial services firms in 2926 and beyond.

• The AI tools you deploy to defend yourself can also become attack surfaces if left ungoverned.

The Attackers side

With AI tools like ChatGPT, Gemini, and Claude now accessible to everyone, hackers are creating the most sophisticated attacks the industry has ever seen. Here are some of the most common ones you should be aware of.

Deepfake fraud and KYC bypass

One of the most alarming developments is the use of generative AI to produce real-time deepfake video and voice. Attackers are using these tools to impersonate executives, clients, and counterparties.

This allows them to bypass the standard Know Your Customer (KYC) checks that financial institutions have long relied on as a first line of defense. What once required significant technical skill can now be executed with widely available tools in a matter of minutes.

An example of this is a finance worker at Arup who was tricked into transferring $25.6 million after joining a video call where every participant, including the CFO and colleagues, turned out to be AI-generated deepfakes.

Adversarial attacks on AI systems

As financial institutions deploy their own AI models, attackers are developing techniques specifically designed to subvert them.

This includes data poisoning to corrupt the training data that AI models rely on and model manipulation, where adversaries subtly alter model behavior to produce incorrect outputs.

The goal with these attacks is not always to break a system outright, but to quietly degrade its reliability in ways that go unnoticed until significant damage has been done. Such attacks are even more dangerous since the targets cannot instantly know they’re happening.

Synthetic identities

AI is also enabling the creation of synthetic identities. These are fabricated profiles that convincingly pass credit decisioning models.

Synthetic identities are not stolen identities. They are entirely constructed ones, blending real and fictitious data in ways that traditional fraud detection systems are not built to catch.

The financial losses associated with this type of fraud are growing, and many institutions are only beginning to understand the scale of the problem. For instance, U.S. lenders were exposed to $3.3 billion in suspected synthetic identity fraud in just the first half of 2025.

Hyper-personalized phishing

Gone are the days of poorly written phishing emails that ended up in spam folders. Even when they didn’t go to the spam folders, anyone could easily identify and ignore these emails.

With AI tools, attackers can now craft highly personalized messages that reference real relationships, recent transactions, and specific business contexts.

By early 2025, AI-assisted phishing made up over 80% of all observed social engineering attacks globally according to ENISA.

C-suite executives that have historically been harder to deceive are increasingly being targeted today. A convincing email from what appears to be a trusted counterpart, arriving at exactly the right moment, is a formidable threat that every financial firm needs to prepare for.

Automated vulnerability discovery and lateral movement

Perhaps most concerning is the use of AI to automate the discovery of vulnerabilities and navigate cloud environments without detection.

Attackers can now identify weak points in your systems, exploit them, and move laterally through systems at machine speed. This reduces the window between initial access and serious damage to near zero.

How Defenders Must Respond

The good news is that AI is just as powerful, if not more powerful when properly utilized, in the hands of defenders.

The challenge lies in deploying it with the right intent, governance, and urgency. The two main ways defenders are utilizing AI are by scaling operations and leveraging behavioral analytics. Let me explain.

Scaling security operations

Leading firms are already using AI to do what no human team could do alone. Analyzing billions of log events every day, correlating signals across dozens of systems, and surfacing patterns that would otherwise remain buried in noise is now being done by AI.

Such tasks used to take human analysts weeks to complete, if they could complete them at all. AI does it continuously, in real-time.

One of the most valuable applications is detecting so-called "slow-and-low" data exfiltration — a technique where attackers deliberately move small amounts of data over extended periods to stay below the threshold of conventional alerts.

This data exfiltration technique is a patient, methodical approach to theft, and it works precisely because traditional monitoring is not built to connect the dots across long timeframes. AI is.

By tracking subtle patterns across weeks or months of activity, these systems can identify exfiltration attempts that would otherwise go completely unnoticed until it is too late.

Modern security platforms are increasingly integrating these capabilities into unified dashboards that give security teams a consolidated, real-time view of their threat landscape. This has resulted in faster detection, faster response, and significantly less reliance on manual triage.

Behavioral analytics and anomaly detection

AI-powered behavioral analytics are becoming essential for identifying threats from within as much as from outside.

By establishing a dynamic baseline of what "normal" looks like for each user, device, and system, AI can detect subtle deviations that no human analyst and no static rule would catch.

Anomalous transaction patterns such as a sudden spike in high-value transfers at unusual hours can be flagged in near real-time.

Unusual access behaviors, like an employee pulling large volumes of client data outside their normal workflow, can trigger an immediate investigation.

API abuse, which is increasingly being used by attackers to quietly extract data from financial systems, can now be identified before significant damage is done.

Behavioral analytics tools are also increasingly making it easier to detect insider threats, which remain one of the most underreported risks in financial services.

The governance question institutions cannot afford to ignore

Finally, I would like to briefly talk about governance.

As financial services firms integrate large language models into compliance workflows, finance operations, and client-facing tools, these AI solutions become high-value targets in their own right.

Prompt injection attacks were ranked as the number one AI security risk by OWASP in its 2025 Top 10 for LLMs. A zero-click exploit in Microsoft 365 Copilot demonstrated just how real this threat is, allowing attackers to exfiltrate data through a single crafted email.

Moving fast during AI adoption can create new attack surfaces faster than you can secure them.

So, as you deploy, remember that speed without governance is not progress, but increased exposure. Every model you push into production without proper access controls, adversarial testing, and clear accountability frameworks is a potential entry point.