Deepfake AI Is Rewriting the Rules of Fraud

Social engineering used to rely on a convincing email and a sense of urgency. It now comes with a voice you recognize and a face you trust, generated in seconds by AI. As agentic AI reshapes how attacks are executed behind the scenes, generative AI is doing the same to the oldest attack vector in the field: getting a person to simply do what the attacker wants. This piece looks at how deepfake voice and video cloning have moved from novelty to mainstream fraud tool, what the data shows, and what actually stops it.

From Suspicious Emails to Synthetic Executives

Traditional social engineering worked by exploiting attention: a rushed employee, a spoofed domain, a plausible request. AI has not replaced that playbook so much as upgraded every part of it. Phishing emails are now grammatically flawless and formatted to match the brands they impersonate. Voice cloning tools need as little as three seconds of audio, often pulled from a public earnings call, a conference talk, or a social media clip, to produce a convincing match. Video deepfakes have advanced to the point where they can hold up across a live conference call, not just a pre-recorded clip.

The result is that the two identifiers people have always used to verify identity, the sound of a voice and the sight of a face, can no longer be trusted on their own.

The Scale of the Problem

The FBI logged more than 22,000 AI-related fraud complaints in 2025, totaling over $893 million in losses. US organizations reported $1.1 billion in deepfake-related fraud losses in 2025, up from $360 million the year before, and Deloitte projects generative AI-enabled fraud losses could reach $40 billion annually in the US by 2027. Deepfake-enabled voice phishing surged over 1,600 percent in the first quarter of 2025 compared with the last quarter of 2024, and deepfakes now account for 6.5 percent of all fraud attempts globally, up from 0.1 percent in 2022. Financial services firms are targeted 300 times more often than other industries, and an estimated 400 companies are targeted by CEO fraud attempts every day.

Real Incidents, Real Losses

The largest publicly documented case remains the Arup Hong Kong incident, in which a finance employee transferred $25.6 million after joining a video conference where every other participant, including an apparent CFO, was an AI-generated deepfake. The employee’s early suspicion of phishing was overridden by the visual confirmation of colleagues he recognized.

Smaller, less publicized cases follow the same pattern. A UK energy firm’s CEO authorized three wire transfers totaling roughly $243,000 after receiving calls cloning his German parent company boss’s voice, accent and speech patterns included. A businessman in China lost the equivalent of $622,000 after a face-swapped video call. It impersonated someone he trusted. The fraud came to light when the real contact confirmed the conversation never happened.

Why These Attacks Work

Unaided human detection of AI-generated video sits at around 24.5 percent accuracy, and detection of voice clones is lower still. AI detection tools reach up to 96 percent accuracy in lab conditions. Real-world performance drops 45 to 50 percent outside controlled testing. Among targets actually reached by a voice cloning attempt, 77 percent lost money.

AI Is Reshaping Everyday Phishing Too

Deepfakes reach headlines, but AI’s bigger near-term impact may be on ordinary phishing volume and quality. An industry report shows AI-generated phishing campaigns rose from 4 percent of attacks in November to 56 percent in December. They then hovered near 40 percent in January 2026. These campaigns still rely on familiar mechanics, malicious links, and attachments impersonating Microsoft, Docusign, or HR. Generative AI removes the grammatical and formatting cues once used to reveal them.

Closing the Gap

None of this is unsolvable. Organizations that manage this risk well have rethought verification protocols. Approval workflows and employee reporting practices have also been redesigned. So a single convincing call or video can no longer authorize a high-value action on its own. Most internal teams get stuck here, and fixes are rarely one-size-fits-all.

If your organization has not pressure-tested its wire transfer and approval processes against a deepfake scenario, you should consider remediation. Topgallant Partners can help you identify gaps and design a verification framework aligned with how your business operates.