Defence Against the Dark Art of AI Fraud
AI is a divisive topic, and everyone has an opinion on how much of a threat or a benefit it may ultimately prove to be to the way we live our lives. But whatever your personal feelings, there is one thing AI has done that no one can claim is a positive (except, of course, the bad guys):
Lowered the barriers to entry to faking evidence.
The South African financial and corporate sectors are facing a growing threat from the rapid escalation in artificial intelligence-driven fraud. A significant part of the problem is that traditional verification methods are built on physical document checks, static identity data, and good old human intuition. Designed for a world in which faking evidence required massive effort and considerable skill, they are simply no match for the convincing, high value fraud modern generative AI facilitates at scale.
We can no longer afford to pretend it isn’t happening, or to claim ignorance of how criminals are pulling off such widespread and lucrative fraud. To protect assets and maintain operational security, corporate leaders must understand how AI bypasses traditional defences and appreciate the urgent need for smarter, automated verification systems.
The hard reality is that static identity is dead.
ID numbers, tax references, bank statements and utility bills used be all you needed to confirm you are who you say you are and live where you say you live. They were enough to establish trust and credibility because they could be checked and verified against an official database.
Today, relentless data breaches mean untold amounts of personal information have been leaked onto the dark web. Thanks to AI, criminals can weaponise this stolen data, creating completely false documents from scratch.
So, how does it work?
When AI was still in its infancy, criminals used basic photo-editing software to alter documents. Sometimes they got away with it, but a savvy bank or Home Affairs employee could usually smell a rat. These days, fraudsters use sophisticated generative AI models to build new originals, pixel by pixel.
This means they can create:
Realistic physical details such as paper textures, holograms, light reflections and signatures that look authentic and natural.
Tamper-proof file data, such as fake camera info or file creation timestamps, so that a document passes basic digital checks.
Valid personal info, using real, stolen personal details, that dupe computer system database checks.
When a fake bill or bank statement looks real, carries valid hidden data, and uses real personal information, human reviewers and basic software simply can’t tell the difference.
Even more concerning is the rise of real-time voice cloning and video deepfakes. Criminals use these to target everyday operations like resetting passwords, approving money transfers, or authorising emergency payments.
Creating even a semi-realistic, cloned voice used to take days and required hours of studio-quality voice recordings. Today, modern AI means criminals need as little as three seconds of audio, which they easily pull from social media videos, online interviews or recorded phone calls. McAfee Labs researchers found that as little as three seconds of audio produced an 85% voice match, and Microsoft’s own VALL-E research demonstrated the same three-second threshold.
Sophisticated AI tools use this to match tone and speech patterns, even copying breathing rhythms, accents, pitch and subtle speaking habits. The technology has also evolved to the point where it now works fast enough for criminals to hold live, unscripted conversations over the phone without noticeable AI “tells” such as unnatural pauses or out of place responses.
And as an extra layer of insurance, they frequently run fake voices through low-quality phone lines or cell phone networks, taking advantage of any natural static on the line to hide any potential robotic glitches.
There’s no escaping the cold, hard fact that yesterday’s defences are failing against today’s AI:
Basic scans can’t tell the difference between a photo in a genuine document and a perfectly generated AI image.
Simple camera checks, such as taking a selfie or blinking into a camera can be tricked using video streams generated in real time.
Human ears cannot reliably tell a cloned voiced from a real one over a standard phone line.
As human beings, we frequently rely on our “gut feelings.” If something feels off, double-check it. And I’m certainly not suggesting we stop trusting our gut – we just cannot make it a primary line of defence. Counting on staff to spot a cloned voice in the middle of a hectic workday is not a realistic expectation. Criminals know this, which is why high-pressure tactics are common – false claims of an urgent payment deadline, a system emergency or an unexpected deal being approved by an executive who is out of the office.
In stressful situations like these, it’s natural to focus on the urgency instead of question the authenticity of the voice on the line.
The irony of the situation does not escape me:
The problem is AI-led, but the solution, too, lies in smarter, automated protection.
Corporate and financial institutions must change how they verify identify, moving beyond what employees can simply see and hear.
Organisations need:
Hardware-based security instead of words or code. High-risk actions, such as approving the transfer of large sums of money, can no longer rely on email or phone verification, or secret security questions. Physical security keys and trusted devices – items attackers cannot clone over a phone line – are the solution.
We also need deeper audio and image analysis. Sophisticated software has a far greater chance of spotting microscopic digital glitches or tiny inconsistencies that human eyes and ears will miss. In addition, behaviour analysis can look at the ways in which a user interacts with a platform. Automated tools track typing rhythms, mouse-moving habits and device locations, so any significant deviations from regular patterns are instantly flagged as suspicious.
It’s not fair on your employees to expect them be vigilant and diligent enough to spot modern fakes. Businesses must remove the burden of defence from employee intuition to dynamic, multi-factor forensic verification frameworks.
Employing tools that verify devices, track technical data and monitor behavioural patterns can help stop AI scams before any damage is done.