Our recent look at overprivileged AI agents covered what happens when an agent holds more access than its task requires. Prompt injection is the mechanism that most often turns that excess access into an actual breach. It is the single most cited risk in frameworks covering AI applications, and unlike a lot of AI security concerns, it is not theoretical. It has already produced real, named vulnerabilities in production systems.
What Prompt Injection Actually Is
A prompt injection attack manipulates a large language model or AI agent into following an attacker’s instructions instead of its intended ones. The vulnerability comes from a structural property of how these systems work: an LLM cannot reliably distinguish an instruction from data, because both arrive as the same natural-language text. Tell it what to do and hand it something to read, and it has no built in way to tell the two apart.
Why Indirect Injection Is the Real Enterprise Threat
Direct injection, typing a malicious instruction straight into a chat prompt, is the version most people picture, and it still works against consumer chatbots and loosely secured internal tools. But indirect injection is the pattern behind most serious enterprise incidents. Attackers hide instructions inside content the AI reads on its own: an email, a document, a web page, a file pulled from a repository, or output from another agent. The AI processes that content as part of a normal task and follows the embedded command without the user ever seeing it happen. A related variant, stored injection, plants instructions in an agent’s long-term memory or knowledge base, where they sit dormant until they activate against a completely unrelated future task.
The Lethal Trifecta
Security researchers studying these incidents keep finding the same three conditions present together: the agent has access to private data, it is exposed to untrusted content, and it has some way to communicate externally. Any agent that combines all three, which describes a growing share of agents built to read email, browse the web, or call outside tools, is exploitable in principle. Removing any one leg of that trifecta closes off most known attack paths, which is a big part of why the concept has become a common reference point in agent security design.
This Is Not Theoretical: Real Attacks Already Happened
A malicious instruction hidden in a public Slack channel in 2024 caused an AI assistant to retrieve confidential data and encode it into a link controlled by the attacker. In 2025, a vulnerability nicknamed EchoLeak became the first documented zero-click data exfiltration attack against Microsoft 365 Copilot, leaking sensitive data the moment Copilot processed a booby-trapped email, with no user interaction required. A separate flaw tracked as CVE-2025-54135 let an indirect injection trick an AI coding agent into creating a malicious configuration file, resulting in remote code execution on a developer’s machine. Another attack, dubbed a toxic agent flow, used a manipulated GitHub issue to steer a developer’s agent into private repositories and leak data back out through a public pull request, without ever stealing a credential. Attackers have also smuggled hidden commands into project configuration files like .cursorrules, and in one case seeded false memories into an AI agent that later triggered unauthorized transfers of digital assets.
Why This Is Hard to Fully Fix
No single defense solves prompt injection. Research cited by security teams working this problem shows that adaptive attacks, ones tailored to a specific defense, bypass more than 90 percent of published mitigations. The UK’s National Cyber Security Centre has been direct about this, noting that an LLM’s inability to separate instructions from data may simply be an inherent limitation of the technology rather than a bug that gets patched. Guidance from the Five Eyes intelligence alliance recommends incremental adoption of agentic AI paired with human oversight at consequential decision points, rather than treating the risk as fully solvable today. Frameworks like the OWASP Top 10 for LLM Applications, MITRE ATLAS, and NIST’s AI Risk Management Framework now give security teams a shared way to name and track the risk, which is progress, even though none of them make it disappear.
Closing the Gap
The organizations handling this well are not waiting for a perfect technical fix. They are deciding, deliberately, which agents get access to sensitive data, which get exposed to untrusted content, and which get to act on the outside world, and they are not letting any single agent hold all three by default. What that looks like in practice depends heavily on the specific tools, data, and workflows a given agent touches.
We are here to help ease any concerns regarding AI please reach out to Topgallant Partners.
0image sources
- pexels-thales13-37709121: Photo by Thales
