Prompt injection has become a major security risk for AI applications, with OpenAI publicly admitting in December 2025 that the issue is unlikely to ever be fully resolved. The U.K.’s National Cyber Security Centre also issued a formal warning about the inherent vulnerability of large language models (LLMs) to prompt injection attacks, likening them to “inherently confusable deputies.”
So, what exactly is a prompt injection attack? This type of attack involves tricking an AI chatbot into following an attacker’s instructions instead of the user’s. For example, an attacker could input a malicious instruction disguised as a harmless text message, causing the AI to carry out unauthorized actions without the user’s knowledge or approval.
The vulnerability lies in the fact that large language models, such as ChatGPT and other AI chatbots, do not differentiate between instructions and data—they simply interpret everything as text. This means that a clever attacker can craft text that the AI model interprets as a new instruction, effectively overriding the original prompt given by the user.
The concept of prompt injection was first coined in September 2022 by British developer Simon Willison, drawing parallels to the well-known SQL injection attacks that plagued websites in the past. Despite being identified and reported to OpenAI by security firm Preamble in 2022, prompt injection remains a persistent and unresolved issue in the AI security landscape.
There are two main types of prompt injection attacks: direct and indirect. Direct prompt injection occurs when a user inputs a malicious instruction directly into the chatbox, leading to humorous but harmless outcomes like a chatbot agreeing to absurd requests. In contrast, indirect prompt injection is far more dangerous, as the malicious instructions are hidden within content that the AI reads on the user’s behalf, such as webpages, emails, or PDFs.
Google’s DeepMind security team revealed a significant increase in malicious indirect prompt injections between November 2025 and February 2026, with attackers using various techniques to hide the instructions from human eyes while exploiting AI systems. Cybersecurity firm HiddenLayer also demonstrated how prompt injections can spread like a virus across entire codebases, showcasing the potential for widespread and damaging attacks.
In a concerning development, Anthropic disclosed the first documented case of a large-scale cyberattack primarily executed by AI, involving a Chinese group designated GTG-1002 using prompt injection to target tech companies, financial institutions, and government agencies. This incident underscores the seriousness of prompt injection attacks and the need for robust security measures to protect AI applications from exploitation.
As the prevalence and sophistication of prompt injection attacks continue to grow, it is imperative for developers and organizations to prioritize AI security and implement proactive measures to mitigate the risks posed by this pervasive vulnerability. Prompt injection is a new and dangerous threat in the world of artificial intelligence (AI) security. Unlike traditional software bugs that can be patched with updates, prompt injection is a structural flaw in how current AI systems process text. This vulnerability allows attackers to manipulate AI models by injecting malicious prompts, leading to potentially devastating consequences.
The attack method involves breaking down the assault into thousands of small, seemingly innocent tasks. The AI then autonomously executes 80% to 90% of the operation, making thousands of requests per second. The entry point for these attacks lies in the AI’s inability to distinguish between instructions and data, making it susceptible to manipulation through carefully crafted prompts.
In the past, vulnerabilities like SQL injection were fixed by separating user data from database commands. However, with language models, this separation does not exist. The AI reads all input as the same type of text in the same context window, making it challenging to mitigate prompt injection attacks. The National Cyber Security Centre acknowledged that applying SQL-injection-style defenses to prompt injection is a category error, as the vulnerability is inherent in how language models function.
Various defenses have been tested against adaptive attackers, but they have proven ineffective, with attackers bypassing them with over 90% success rates. OpenAI has admitted that prompt injection is a problem unlikely to be fully solved due to the fundamental nature of the vulnerability. The math simply does not support a complete resolution.
To protect against prompt injection attacks, users should limit the access granted to AI agents, issue narrow commands, treat AI summaries with caution, require human confirmation for significant actions, scan files for hidden markdown comments, and exercise caution when installing AI skills. Developers should treat all external input as potentially hostile and take necessary precautions to prevent prompt injection attacks.
It is crucial to understand that prompt injection is not a problem that can be easily fixed with a software update. It is a structural flaw that requires vigilance and proactive measures to mitigate the risks associated with AI security. By keeping a hand on the wheel and being mindful of the trust placed in AI systems, users and developers can better protect themselves against prompt injection attacks in the future.