Data poisoning was long discussed primarily in the context of adversarial ML, training datasets, and controlled research scenarios — as a theoretical possibility whose realisation would require significant resources and complex conditions. As recently as early 2025, most teams treated it as an academic conversation: important for the research community, but not always a priority for protecting production systems.
Since then, the situation has escalated rapidly. Real and research-demonstrated cases have emerged: backdoors planted through comments in GitHub repositories, hidden instructions embedded in MCP tool descriptions, poisoned content in search results that agents consume as trusted sources. Poisoning has moved beyond the training pipeline and begun appearing wherever models receive context at runtime.
For teams developing or integrating GenAI solutions, this means that protection against data poisoning can no longer be deferred.
What Has Changed
The classic data poisoning model was relatively straightforward: manipulated or malicious examples are inserted into a training set, which subsequently influences model behaviour. This could result in backdoor behaviour, systematic classification errors, bias, or quality degradation.
Today, a model rarely operates in isolation. It queries RAG systems, corporate knowledge bases, vector databases, APIs, external tools, MCP servers, synthetic data pipelines, and internal business systems. The attack surface is no longer limited to the training pipeline — risk arises at any point where the system receives external context or uses data to make decisions.
Poisoned data can enter a fine-tuning dataset through an open-source repository — undetected, alongside legitimate code or documentation. It can be indexed by a RAG system via web sources or internal files that the system treats as trusted by default. It can be concealed in a tool description loaded by an agent as a trusted tool, activating only after deployment. And it can propagate through synthetic data — amplifying errors when generated content is recycled back into training samples.
Why Data Poisoning Is Especially Critical for Agentic Systems
Data poisoning becomes significantly more dangerous when a model operates not only in an informational capacity but also within an operational loop.
An LLM agent that answers questions in an isolated environment has limited potential for harm. But an agent capable of executing SQL queries, working with a CRM, triggering workflows, interacting with file storage, ticketing systems, or financial services becomes part of the corporate operational infrastructure. In that context, a poisoned instruction or piece of context no longer determines the model’s response — it determines the real action the system will take.
This is where the line between a “model error” and the execution of an unintended or harmful action by the system is crossed.
RAG deserves particular attention. A corporate knowledge base should not automatically be treated as a trusted source simply because it is internal. It is important to define who can add, modify, or delete documents; how currency is validated; whether data provenance is maintained; and whether hidden instructions can reach the query context.
The tool layer itself must also be treated as part of the attack surface. If an agent receives a tool description, endpoint, or available action as part of its context, these elements require verification, version control, and access restrictions.
Expert Commentary
Data poisoning is a relatively new concept, and it is only this year that we have seen real-world cases of these risks materialising. Defence mechanisms are actively being developed. There are promising approaches, but no “gold standard” that fully eliminates the problem.
For enterprise clients — or in any situation where a data breach or unintended agent action could have serious negative consequences — hygiene in the development and deployment of agentic solutions becomes critical. If an agent is architecturally constrained from causing harm (for example, it has no internet access beyond pre-approved endpoints, interacts with databases exclusively through read-only APIs, and so on), one of the key attack surfaces with operational damage potential that remains is the human who interacts with that agent.
— AI/ML Architect, AM-BITS
How to Minimise Risk
In enterprise scenarios, maximum agent autonomy is not always an advantage. In many cases, the value of an AI solution lies in having the agent controllably accelerate specific stages: finding information, preparing a summary, classifying a request, forming a recommendation, or suggesting the next step — without independently executing the entire process from analysis to action. Restricting agent autonomy is an effective way to reduce the risk of systemic LLM agent errors, including data poisoning attacks.
During testing, in addition to the “normal scenario,” teams should evaluate the “worst-case scenario” — one in which retrieved context, tool metadata, or user input has been compromised. Red teaming exercises should cover not only prompt injection but also poisoned context in documents, tool descriptions, metadata, datasets, and synthetic data.
Targeted training for employees who work with agents is an essential element of security uplift.
None of these approaches provides complete protection, and data poisoning has no universal antidote. However, impact can be substantially reduced when the architecture clearly defines the agent’s access scope, degree of autonomy, and human oversight checkpoints.
