Quick answer
Data poisoning is inserting malicious content into the data an AI system learns from or retrieves, so it gives attacker-chosen answers or follows hidden instructions. In retrieval-augmented generation (RAG) systems, an attacker only needs to add or edit a document in the knowledge base to influence answers for everyone.
On this page
How It Works
RAG systems answer questions by retrieving relevant documents and passing them to a language model. If an attacker can place content in the sources being indexed, such as a wiki, shared drive, ticketing system or public web page, their content can be retrieved and trusted. Poisoned documents can change facts, insert links, or carry indirect prompt injection.
Training data poisoning works earlier in the pipeline, corrupting the data used to train or fine-tune a model so it behaves badly under specific conditions.
Why It Matters
Poisoning undermines the trust that makes AI useful. A poisoned internal assistant can give staff wrong procedures, direct customers to fraudulent payment details or quietly leak information through planted instructions. RAG systems also commonly fail to enforce document permissions, letting users retrieve content they should not see.
How to Test for It
- Map every source feeding the knowledge base and who can write to each.
- Plant test content through each write path and check whether it is retrieved and trusted.
- Test whether retrieval respects document-level and tenant permissions.
- Attempt to make the system cite or link to attacker-controlled content.
- Review ingestion, embedding and vector store access controls.
How to Reduce the Risk
Limit who can write to indexed sources, enforce source permissions at retrieval time, track the provenance of every document, monitor for unexpected changes and show users which sources an answer came from.
Regulatory and Framework Context
For Australian organisations, RAG testing is especially important where assistants index documents containing personal information covered by the Privacy Act.
Common questions
What is RAG?
Retrieval-augmented generation: an AI system retrieves relevant documents from a knowledge base and uses them to answer a question, instead of relying only on what the model learned in training.
Is data poisoning in the OWASP Top 10 for LLM Applications?
Yes. Data and model poisoning, and vector and embedding weaknesses, both appear in the 2025 list.
Change log
- 13 November 2026First published.