Skip to content
StrikeCyberStrikeCyber
Field Manual · Attack technique

Sensitive Information Disclosure in LLMs

Reviewed 13 November 2026Attack technique 2 min read

Quick answer

Sensitive information disclosure is when an AI application reveals data the person asking should not see: personal information, credentials, confidential documents or details of other customers. It usually happens because the model or its retrieval layer can reach more data than the user is entitled to. It is ranked second in the OWASP Top 10 for LLM Applications (2025).

On this page
01

How It Works

Language models leak data through several paths. A retrieval-augmented assistant may search a document store using its own broad service account rather than the user's permissions, so a well-phrased question surfaces HR files, board papers or another customer's records. Fine-tuned models can reproduce fragments of their training data, including names, emails or keys that were never meant to be in it. Conversation history, shared caches and logs can mix data between sessions or tenants. And a successful prompt injection can instruct the model to send what it can see to an attacker, for example by embedding it in a link or image URL that the interface fetches automatically.

02

Why It Matters

The model does not understand confidentiality. It answers with whatever is in its context, so every access control decision has to be made before data reaches it. Disclosure through an AI assistant is still a data breach, and copilots connected to email, file shares and CRM systems put a very large amount of data one question away from every user. Data sent to a third-party model provider may also be retained or logged under terms the organisation has not reviewed.

03

How to Test for It

  1. Map every data source the application can reach, and whose identity it uses to reach each one.
  2. Sign in as low-privilege users and ask for data they should not have: other customers' records, HR and finance documents, credentials in shared files.
  3. Test cross-tenant and cross-session isolation by planting unique marker data in one account and trying to retrieve it from another.
  4. Attempt to extract training or fine-tuning data with completion and repetition prompts.
  5. Combine with indirect prompt injection to test exfiltration channels such as rendered images, links and outbound tool calls.
  6. Review logs, analytics and the model provider's retention settings for sensitive data stored outside your control.
04

How to Reduce the Risk

Enforce authorisation in the retrieval layer using the end user's identity, so the model only ever receives data that user may see. Remove or mask personal and secret data before training or fine-tuning, and keep secrets out of documents the assistant indexes. Isolate tenants and sessions, block automatic rendering of external images and untrusted links, apply output filtering for known sensitive patterns, minimise and redact what is logged, and review provider data retention and training settings before go-live.

05

Regulatory and Framework Context

In Australia, personal information disclosed through an AI system is subject to the Privacy Act 1988 and the Australian Privacy Principles, and a serious disclosure may be notifiable under the Notifiable Data Breaches scheme. The OAIC has published guidance on privacy and the use of commercially available AI products, and ASD's guidance on engaging with artificial intelligence highlights data leakage as a core risk.

Latest intel on this topic

Updated hourly from the daily briefing.

Common questions

Can't the system prompt just tell the model not to reveal sensitive data?

That is not a control. Instructions in the prompt can be bypassed. The only reliable approach is to stop the model receiving data the user is not allowed to see.

Do Microsoft 365 Copilot and similar tools respect permissions?

They generally work within the user's existing permissions, which means oversharing in SharePoint, Teams and OneDrive becomes instantly discoverable. Fixing permissions is part of the rollout.

Does fine-tuning on internal data create risk?

Yes. Models can repeat fragments of fine-tuning data. Clean the data first and test the tuned model for leaked training content before release.

Change log

  1. 13 November 2026First published.
1300 654 898