Quick answer
Improper output handling is when an application passes a language model's output to a browser, database, shell, API or other component without validating it first. Because attackers can influence what the model says, unfiltered output turns into classic attacks such as cross-site scripting, SQL injection and remote code execution. It is ranked fifth in the OWASP Top 10 for LLM Applications (2025).
On this page
How It Works
Developers often treat model output as trusted because it comes from their own system. In reality it is shaped by user input, retrieved documents and web content, any of which can carry a prompt injection. If the chat interface renders model output as HTML, injected script becomes cross-site scripting. If the model writes SQL that is executed directly, it becomes SQL injection. If its output is used as a URL the server fetches, it becomes server-side request forgery. If it generates code that is run, or file paths and shell commands that are used as given, the result can be full code execution on the server. Markdown rendering is a common case: an injected image tag can make the user's browser send conversation data to an attacker's server.
Why It Matters
This is where AI risk turns into conventional, well-understood impact. A text-to-SQL feature, an agent that runs code, or a support bot whose answers are inserted into emails and web pages all pass model output across a trust boundary. The vulnerability is in the application rather than the model, so it is fixable with proven controls, but only if the model is treated as an untrusted user.
How to Test for It
- Trace every place model output goes: rendered in a browser, stored, executed, used in queries, URLs, file paths, emails or calls to other systems.
- Use direct and indirect prompt injection to make the model emit payloads for each destination: script tags, SQL fragments, internal URLs, path traversal sequences, shell metacharacters.
- Check whether markdown images and links render automatically and whether they can carry data to external domains.
- Test code execution features for sandbox escape, network access and access to secrets.
- Confirm that structured outputs, such as JSON for tool calls, are validated against a schema before use.
How to Reduce the Risk
Treat model output exactly like untrusted user input. Apply context-aware output encoding before rendering, use parameterised queries and read-only database roles for text-to-SQL, validate structured output against strict schemas, allowlist URLs and domains for any fetch, run generated code in an isolated sandbox with no secrets and limited network access, apply a strict content security policy, and block automatic loading of external images in chat interfaces.
Regulatory and Framework Context
In Australia, improper output handling maps to the same secure development expectations ASD applies to any web application, including its secure-by-design guidance and the Information Security Manual's application development controls. Testing provides evidence for ISO/IEC 42001 and for ASD's guidance on engaging with artificial intelligence.
Latest intel on this topic
- Video: What is Tabnabbing?Embrace The Red · 10 October 2021
- Video: Web Application Security FundamentalsEmbrace The Red · 6 September 2021
- Web Application Security Principles RevisitedEmbrace The Red · 13 February 2020
Updated hourly from the daily briefing.
Common questions
Is improper output handling the same as prompt injection?
No. Prompt injection controls what the model says. Improper output handling is the application trusting what the model says. The two are usually chained together.
Do output guardrails solve it?
Content filters help, but the reliable fix is the same as for any untrusted input: encoding, parameterised queries, schemas and sandboxing at each destination.
Change log
- 13 November 2026First published.