OWASP LLM05:2025 Improper Output Handling — multimodal dimension

The four multimodal attack surfaces in LLM05 Improper Output Handling

1. VLM-to-SQL pipelines — image fields that feed structured query generation. Document intelligence pipelines — invoice processing, form OCR, ID verification, medical record extraction — commonly use a VLM to extract structured fields from a document image and then construct a SQL query to persist or retrieve records from a database. The LLM05 text baseline addresses SQL injection in the model's output string: if the VLM returns a value containing SQL metacharacters, output sanitisation catches it. The multimodal extension: an adversarially crafted invoice image can instruct the VLM to return a structurally valid JSON field value that is simultaneously a SQL injection payload — for example, a "vendor name" field rendered as Acme Corp'; DROP TABLE invoices;--. If the VLM is steered by pixel-level instructions in the image, the injection arrives at the output sanitisation layer disguised as a legitimate structured field value, not a raw SQL fragment. Parameterised queries stop naive SQL injection from VLM text output; they do not stop a VLM that was instructed by the image to emit a specific payload value in a specific JSON key that the application code unwraps and passes as a query parameter.

2. VLM-to-code interpreter — image descriptions that trigger code execution. Agentic pipelines built on LangChain, LlamaIndex, AutoGen, and CrewAI commonly include steps where a VLM processes a screenshot or diagram to generate a Python code block, a shell command, or a tool call that is then executed by a code interpreter or subprocess. The LLM05 risk here is that VLM-generated code is executed with insufficient sandboxing. The multimodal injection path: an adversarial image (a diagram, a screenshot, a scanned document) instructs the VLM to emit a specific shell command, file operation, or API call in its output. The code generation pattern is intended behaviour — the pipeline is designed to take VLM output and execute it. The adversarial image subverts the intended semantics of that output by making the VLM emit attacker-controlled commands rather than the task-appropriate commands the pipeline operator intended. Sandboxing the code interpreter (LLM05's recommended mitigation) limits blast radius but does not prevent the VLM from producing and executing attacker-specified code within the sandbox's permission boundary.

3. VLM-to-API client — structured output that triggers SSRF or credential exfiltration. Automation pipelines that process visual content from external sources — customer-submitted screenshots, uploaded product images, external web page captures — often use a VLM to extract a URL, an API endpoint reference, or an action identifier from the visual content, then invoke that URL or API endpoint. The LLM05 SSRF surface: an adversarial image instructs the VLM to extract a URL that points to an internal service, metadata endpoint (AWS 169.254.169.254, GCP metadata server), or attacker-controlled webhook. If the application uses VLM output to construct an HTTP request without allowlisting permitted domains, the adversarial image creates an SSRF chain through the VLM output handling layer. This is distinct from classical SSRF — the attacker does not control a URL parameter directly; they control an image that steers a VLM to produce their desired URL as an extracted structured field.

4. VLM-to-rendered-HTML — image analysis output injected into web application responses. Customer-facing applications that allow users to upload images and display AI-generated descriptions, captions, or analyses — product catalogues, social platforms, document review portals — face the LLM05 XSS risk if VLM output is rendered as HTML without escaping. The multimodal injection: a user-uploaded image crafted with pixel-level instructions causes the VLM to produce an output string containing JavaScript payloads, HTML injection, or markdown links pointing to phishing URLs. The application's HTML rendering layer — designed to display a VLM-generated product description or image caption — renders the attacker-specified content. Standard LLM05 output encoding (HTML entity escaping, Content-Security-Policy headers) mitigates the rendering step; the adversarial image still executes the injected instruction within the model's response before the output sanitisation layer sees it.

Integration: pre-VLM scan gate for multimodal output handling pipelines

import base64
import requests
import openai
import json

GLYPHWARD_KEY = "<your-glyphward-api-key>"
GLYPHWARD_THRESHOLD = 65

client = openai.OpenAI()


def scan_image_before_vlm(image_bytes: bytes, source: str = "pipeline") -> dict:
    """Scan image for adversarial PI before passing to VLM in output-handling chain."""
    encoded = base64.b64encode(image_bytes).decode()
    resp = requests.post(
        "https://glyphward.com/v1/scan",
        json={"image": encoded, "source": source},
        headers={"Authorization": f"Bearer {GLYPHWARD_KEY}"},
        timeout=8,
    )
    resp.raise_for_status()
    return resp.json()


def extract_structured_fields_safe(
    image_bytes: bytes,
    extraction_schema: dict,
    system_prompt: str,
) -> dict:
    """
    LLM05 multimodal pattern: scan image BEFORE VLM structured output call.

    Returns {'status': 'ok', 'fields': {...}} or {'status': 'blocked', 'reason': '...'}.
    Apply output sanitisation to 'fields' values AFTER this function returns 'ok'.
    """
    scan = scan_image_before_vlm(image_bytes, source="structured_extraction")
    if scan["score"] >= GLYPHWARD_THRESHOLD:
        return {
            "status": "blocked",
            "reason": "adversarial_image_detected",
            "score": scan["score"],
            "scan_id": scan["scan_id"],
            "action": "route_to_manual_review",
        }

    encoded = base64.b64encode(image_bytes).decode()
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": system_prompt},
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": f"Extract fields matching this schema: {json.dumps(extraction_schema)}. Return JSON only.",
                    },
                    {
                        "type": "image_url",
                        "image_url": {"url": f"data:image/png;base64,{encoded}"},
                    },
                ],
            },
        ],
        response_format={"type": "json_object"},
        max_tokens=512,
    )

    raw_output = response.choices[0].message.content

    # Output sanitisation layer (LLM05 baseline — still necessary after input scan)
    try:
        fields = json.loads(raw_output)
    except json.JSONDecodeError:
        return {"status": "blocked", "reason": "invalid_json_output"}

    # Sanitise string values — strip SQL metacharacters, HTML, etc.
    sanitised = {}
    for key, value in fields.items():
        if isinstance(value, str):
            # Strip characters with SQL injection significance
            sanitised[key] = value.replace("'", "''").replace(";", "").replace("--", "")
        else:
            sanitised[key] = value

    return {"status": "ok", "fields": sanitised, "scan_id": scan["scan_id"]}

The pattern above applies both layers: Glyphward pre-scan at the image input layer (blocking adversarial images before the VLM call) and output sanitisation at the model output layer (stripping SQL metacharacters from extracted field values). Both layers are necessary because they address different threat vectors — the pre-scan addresses adversarial steering of model behaviour; the output sanitisation addresses non-adversarial model errors and edge-case output formatting. For SQL persistence, use parameterised queries with the sanitised field values — never string-interpolate model output into SQL statements regardless of whether a pre-scan was performed. For HTML rendering, apply HTML entity encoding to all VLM output strings before inserting into page templates. Get early access

Coverage matrix

Related questions

Further reading

• OWASP LLM01 Prompt Injection — multimodal dimension — the root cause that feeds LLM05 chains via adversarial image and audio inputs
• OWASP LLM08 Vector and Embedding Weaknesses — multimodal dimension — adversarial images in CLIP-based RAG indexes that compound LLM05 risks in retrieval pipelines
• OWASP LLM09 Misinformation — multimodal dimension — VLMs generating false authoritative output triggered by adversarial input images
• Agentic RAG pipeline prompt injection — the specific LLM05 risk in retrieval-augmented generation pipelines that process multimodal documents
• Multimodal AI security checklist — structured review covering LLM01, LLM05, LLM08, and LLM09 across image and audio pipeline surfaces