Prompt-injection scanner for Google Vertex AI and Gemini API

Why Gemini creates a larger multimodal PI surface than earlier LLM APIs

Older LLM APIs were text-first; multimodal was a later addition behind a separate endpoint or model flag. Gemini was designed multimodal from the start. The practical consequence: in a Gemini application, multimodal input is the default path, not a feature branch. Images, audio, and video arrive in the same contents list as text, as Part objects with a mime_type and either inline bytes (inline_data) or a File API URI (file_data).

This default-multimodal design means that any application built on Gemini — a chatbot that accepts image uploads, a voice agent that passes audio clips, a document-understanding pipeline that sends PDFs, a screen-reading agent that captures screenshots — is operating on the same PI-exposed surface. The attack class is not a corner case; it is the normal operating mode.

Google's Gemini safety system scores the entire response for harm categories after generation. It does not scan input parts for injected instructions before generation. A typographic PI payload embedded in a user-supplied image goes through the vision encoder, is acted on by the model, and may produce a response that fully satisfies every harm-category safety filter while also following the injected instruction.

The three multimodal injection surfaces in Gemini applications

Prompt injection in Gemini applications arrives through three distinct channels, each with a different intercept point.

1. Inline image and audio parts in generate_content calls. The most direct path: a user uploads an image or audio clip, your code converts it to an inline_data part, and it goes into contents alongside the system prompt. The image or audio bytes reach the multimodal encoder on that turn without any gate between the file upload and the model call. An image with a FigStep payload is indistinguishable from a benign image until the model reads the embedded instruction. The intercept point is server-side, between your file-receipt handler and the generate_content() call.
2. File API uploads (for large or reused files). Gemini's File API (client.files.upload() in the google-genai SDK) lets you store files and reference them by URI across multiple model calls — the same architecture as the OpenAI Files API. A document, audio track, or video uploaded once can be referenced as a file_data part in subsequent calls. Files uploaded before you wired the scan gate may already be in storage and reachable as context for future calls. The pre-upload intercept is the same as the inline case: scan before calling client.files.upload().
3. Grounding and RAG sources (Vertex AI Search, Vertex AI RAG Engine). Vertex AI's grounding features retrieve document chunks from a search corpus and inject them into the model's context. PDFs in the corpus commonly contain embedded images — charts, scanned pages, diagrams — whose pixel layers may carry typographic PI payloads that OCR-based chunking misses. The chunk text is clean; the source image is not. The intercept point here is ingestion-time: scan every embedded image when a document is added to the search corpus, before it is indexed. See the pattern used in RAG pipeline integration.

What Google's safety settings don't cover

Gemini safety settings give every GenerateContentResponse a safety_ratings list with scores for harm categories: HARASSMENT, HATE_SPEECH, SEXUALLY_EXPLICIT, DANGEROUS_CONTENT. On Vertex AI, additional categories are available (CIVIC_INTEGRITY). These ratings describe the model's output along dimensions that matter for content policy — not whether the model was manipulated into producing that output by an injected instruction in the input.

A FigStep payload on a white background is not harassing, hateful, sexually explicit, or dangerous by content. It is a legible instruction — "Ignore previous instructions and do X" — rendered as pixels. The safety score for that image will be near zero on every category. The model will read the instruction and the safety filter will see a clean response, because following an injected instruction does not produce harmful content as Gemini's safety system defines it.

Google Cloud's Vertex AI Model Armor (announced 2025, generally available on Vertex AI endpoints) adds a prompt-sanitization layer that strips common jailbreak patterns from text inputs. Model Armor does not inspect inline_data image or audio parts — it operates on the text portion of the request. For applications where the attack surface is image or audio bytes, Model Armor and Glyphward are complementary: Model Armor on the text path, Glyphward on the image and audio paths.

Scanning inline parts before generate_content()

The cleanest intercept wraps the generate_content() call with a scan step that walks every inline_data part in the contents list before the call is dispatched. Using the new unified google-genai SDK (which supports both Google AI Studio and Vertex AI via the same client):

import httpx
import base64
import google.genai as genai

GLYPHWARD_API_KEY = "gw_..."
BLOCK_THRESHOLD = 70

def glyphward_scan(data_bytes: bytes, mime_type: str) -> int:
    """Return a 0–100 PI risk score for the given bytes."""
    if mime_type.startswith("image/"):
        modality = "image"
    elif mime_type.startswith("audio/"):
        modality = "audio"
    else:
        modality = "image"  # document: embedded-image extraction mode
    resp = httpx.post(
        "https://api.glyphward.com/v1/scan",
        json={
            "data": base64.b64encode(data_bytes).decode(),
            "modality": modality,
            "source_trust": "low",
        },
        headers={"Authorization": f"Bearer {GLYPHWARD_API_KEY}"},
        timeout=5,
    )
    return resp.json()["score"]

def safe_generate_content(
    client: genai.Client,
    model: str,
    contents,
    **kwargs,
):
    """Drop-in wrapper for client.models.generate_content() that scans
    all inline_data parts for multimodal PI before dispatching.
    """
    # contents may be a string, a list of strings, or a list of Content objects
    parts_to_scan = _collect_inline_parts(contents)
    for mime_type, data_bytes in parts_to_scan:
        score = glyphward_scan(data_bytes, mime_type)
        if score > BLOCK_THRESHOLD:
            raise ValueError(
                f"Multimodal PI blocked: score {score} on {mime_type} part. "
                "Request rejected before reaching Gemini API."
            )
    return client.models.generate_content(model=model, contents=contents, **kwargs)

def _collect_inline_parts(contents):
    """Yield (mime_type, bytes) for every inline_data part in contents."""
    if isinstance(contents, str):
        return
    items = contents if isinstance(contents, list) else [contents]
    for item in items:
        if hasattr(item, "parts"):
            for part in item.parts:
                if hasattr(part, "inline_data") and part.inline_data:
                    yield part.inline_data.mime_type, part.inline_data.data
        elif hasattr(item, "inline_data") and item.inline_data:
            yield item.inline_data.mime_type, item.inline_data.data

Replace calls to client.models.generate_content(model=..., contents=...) with safe_generate_content(client, model, contents). The scan adds ~150–200 ms per multimodal part; if the contents list is text-only, the scan loop exits immediately at zero cost. For applications where most requests are text-only with occasional image uploads, the overhead is paid only when there is something to scan.

Pre-upload scanning for the File API

For applications that use Gemini's File API to store images, audio, or video for multi-turn use, the intercept belongs at upload time — before the file is stored and associated with a project. Once a file is in the File API, it can appear in any future model call as a file_data part; a file that passes upload unscanned is a persistent risk surface.

def safe_upload_file(
    client: genai.Client,
    file_bytes: bytes,
    display_name: str,
    mime_type: str,
) -> genai.types.File:
    """Upload a file to the Gemini File API after scanning for multimodal PI.
    Returns the File object, or raises if the scan blocks it.
    """
    score = glyphward_scan(file_bytes, mime_type)
    if score > BLOCK_THRESHOLD:
        raise ValueError(
            f"File '{display_name}' blocked: PI score {score}. "
            "File not uploaded to Gemini File API."
        )
    import io
    return client.files.upload(
        file=io.BytesIO(file_bytes),
        config={"mime_type": mime_type, "display_name": display_name},
    )

If you have already uploaded files before wiring the scan gate, a backfill is the remediation path: list files via client.files.list(), download each one, scan it, and delete files that fail. The File API's list and delete endpoints make this scriptable; a backfill for a typical corpus runs in minutes.

For video files (which Gemini 1.5 Pro and Gemini 2.0 process natively), the scan sends the video in document mode — Glyphward extracts key frames and audio track and returns the maximum score across all extracted components. A video that contains FigStep-class frames in the background of a screen recording produces a flagged result even when the audio track and dominant visual content are benign.

Audio inputs and the Gemini native audio path

Gemini 1.5 Pro and Gemini 2.0 Flash process audio natively — the audio bytes go directly to Gemini's audio encoder rather than being transcribed first. This is architecturally different from a voice agent that routes through Whisper-style ASR before passing text to an LLM. In the native audio path, the full waveform is the model input; the transcription is a byproduct, not the gate.

This matters for audio prompt injection because WhisperInject-class payloads work by encoding instructions at frequencies, intensities, or timing windows that ASR drops from the transcript while the model's audio encoder still processes them. In a Gemini application that passes audio bytes directly, those payloads reach the model's encoder with no transcription step in between — the attack has a shorter path to execution than in a pipeline that routes through a separate STT service.

The scan for audio inline_data parts follows the same pattern as the image case: Glyphward's waveform analysis runs before the generate_content() call. For real-time voice applications, the scan adds one API round-trip per audio segment to the pipeline latency; the Glyphward API returns under 200 ms p95 on audio clips up to 30 seconds.

For longer audio, the File API upload pattern above is the right path: scan before upload, block flagged files before they are stored. See audio prompt-injection detection for the full waveform analysis method and the four audio-PI subtypes it covers.

How Glyphward fits alongside Vertex AI Model Armor

Vertex AI Model Armor operates on the text portions of a request — it strips common jailbreak patterns and sensitive data from prompts and responses. Its input-sanitization layer reads the text strings in the contents list. It does not read inline_data bytes. The two tools cover non-overlapping attack surfaces: Model Armor on text jailbreaks and data leakage; Glyphward on image and audio prompt injection.

The combined architecture is a two-stage pre-request filter:

1. Stage 1 — multimodal scan (Glyphward): POST every inline_data part to /v1/scan. Block if score exceeds threshold.
2. Stage 2 — text sanitization (Model Armor or equivalent): Sanitize the text portions of contents. Block or redact as appropriate.
3. Stage 3 — generate: Call client.models.generate_content() with the cleared contents. Gemini's safety settings run on the output as the final layer.

Both stages run before the model call; neither depends on the other's output. For applications deployed outside Google Cloud (on AWS, GCP + non-Vertex infrastructure, or Cloudflare Workers calling the Gemini API directly via the google-genai SDK), Model Armor may not be available — Glyphward is the multimodal layer regardless of hosting environment. See cross-cloud scanner patterns for the broader discussion of provider-independent multimodal defense.

Glyphward's /v1/scan is a single HTTPS POST with a base64-encoded payload and an API key header. No SDK installation is required. The scan result includes the risk score, the flagged pixel region (for images), the time window (for audio), and per-signal confidences for FigStep, AgentTypo, and WhisperInject classes. Get early access

Related questions

Further reading

• The multimodal prompt-injection threat model for AI product teams (2026) — full threat model and the defender's playbook.
• Prompt-injection scanner for RAG pipelines — ingestion-time scanning for PDF corpora and vector stores, including the Vertex AI RAG Engine pattern.
• Audio prompt-injection detection — the four audio-PI subtypes Glyphward's waveform analysis covers, including the Gemini native-audio path.
• FigStep detection — the typographic attack class Gemini's vision encoder is exposed to through inline_data image parts.
• WhisperInject detection — the audio carrier attack class relevant to Gemini's native audio path.
• Indirect prompt injection via images — the PDF-embedded-image vector that Vertex AI grounding and RAG Engine create.
• Prompt-injection scanner for OpenAI Assistants API — the equivalent pattern for OpenAI's Files API and vision-enabled thread messages.
• OWASP LLM01:2025 multimodal — compliance vocabulary for Gemini application AppSec reviews.
• Multimodal LLM security API — the category-level overview of what a multimodal scanner covers and where it sits in the inference pipeline.