Prompt injection scanner for HuggingFace Inference Endpoints

The four multimodal attack surfaces in HuggingFace Inference Endpoints

1. Serverless Inference API — shared VLM endpoints with no per-tenant isolation. HuggingFace's Serverless Inference API provides free or low-rate-limited access to popular VLMs hosted on the Hub, including LLaVA, IDEFICS, and Phi-3-Vision, via a simple OpenAI-compatible chat/completions endpoint. Teams prototyping multimodal features frequently start with the Serverless API before graduating to dedicated endpoints — and many production-adjacent integrations remain on the serverless tier to avoid dedicated endpoint costs. Because the serverless tier auto-provisions shared compute, there is no per-tenant ingestion filter: every image submitted to https://api-inference.huggingface.co/models/<model-id> is passed directly to the model's input pipeline. An adversarially crafted image embedded in a user-uploaded document or screenshotted from a malicious web page will reach the model's vision encoder unscanned, giving the injection payload a direct channel to the model's instruction-following stack.

2. Dedicated Inference Endpoints with custom inference handlers. Dedicated Inference Endpoints allow teams to deploy any Hub model with a custom handler.py that pre-processes inputs, applies custom tokenisation, or adds application-specific logic before model inference. A common pattern is to override the default pipeline handler to add structured output parsing, custom system prompt injection, or multi-image batch support. These custom handlers run inside the endpoint container with no platform-enforced content inspection step — HuggingFace's platform trusts that the handler author has implemented their own input validation. In practice, most custom handlers focus on formatting and parsing, not security scanning. An image that arrives at a dedicated endpoint with a custom handler bypasses both the default HuggingFace content filters (which apply at the model-hosted-API layer, not the custom container layer) and any security scanning that the handler author has not explicitly added. The combination of elevated capabilities (custom system prompts, tool-call formatting, structured output) and absent input scanning makes custom-handler endpoints the highest-risk HuggingFace deployment configuration.

3. Multi-image batch inference for document processing pipelines. Both the Serverless API and dedicated endpoints support multi-image batching — submitting an array of images in a single request for batch extraction (invoices, product catalogue pages, scanned forms, slides). Document processing pipelines built on HuggingFace Inference Endpoints commonly fetch a set of PDF page images from S3, GCS, or a document management system and submit them as a batch to a VLM for structured field extraction. In a batch of 30 PDF page images, a single adversarially crafted page — planted by a supplier who can influence what documents enter the pipeline, or introduced through a compromised ingestion step — will be processed alongside the legitimate pages with the same model context and system prompt. The adversarial instruction on that one page can override the extraction behaviour for the entire batch or for all subsequent pages in a long-context multi-image call, since VLMs process images sequentially within a single context window. Scanning images individually before batch submission is the only reliable mitigation.

4. HuggingFace Spaces inference backends exposed as API targets. HuggingFace Spaces frequently serve dual duty as both interactive demos and production inference backends — teams embed a Gradio or Streamlit Space's /predict endpoint in their application code, bypassing the Spaces UI. Spaces running VLMs (a common pattern for open-source multimodal demos) accept arbitrary image inputs from the application layer with no HuggingFace platform-level PI scanning; the Space owner is solely responsible for input validation. When a Space's endpoint is used as a production backend — often for cost reasons, since public Spaces run on free HuggingFace infrastructure — it inherits all the access-control and input-scanning gaps of the underlying Gradio or FastAPI handler. Any application POSTing images to a Space-backed VLM endpoint is exposed to the same multimodal injection risk as a dedicated endpoint, with the additional risk that the Space's system prompt may be visible in the public source code, giving attackers prior knowledge of the instruction context they are trying to override.

Integration: HuggingFace Inference Client with Glyphward pre-scan gate

import base64
import requests
from huggingface_hub import InferenceClient

HF_TOKEN = "<your-huggingface-token>"
ENDPOINT_URL = "<your-dedicated-endpoint-url>"  # or use model ID for serverless

GLYPHWARD_KEY = "<your-glyphward-api-key>"
GLYPHWARD_THRESHOLD = 65  # fail-closed for document and user-upload workloads

client = InferenceClient(
    model=ENDPOINT_URL,
    token=HF_TOKEN,
)


def scan_image_for_injection(image_bytes: bytes) -> dict:
    """Scan image bytes for multimodal prompt injection before HF inference call."""
    encoded = base64.b64encode(image_bytes).decode()
    resp = requests.post(
        "https://glyphward.com/v1/scan",
        json={"image": encoded, "source": "huggingface_inference_endpoints"},
        headers={"Authorization": f"Bearer {GLYPHWARD_KEY}"},
        timeout=8,
    )
    resp.raise_for_status()
    return resp.json()


def describe_image(image_bytes: bytes, prompt: str = "Describe the content of this image.") -> str:
    """Gate image through Glyphward before forwarding to HuggingFace Inference Endpoint."""
    try:
        scan = scan_image_for_injection(image_bytes)
    except Exception as exc:
        raise RuntimeError(
            "Image security scan unavailable — request blocked. Please retry."
        ) from exc

    if scan["score"] >= GLYPHWARD_THRESHOLD:
        raise ValueError(
            f"Image blocked: adversarial content detected "
            f"(score {scan['score']}/100, scan_id={scan['scan_id']})"
        )

    # Safe — forward to HuggingFace Inference Endpoint
    encoded = base64.b64encode(image_bytes).decode()
    response = client.chat_completion(
        messages=[
            {
                "role": "user",
                "content": [
                    {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{encoded}"}},
                    {"type": "text", "text": prompt},
                ],
            }
        ],
        max_tokens=512,
    )
    return response.choices[0].message.content


def process_document_batch(image_bytes_list: list[bytes], prompt: str) -> list[str]:
    """Scan all batch images before any inference call — reject entire batch on first fail."""
    for i, img_bytes in enumerate(image_bytes_list):
        scan = scan_image_for_injection(img_bytes)
        if scan["score"] >= GLYPHWARD_THRESHOLD:
            raise ValueError(
                f"Batch aborted: image at index {i} blocked "
                f"(score {scan['score']}/100, scan_id={scan['scan_id']})"
            )
    # All images passed — submit to endpoint
    return [describe_image(img, prompt) for img in image_bytes_list]

The process_document_batch() function scans all images in a batch before submitting any to the endpoint — this is the correct pattern for document pipeline batch jobs, because a single adversarial image in the middle of a 30-page batch should abort the entire batch, not just its own inference call. Partial batches (rejecting only the flagged image and continuing with the rest) are appropriate only if your downstream processing is strictly per-page with no shared context; most VLM multi-image calls share context across all images in the request. The source field in the scan request lets Glyphward's corpus tagging distinguish HuggingFace Inference Endpoint traffic for reporting and false-positive tuning. For serverless API use, replace ENDPOINT_URL with the model ID string (e.g. "llava-hf/llava-1.6-mistral-7b-hf"); the scan gate is identical. Get early access

Coverage matrix

Related questions

Further reading

• Prompt injection scanner for HuggingFace Transformers — the same multimodal gate pattern for self-hosted Transformers pipelines (local GPU or CPU inference, no external API call)
• Prompt injection scanner for Gradio and Streamlit apps — covering gr.Image() and st.file_uploader() entry points, often backed by HuggingFace Inference Endpoints or Spaces
• Vision-language model security — architecture overview of how VLM vision encoders process adversarial pixel patterns and why CLIP-space embeddings are the correct inspection point
• Agentic RAG pipeline prompt injection — how adversarial document images propagate through retrieval-augmented generation pipelines, including HuggingFace-hosted embedding and reranking models
• Prompt injection in autonomous AI research agents — covering multi-image batch injection in long-running research agents that use HuggingFace models to process arXiv papers and web-scraped content