Prompt injection scanner for IBM watsonx

The four multimodal attack surfaces in IBM watsonx

1. watsonx.ai Document AI — PDF and image ingestion for RAG. watsonx.ai RAG pipelines typically ingest documents through Watson Discovery or direct IBM Cloud Object Storage indexing. Watson Discovery converts PDF pages to images internally before extracting text; a PDF with adversarial pixel-level content embedded in a page image will have its injected instruction surfaced alongside the legitimate document text in the retrieval context. Pipelines using the ibm-watson-discovery SDK or the watsonx.ai Python library to index COS buckets receive these pages without any intermediate visual-content inspection. When the retrieved context is passed to a Granite or Llama foundation model via the ModelInference API, the adversarial instruction is presented as trusted retrieval context — a textbook indirect prompt injection executed through the document ingestion layer.

2. watsonx Assistants multimodal file uploads from enterprise users. IBM watsonx Assistants (formerly Watson Assistant) supports custom extensions and skills that accept file uploads from enterprise end-users — HR document reviewers, procurement analysts, and compliance teams routinely upload scanned contracts, invoices, and forms. Each uploaded file is processed by the assistant's configured skill, which may pass image content to a watsonx.ai foundation model for analysis. An employee or external counterparty who can influence the content of an uploaded file — even a page embedded inside a multi-page PDF — can inject instructions that override the assistant's system prompt or extract information from the conversation context. watsonx Assistants' built-in content moderation operates on conversational text, not on the pixel content of uploaded files.

3. IBM Cloud Object Storage and watsonx.data lakehouse — adversarial images in document stores. watsonx.data is IBM's open lakehouse built on Apache Iceberg and Presto. Enterprise teams store Parquet and Delta tables in IBM COS buckets; those tables frequently include binary columns containing document images, scanned receipts, or product photographs that are passed to vision-capable foundation models for enrichment queries. An attacker who can write a single adversarially crafted image into a COS bucket — through a compromised upload workflow, a poisoned vendor data feed, or a misconfigured bucket policy — can corrupt an entire batch enrichment run. Because watsonx.data's query engine processes these images as opaque binary data and passes them to the foundation model without inspection, the injection travels undetected from the data lakehouse layer into the AI output layer.

4. Watson Visual Recognition and Granite Vision in IBM RPA and workflow automation. IBM RPA (Robotic Process Automation) and business automation workflows built on IBM Cloud Pak for Business Automation frequently use watsonx.ai foundation models with image input capabilities — including Granite Vision and partner vision models available on the watsonx.ai model library — to extract structured data from scanned forms, invoices, and screenshots captured during automation runs. These automations operate with elevated permissions: they may write to ERP systems, trigger financial approvals, or update HR records. A manipulated screenshot or scanned form injected into an RPA workflow can redirect the model's output to produce structured extraction results that cause the automation to perform unintended privileged actions. Unlike user-facing chatbot deployments, RPA workflows often run unattended with no human review step between model output and downstream action execution.

Integration: watsonx.ai Python SDK with Glyphward pre-scan gate

import base64
import requests
from ibm_watsonx_ai import Credentials
from ibm_watsonx_ai.foundation_models import ModelInference
from ibm_watsonx_ai.metanames import GenTextParamsMetaNames as GenParams

IBM_API_KEY = "<your-ibm-cloud-api-key>"
IBM_PROJECT_ID = "<your-watsonx-project-id>"
IBM_REGION_URL = "https://us-south.ml.cloud.ibm.com"

GLYPHWARD_KEY = "<your-glyphward-api-key>"
GLYPHWARD_THRESHOLD = 65  # fail-closed threshold for enterprise document content

credentials = Credentials(
    url=IBM_REGION_URL,
    api_key=IBM_API_KEY,
)

model = ModelInference(
    model_id="ibm/granite-vision-3-2-2b",
    credentials=credentials,
    project_id=IBM_PROJECT_ID,
    params={
        GenParams.MAX_NEW_TOKENS: 512,
        GenParams.TEMPERATURE: 0.0,
    },
)


def scan_image_for_injection(image_bytes: bytes) -> dict:
    """Scan image bytes for multimodal prompt injection before watsonx.ai call."""
    encoded = base64.b64encode(image_bytes).decode()
    resp = requests.post(
        "https://glyphward.com/v1/scan",
        json={"image": encoded, "source": "ibm_watsonx"},
        headers={"Authorization": f"Bearer {GLYPHWARD_KEY}"},
        timeout=8,
    )
    resp.raise_for_status()
    return resp.json()


def analyse_document_image(image_bytes: bytes, prompt: str = "Extract all text and summarise this document.") -> str:
    """Gate image through Glyphward before passing to watsonx.ai foundation model."""
    # Fail-closed: if scanner is unreachable, block the request
    try:
        scan = scan_image_for_injection(image_bytes)
    except Exception as exc:
        raise RuntimeError(
            "Image security check unavailable — request blocked. Please retry."
        ) from exc

    if scan["score"] >= GLYPHWARD_THRESHOLD:
        raise ValueError(
            f"Image blocked: adversarial content detected "
            f"(score {scan['score']}/100, scan_id={scan['scan_id']})"
        )

    # Safe — forward to watsonx.ai Granite Vision model
    encoded = base64.b64encode(image_bytes).decode()
    messages = [
        {
            "role": "user",
            "content": [
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/png;base64,{encoded}"},
                },
                {"type": "text", "text": prompt},
            ],
        }
    ]
    response = model.chat(messages=messages)
    return response["choices"][0]["message"]["content"]

The scan_image_for_injection() gate runs before every model.chat() call regardless of the source of the image — whether it arrives from a COS document bucket, a watsonx Assistants file upload, or an RPA screenshot capture. The GLYPHWARD_THRESHOLD of 65 is deliberately conservative for enterprise document content, where false negatives carry higher regulatory risk than false positives; raise it only after validating false-positive rates against your specific document corpus. The fail-closed pattern on scanner failure ensures that a transient network error never silently allows an unscanned image through to the foundation model. Swap model_id from ibm/granite-vision-3-2-2b to any other vision-capable model in the watsonx.ai model library — the same scan gate applies.

Get early access

Coverage matrix

Related questions

Further reading

• Prompt injection scanner for Azure AI Foundry — the same multimodal gate pattern applied to Phi-4-Vision, Llama 3, and GPT-4o on Microsoft's multi-model hub
• Prompt injection scanner for Vertex AI Agent Builder — closing image attack surfaces in Google Cloud's enterprise agent orchestration layer
• Prompt injection scanner for AWS Bedrock Agents — scanning image inputs in Bedrock multi-agent pipelines and Knowledge Base document ingestion
• Agentic RAG pipeline prompt injection — a deep-dive into how adversarial documents propagate through retrieval-augmented generation architectures, including Watson Discovery-backed pipelines
• Prompt injection prevention best practices — platform-agnostic defence-in-depth patterns including input validation, output monitoring, and the NIST AI RMF controls that apply to multimodal injection risks