Prompt-injection scanner for Azure AI Foundry

Attack surface: where Azure AI Foundry receives unscanned images

Prompt Flow image nodes. Azure AI Foundry's Prompt Flow (a visual pipeline builder) has a load_image built-in node that accepts image URLs or base64 strings. Flows that accept user-supplied image URLs — for example, a document-summarisation flow where the user provides a PDF page as an image — pass the image bytes directly to the downstream LLM node. Any malicious pixel-level content in that image propagates unchecked through the flow graph. Prompt Flow does not have a built-in injection-detection step in its node catalogue.

Managed Online Endpoints via Inference SDK. Applications that call Foundry-deployed endpoints using the azure-ai-inference SDK (ChatCompletionsClient with ImageContentItem) pass image bytes directly to the model. Apps built this way typically serve external users who can influence the image input — avatars, uploaded documents, screenshots. The SDK call is synchronous and model-agnostic: the same complete() method works for GPT-4o, Phi-4-Vision, and custom fine-tuned endpoints, meaning a single unscanned input path reaches all models in the project.

AI Foundry Playground and evaluation harness. The AI Foundry web UI allows developers and evaluators to upload images directly in the Playground tab for interactive testing. While the Playground is intended for internal use, API keys and project endpoints are often shared across teams in enterprise setups. A malicious image submitted by a low-trust evaluator or tester in the Playground environment can be used to probe the model's susceptibility before designing a production-ready attack. Playground inputs are not scanned by any default content filter specifically for adversarial pixel-level payloads.

Batch inference jobs with image datasets. Foundry supports batch inference via the BatchInferenceClient, which accepts a dataset of image paths or URLs from Azure Blob Storage. AI-powered content moderation pipelines, document processing jobs, and training-data preparation workflows commonly use this pattern. If the input dataset contains adversarially crafted images, the batch job will process them all against the vision model, potentially producing corrupted outputs that flow downstream into fine-tuning datasets, labeled datasets, or automated business decisions.

Integration: Azure AI Inference SDK (Python)

The function below inserts a Glyphward scan gate between the image source and any ChatCompletionsClient.complete() call in a Foundry-backed application. The gate runs before model selection — a single scan covers all vision models in the project.

import base64
import requests
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import SystemMessage, UserMessage, ImageContentItem, TextContentItem
from azure.core.credentials import AzureKeyCredential

FOUNDRY_ENDPOINT = "https://<project>.services.ai.azure.com/models"
FOUNDRY_KEY = "<your-ai-foundry-api-key>"
GLYPHWARD_KEY = "<your-glyphward-api-key>"
GLYPHWARD_THRESHOLD = 70

client = ChatCompletionsClient(
    endpoint=FOUNDRY_ENDPOINT,
    credential=AzureKeyCredential(FOUNDRY_KEY),
)

def scan_image_for_injection(image_bytes: bytes) -> dict:
    """Scan image bytes for multimodal prompt injection before Foundry call."""
    encoded = base64.b64encode(image_bytes).decode()
    resp = requests.post(
        "https://glyphward.com/v1/scan",
        json={"image": encoded, "source": "azure_ai_foundry"},
        headers={"Authorization": f"Bearer {GLYPHWARD_KEY}"},
        timeout=8,
    )
    resp.raise_for_status()
    return resp.json()

def analyse_image_with_foundry(image_bytes: bytes, model: str = "gpt-4o") -> str:
    # Gate: scan before any Foundry model call
    try:
        scan = scan_image_for_injection(image_bytes)
    except Exception:
        # Fail-closed: scanner unreachable → block the request
        raise RuntimeError("Image security check unavailable. Please retry.")

    if scan["score"] >= GLYPHWARD_THRESHOLD:
        raise ValueError(
            f"Image blocked: adversarial content detected "
            f"(score {scan['score']}/100, ref {scan['scan_id']})"
        )

    # Safe — forward to Foundry endpoint
    encoded = base64.b64encode(image_bytes).decode()
    response = client.complete(
        messages=[
            SystemMessage(content="You are a helpful image analysis assistant."),
            UserMessage(content=[
                TextContentItem(text="Analyse this image and summarise its content."),
                ImageContentItem(image_url={"url": f"data:image/png;base64,{encoded}"}),
            ]),
        ],
        model=model,
        max_tokens=512,
    )
    return response.choices[0].message.content

The same scan_image_for_injection() function works identically for any Foundry-managed model that accepts image inputs — swap the model parameter in client.complete() to target Phi-4-Vision, Mistral Pixtral 12B, Llama 3.2 Vision, or a custom fine-tuned endpoint in your project. The scan runs before the model selection, so a single scan gate covers all vision models in your Foundry project.

Get early access

Coverage matrix

Related questions

Further reading

• Prompt-injection scanner for Azure OpenAI Service — Azure OpenAI Service direct API pattern.
• FedRAMP AI security and prompt injection — FedRAMP Moderate/High requirements for AI image input validation.
• Prompt-injection scanner for Microsoft Copilot Studio — Copilot Studio (Power Automate) pattern.
• Multimodal LLM security API — general Glyphward API overview.
• Real-time vs batch prompt-injection scanning — choosing between real-time and batch scan modes.