Prompt injection in highway bridge inspection AI

Four adversarial injection surfaces in highway bridge inspection AI

1. Drone RGB camera crack detection AI (Fugro BridgeView, AECOM BridgeScan, DroneDeploy for Infrastructure)

Unmanned aerial vehicle (UAV) inspection of highway bridges uses RGB cameras and photogrammetry to acquire high-resolution orthomosaic images of bridge decks, girders, piers, and abutments, which AI classifiers process to detect and characterise surface cracks, spalling, delamination, and efflorescence. Fugro BridgeView uses a multi-rotor drone equipped with a 61MP camera to image bridge structural elements from 0.5–2m standoff distance at ground sampling distances of 0.5–2mm per pixel, generating georeferenced orthomosaic TIFF images that a deep convolutional crack detection AI — typically a U-Net semantic segmentation architecture trained on the CrackForest, AigleRN, and CODEBRIM labelled concrete crack datasets — classifies into crack type (transverse deck cracks / longitudinal deck cracks / diagonal shear cracks in girders / map cracking indicating alkali-silica reaction / efflorescence indicating water infiltration paths) and crack width (estimated from the pixel width at the image ground sampling distance, critical threshold: 0.3mm in concrete, 0.025mm in steel welds for bridge element rating per AASHTO Manual for Bridge Element Inspection (MBE)).

An adversarial perturbation on a rendered bridge orthomosaic TIFF that suppresses a hairline diagonal shear crack in a reinforced concrete pier cap — smoothing the thin dark-pixel linear crack signature against the concrete texture background using a ±6 DN pixel shift within the JPEG compression noise floor — causes the crack detection AI to classify the pier cap as “no cracks observed, Element Condition State 1 (good)” rather than “diagonal shear cracking present, Element Condition State 3 (serious)” in the FHWA-required bridge element inspection report. A pier cap rated Condition State 3 requires a load posting review under AASHTO LRFR (Load and Resistance Factor Rating) to assess whether the legal posting load limit must be reduced to protect remaining structural capacity; adversarial suppression of the crack detection prevents this review trigger. Diagonal shear cracking in a reinforced concrete pier cap is a precursor to shear failure — a brittle collapse mode with no significant warning deformation prior to failure, distinguishing it from ductile flexural failure modes that produce visible permanent deflection before collapse.

2. LiDAR point-cloud structural deformation AI (Leica ScanStation, RIEGL VZ-2000i, Trimble X7)

Terrestrial LiDAR (Light Detection and Ranging) scanning of bridges captures three-dimensional point clouds of the entire bridge structure by scanning from multiple ground and elevated vantage points with sub-millimetre range accuracy (Leica ScanStation P40: 0.8mm accuracy at 80m range; RIEGL VZ-2000i: 5mm accuracy at 500m). AI classification of rendered LiDAR point-cloud images — specifically, colour-coded 2D projections of the 3D point cloud (elevation maps, deviation maps relative to as-built geometry, and displacement maps comparing sequential scans from different inspection cycles) — determines whether the bridge structure shows: vertical sag (downward displacement of the bridge deck midspan, indicating loss of bearing capacity in primary girders); lateral movement (horizontal displacement of piers indicating scour undermining or foundation settlement); out-of-plumb of vertical members (column or pier tilt beyond L/500 tolerance); and differential settlement between adjacent spans (indicating differential bearing settlement or abutment rotation). Bentley OpenBridge Inspector AI and Jacobs SPAN+ AI process rendered LiDAR deviation maps — false-colour 2D projections of 3D point cloud displacement relative to the original design geometry — using convolutional classifiers trained on labelled bridge deformation datasets from state DOT bridge management systems (California DOT BMS, TxDOT BridgeInspect, MDOT BridgeReview).

An adversarial perturbation on a rendered LiDAR deviation map image that reduces the apparent displacement magnitude of a midspan sag anomaly — shifting the colour gradient from yellow (10mm deflection) to green (3mm deflection) using a targeted hue-rotation perturbation on the false-colour scale — causes the deformation AI to classify the bridge span as “within acceptable deflection limits” rather than flagging it for load rating review. Under AASHTO LRFR, a span with a measured live-load deflection exceeding L/800 (where L is the span length — for a 100m span, threshold = 125mm; for a 40m span, threshold = 50mm) triggers an emergency rating evaluation. An adversarial injection that suppresses a 130mm sag reading to 40mm in the rendered deviation map prevents this trigger for a 100m span bridge. The 2022 Fern Hollow Bridge collapse in Pittsburgh (no fatalities — early morning collapse before peak traffic; bridge had been rated “poor” since 2011 but had not received emergency funding) and the 2007 I-35W collapse (primary failure initiated by gusset plate undersizing combined with added deck load — a load-rating assessment issue) both illustrate the consequence when bridge structural condition assessment understates the actual deficiency magnitude.

3. Sonar scour bottom-profile imaging AI (Teledyne BlueView, RESON SeaBat, EdgeTech 2200)

Foundation scour — the erosion of soil and rock from around bridge foundations by flowing water during flood events — is the leading cause of highway bridge collapse in the United States, responsible for approximately 60% of bridge failures (FHWA HEC-18, “Hydraulic Engineering Circular No. 18: Evaluating Scour at Bridges”). FHWA’s scour evaluation programme (“Recording and Coding Guide for the Structure Inventory and Appraisal”, Item 113 — Scour Critical Bridges) classifies bridges as “scour critical” if they are built on foundations that may be vulnerable to scour during a 100-year or 500-year flood event. Scour monitoring uses multibeam sonar (Teledyne BlueView P900-130, RESON SeaBat T20-P, EdgeTech 2200 sub-bottom profiler) to image the riverbed and bridge foundation geometry underwater — generating rendered sonar bottom-profile images that AI classifiers process to detect foundation exposure (bare pile or footing visible below the original mudline), scour hole geometry (the bathymetric depression around a bridge pier created by vortex erosion), and lateral channel migration (change in channel thalweg position toward a bridge pier). Teledyne BlueView’s AI sonar analysis system and multi-beam sonar AI systems deployed by AECOM, Mactec (AECOM predecessor), and HNTB process rendered sonar bathymetric images at 0.5–5cm range resolution into FHWA Item 113 scour condition ratings.

An adversarial perturbation on a rendered sonar bathymetric image that fills in the visualised scour hole depression — raising the apparent mudline elevation around a bridge pier foundation back to the pre-scour bed elevation using a targeted fill perturbation on the depth-coded colour channels — causes the scour AI to classify the bridge as “scour condition satisfactory (Item 113 = 8)” rather than “scour critical (Item 113 = 3)” or “bridge closed due to imminent failure (Item 113 = 0).” A bridge classified as scour critical (Item 113 ≤ 3) under FHWA guidelines must have a Plan of Action (POA) for monitoring and countermeasures — and may require real-time scour monitoring instrumentation (tiltmeters, sonar ranging devices) or emergency closure if a flood event exceeds the calculated critical flow velocity. The 1987 Schoharie Creek Bridge collapse in Montgomery County, New York (10 fatalities; NTSB HAR-88-02) and the 2013 US 35 Interstate Bridge collapse in Oso, Washington (catastrophic failure in flooding) illustrate the consequence envelope of scour critical bridge failures — and adversarial suppression of sonar scour AI detections removes the Item 113 trigger that would drive POA implementation.

4. GPR rebar corrosion and deck delamination AI (GSSI StructureScan, IDS GeoRadar, Mala ProEx)

Ground-penetrating radar (GPR) bridge deck scanning uses electromagnetic pulses in the 1–2.5 GHz frequency range to image the subsurface structure of bridge decks without traffic interruption — detecting rebar corrosion delamination (the subsurface crack at the rebar level that forms as corrosion products expand the rebar diameter and debond the concrete cover from the bar), concrete layer boundaries, and chloride ingress extent through the correlation of GPR reflection amplitude with rebar oxidation state. GSSI StructureScan Mini HR, IDS GeoRadar Stream C, and Mala ProEx GPR systems generate rendered B-scan images — cross-sectional views of the deck subsurface where the horizontal axis is the distance along the GPR antenna travel path and the vertical axis is the two-way travel time (depth equivalent), with pixel intensity encoding GPR amplitude at each depth position — that AI classifiers process to identify the spatial extent of rebar corrosion delamination areas (expressed as percentage of deck area in each 30m × 30m grid cell) and classify overall deck condition per FHWA Item 58 (Deck Condition Rating). GSSI’s AI GPR analysis system and AECOM’s GPR deck assessment AI process rendered B-scan JPEG images from deck-mounted GPR antenna arrays towed behind a vehicle at highway speeds.

An adversarial perturbation on a rendered GPR B-scan image that suppresses the characteristic amplitude anomaly signature of rebar delamination — reducing the bright linear reflection amplitude at the rebar depth level and infilling the shadow zone below the delaminated rebar with ambient clutter amplitude — causes the GPR AI to classify the deck grid cell as “Delamination 0–10% of area (good condition)” rather than “Delamination 40–60% of area (serious condition requiring deck replacement).” Under FHWA NBIS, a bridge deck rated Item 58 = 3 (poor) or below triggers an inspection frequency increase to annual inspection and a capital programming review for deck rehabilitation or replacement. Adversarial suppression of GPR delamination detections prevents these triggers. The financial consequence of missed deck delamination detection is also quantified: a bridge deck that progresses from 20% delamination (rehabilitation possible, cost $250–400/m²) to 60% delamination (full deck replacement required, cost $800–1,200/m²) due to missed detection doubles or triples the lifecycle repair cost — providing an economic incentive alongside the safety consequence for adversarial robustness testing of GPR AI classification systems.

Integration: highway bridge inspection AI scanning with Glyphward pre-scan gate

The Glyphward scan gate for bridge inspection AI belongs at the rendered image ingestion boundary before each AI classification step — before drone crack detection AI processes orthomosaic renders, before LiDAR deformation AI processes point-cloud deviation maps, before sonar scour AI processes bathymetric renders, and before GPR AI processes B-scan images. Threshold 40 for bridge inspection AI contexts reflects structural failure consequence severity and the availability of independent physical inspection for cross-validation.

import asyncio, base64, hashlib, json
from datetime import datetime, timezone
from enum import Enum
from pathlib import Path

import httpx

GLYPHWARD_API_KEY = "YOUR_GLYPHWARD_API_KEY"
GLYPHWARD_SCAN_URL = "https://glyphward.com/v1/scan"

# Bridge inspection AI contexts: threshold 40
# FHWA NBIS 23 CFR 650, AASHTO MBE, AASHTO LRFR.
BRIDGE_AI_THRESHOLD = 40


class BridgeAIContext(Enum):
    DRONE_CRACK        = "drone_crack"        # Drone RGB orthomosaic crack AI
    LIDAR_DEFORMATION  = "lidar_deformation"  # LiDAR point-cloud deviation map AI
    SONAR_SCOUR        = "sonar_scour"        # Sonar bathymetric scour imaging AI
    GPR_DELAMINATION   = "gpr_delamination"   # GPR B-scan rebar corrosion AI


class AdversarialBridgeImageError(Exception):
    """Raised when Glyphward detects adversarial pixel content in a bridge
    inspection AI rendered image above threshold 40.

    Consequence if not raised: crack / scour / delamination suppressed from
    element condition rating → no load restriction or POA trigger → structurally
    deficient bridge remains open at legal posting load → potential collapse.
    Fail-safe: suppress AI classification, route to qualified bridge inspector
    (FHWA-certified per 23 CFR 650.307) for manual field verification.
    """

    def __init__(self, scan_id: str, score: int,
                 context: BridgeAIContext,
                 bridge_id: str, element_id: str | None,
                 flagged_region: dict | None = None) -> None:
        self.scan_id = scan_id
        self.score = score
        self.context = context
        self.bridge_id = bridge_id
        self.element_id = element_id
        self.flagged_region = flagged_region
        super().__init__(
            f"Adversarial bridge image: "
            f"context={context.value} score={score} "
            f"bridge={bridge_id} element={element_id} scan_id={scan_id}"
        )


async def scan_bridge_image(
    image_bytes: bytes,
    context: BridgeAIContext,
    bridge_id: str,
    element_id: str | None,
    state_fips: str,
    is_fracture_critical: bool,
    client: httpx.AsyncClient,
) -> dict:
    """Scan a bridge inspection AI image for adversarial content.

    Fail-safe contract: AdversarialBridgeImageError or httpx error →
    suppress AI element condition rating, route to FHWA-certified bridge
    inspector for manual field verification per 23 CFR 650.307.
    Do not submit NBI condition ratings derived from adversarially
    flagged images without human inspector verification.

    Args:
        image_bytes: Drone orthomosaic, LiDAR deviation map render,
            sonar bathymetric image, or GPR B-scan image bytes.
        context: BridgeAIContext identifying the inspection modality.
        bridge_id: NBI structure number (8-digit FHWA identifier).
        element_id: AASHTO MBE element identifier (e.g., "065" for RC pier).
        state_fips: State FIPS code for the bridge location.
        is_fracture_critical: True if bridge has fracture-critical non-
            redundant members (18-month inspection cycle required).
        client: Shared httpx.AsyncClient for connection reuse.

    Returns:
        Glyphward scan result dict.

    Raises:
        AdversarialBridgeImageError: if score exceeds threshold 40.
        httpx.HTTPStatusError: on Glyphward API error (fail-closed).
    """
    image_hash = hashlib.sha256(image_bytes).hexdigest()
    payload = {
        "image": base64.b64encode(image_bytes).decode(),
        "source": f"bridge:{context.value}:{bridge_id}:{element_id}",
        "metadata": {
            "bridge_id": bridge_id,
            "element_id": element_id,
            "state_fips": state_fips,
            "is_fracture_critical": is_fracture_critical,
            "image_sha256": image_hash,
            "context": context.value,
        },
    }
    resp = await client.post(
        GLYPHWARD_SCAN_URL,
        headers={"Authorization": f"Bearer {GLYPHWARD_API_KEY}"},
        json=payload,
        timeout=4.0,
    )
    resp.raise_for_status()
    result = resp.json()

    await _write_bridge_scan_audit(
        image_hash=image_hash,
        scan_id=result["scan_id"],
        score=result["score"],
        context=context,
        bridge_id=bridge_id,
        element_id=element_id,
        is_fracture_critical=is_fracture_critical,
        flagged=result["score"] > BRIDGE_AI_THRESHOLD,
    )

    if result["score"] > BRIDGE_AI_THRESHOLD:
        raise AdversarialBridgeImageError(
            scan_id=result["scan_id"],
            score=result["score"],
            context=context,
            bridge_id=bridge_id,
            element_id=element_id,
            flagged_region=result.get("flagged_region"),
        )
    return result


async def _write_bridge_scan_audit(
    *, image_hash: str, scan_id: str, score: int,
    context: BridgeAIContext, bridge_id: str,
    element_id: str | None, is_fracture_critical: bool, flagged: bool,
) -> None:
    record = {
        "ts": datetime.now(timezone.utc).isoformat(),
        "scan_id": scan_id,
        "image_sha256": image_hash,
        "context": context.value,
        "score": score,
        "threshold": BRIDGE_AI_THRESHOLD,
        "flagged": flagged,
        "bridge_id": bridge_id,
        "element_id": element_id,
        "is_fracture_critical": is_fracture_critical,
        "regulatory_refs": [
            "FHWA NBIS 23 CFR 650 Subpart C (National Bridge Inspection Standards)",
            "AASHTO Manual for Bridge Element Inspection (MBE), 3rd ed. 2019",
            "AASHTO Manual for Bridge Evaluation (LRFR), 3rd ed. 2018",
            "FHWA HEC-18 (Evaluating Scour at Bridges, 5th ed.)",
            "FHWA Hydraulic Engineering Circular No. 23 (Bridge Scour and Stream Instability)",
            "AASHTO LRFD Bridge Design Specifications (8th ed.)",
            "FHWA Recording and Coding Guide for the Structure Inventory and Appraisal (2021)",
        ],
    }
    audit_path = Path("/var/log/glyphward/bridge_ai_scan_audit.jsonl")
    audit_path.parent.mkdir(parents=True, exist_ok=True)
    with audit_path.open("a") as fh:
        fh.write(json.dumps(record) + "\n")

Deploy scan_bridge_image at each bridge inspection AI rendered-image ingestion boundary: before drone crack AI (threshold 40), before LiDAR deformation AI (threshold 40), before sonar scour AI (threshold 40), and before GPR delamination AI (threshold 40). On AdversarialBridgeImageError: suppress AI element condition rating, route the flagged image and bridge/element identifiers to an FHWA-certified bridge inspector (Level III or IV per 23 CFR 650.307(b)) for manual field verification before submitting NBI data. Get early access

Related questions

Further reading

• Prompt injection in pipeline integrity inspection AI — ILI pig MFL AI and corrosion detection adversarial attacks
• Prompt injection in dam safety monitoring AI — seepage detection AI and spillway inspection adversarial attacks
• Prompt injection in transmission line inspection AI — EPRI SmartLine drone RGB+thermal+UV inspection AI adversarial attacks
• Prompt injection in real estate and property inspection AI — structural defect detection AI adversarial attacks
• Prompt injection scanning API free tier — 10 scans/day, no card required