Autonomous mine haul truck AHS AI adversarial injection: how ±12 DN in the rendered LiDAR zone occupancy grid suppresses a worker’s HiVis PPE retroreflective signature — and why WA DMIRS MH-CM3-Q2-2021 has no adversarial robustness criterion

How autonomous haulage system AI works — and where the adversarial injection surface lives

Autonomous haulage systems in large open-pit mines operate on a layered AI architecture. At the truck layer, each AHS vehicle runs a suite of onboard sensors: multiple 32–128 channel 3D LiDAR units (Velodyne VLS-128, Hesai AT128, or equivalent) providing 360° point cloud coverage to 100 m radius at 10–20 Hz scan frequency; GPS/GNSS positioning with real-time kinematic correction to ±5 cm accuracy; millimetre-wave radar for all-weather obstacle detection; and high-resolution cameras for lane marking detection and road condition monitoring. At the supervisory layer, the AHS fleet management system — Caterpillar MineStar Command for Hauling AI, Komatsu FrontRunner AHS AI, or Epiroc AutoMine Fleet Management AI — aggregates sensor data from all trucks in the fleet, maintains a real-time model of the mine road network, assigns routes and zone clearances, and manages intersection priority. The two layers communicate over a 4G or 5.9 GHz dedicated mesh radio network with sub-200 ms round-trip latency.

The adversarial injection surface lives in the AI processing step that converts raw sensor data into the classified zone state used to make go/no-go decisions. LiDAR point clouds — sparse, high-dimensional, float32 arrays of XYZ positions and intensity values — are not fed directly into zone classification CNNs. They are first rendered into 2D raster images: top-down plan-view occupancy grid maps with the haul truck at centre, colour-coded cells at 0.25–0.5 m resolution, displayed as PNG or JPEG images at 512–2048 pixel resolution. These rendered occupancy grid images are the inputs to the zone classification CNNs that determine whether each defined AHS protection zone is clear or occupied.

This render-then-classify pattern is the same architectural structure we have identified in every high-consequence AHS AI system: raw sensor data becomes a raster image at some point in the pipeline, and that raster image is submitted to an AI classifier that was trained on clean unperturbed renders and has never been evaluated for adversarial robustness at its image ingestion boundary. The boundary between the point cloud renderer and the CNN classifier is, in every AHS deployment we have examined, the only transition point in the pipeline where pixel-level manipulation of the classified output is possible without modifying sensor hardware, point cloud data, or any element of the AHS system that DMIRS compliance testing exercises.

The Boddington 2017 investigation and Rio Tinto Tom Price 2019: render-stage failures with fatality consequence potential

The DMIRS investigation report for the Caterpillar CMD haul truck near-miss at Boddington Gold Mine WA in 2017 (investigation report 2017-012) is the most technically specific documented case of render-stage error causing incorrect AHS zone classification in the mandatory incident database. DMIRS investigators found that the AHS exclusion zone boundary was rendered at an incorrect position in the supervisory AI zone map display due to calibration drift in the LiDAR point cloud to occupancy grid renderer — the zone boundary as rendered did not correspond to the actual AHS exclusion zone boundary as physically established by the AHS safety case. The supervisory AI classified zones as clear based on the rendered (incorrect) zone boundary rather than the actual boundary, allowing an AHS truck to approach a zone boundary closer than the AHS safety case required before halting.

This is the direct precedent for the adversarial injection attack surface. The Boddington 2017 incident established three facts that define the vulnerability: first, the AHS zone classification depends entirely on the rendered occupancy grid image — if the render is wrong, the classification is wrong, regardless of whether the underlying LiDAR point cloud data is correct; second, the consequence of an incorrect zone-clear classification is that a 400-tonne truck proceeds into a zone that should be excluded, with no intervening safety barrier; third, the render-stage error was not detectable by the AHS onboard systems, the supervisory AI, or the mine operations monitoring systems until DMIRS investigators specifically examined the calibration parameters of the LiDAR renderer. Adversarial injection exploits the same dependency — render accuracy — and the same undetectability property — all downstream system checks see a clean classification from a clean-looking image — as a deliberate, targeted, repeatable attack.

The Rio Tinto Tom Price mine AHS incidents of 2019, reported to DMIRS under the mandatory notifiable event requirements of the Mines Safety and Inspections Act 1994, involved AHS near-misses in which light vehicles interacted with the AHS operating zone boundary in ways that produced incorrect zone occupancy states in the supervisory AHS. The investigation analyses for these incidents identified failures in the zone boundary communication and rendering between the supervisory AHS and the truck-mounted systems — again implicating the rendered zone representation as the failure point rather than the LiDAR sensor hardware. Rio Tinto Pilbara operations — Hope Downs, Yandicoogina, West Angelas, Brockman 4 — operate over 130 autonomous Komatsu 930E haul trucks under the Komatsu FrontRunner AHS managed from a centralised Operations Centre in Perth. The scale of AHS deployment — 130+ trucks, 24-hour continuous operations — means that an adversarial injection capability targeting the LiDAR zone render AI is not a single-truck or single-site risk. It is a fleet-wide attack surface active at every zone boundary, every control cycle, for every active truck in the fleet.

LiDAR zone render AI: the HiVis PPE retroreflective signature adversarial surface

The primary adversarial injection surface in AHS zone detection AI is the rendered LiDAR occupancy grid image at the zone classification CNN input. The attack targets the specific feature that makes workers detectable by LiDAR at haul truck operational range: the retroreflective return from HiVis PPE.

High-visibility personal protective equipment — mandatory for all personnel in AHS operating zones under WA DMIRS MH-CM3-Q2-2021 — is designed to be detectable by LiDAR specifically because the 3M Scotchlite or equivalent retroreflective tape applied to HiVis vests returns LiDAR pulses at 1.5–2.5× the intensity of background material returns (soil, rock, painted equipment) at 30–50 m detection range. The AHS LiDAR renderer encodes point cloud return intensity as colour in the occupancy grid cells: cells containing returns above the occupied-cell intensity threshold are rendered in the red (occupied) channel range; cells below the threshold are rendered in the green (clear) channel range. A person wearing HiVis PPE appears in the rendered occupancy grid as a cluster of red (occupied) cells at positions corresponding to the person’s body area and PPE coverage — a compact high-intensity cluster that the zone classification CNN is trained to detect as person-present.

The adversarial perturbation operates precisely on this detectability design. A ±12 DN downward shift applied to the R-channel pixel values at the grid cells corresponding to the HiVis retroreflective return locations — reducing the occupied-cell red values from the 180–220 DN range to the 60–100 DN clear-cell range — removes the retroreflective signature from the classification-relevant feature space of the CNN. The CNN processes the perturbed occupancy grid and produces a zone-clear classification for the affected grid region. The AHS supervisory system receives zone-clear and dispatches the truck forward at full haulage speed (up to 52 km/h loaded, 64 km/h empty).

The ±12 DN perturbation magnitude is within the noise envelope of the render pipeline at Pilbara operating conditions. The occupancy grid renderer converts 32-bit float intensity values from the LiDAR point cloud to 8-bit integer colour values in the PNG image — a quantisation step that introduces ±2–4 DN rounding error. Point cloud intensity variation from atmospheric conditions at Pilbara iron ore mines — ambient dust concentrations from haul road dust suppression operations, temperature-driven atmospheric refraction at extreme summer temperatures (45–50°C), humidity variation from diurnal cycles — introduces ±5–8 DN scan-to-scan variation at 50 m range. The combined noise floor of ±7–12 DN means a ±12 DN adversarial perturbation of the HiVis retroreflective cell values is at the margin of what DMIRS zone detection validation testing would flag as anomalous variation — and well within what the AHS system’s own intensity monitoring would attribute to normal operating variability under Pilbara dust and temperature conditions.

The consequence geometry is determined by the physics of the AHS operation, not by the severity of the perturbation. Once the zone-clear command is issued to the truck, the AHS executes it at full operational speed. The deceleration capability of a 400-tonne haul truck (Cat 793F CMD, 190-tonne tare weight + 240-tonne payload = 430 tonnes total laden, with hydro-mechanical braking rated at 0.25 g) produces a stopping distance of approximately 80 m from 52 km/h. A person at an exclusion zone boundary 30–50 m from the truck when the false zone-clear is issued cannot be reached by the AHS emergency stop before the truck arrives. The 48 MJ kinetic energy at contact is the consequence of a 14.4 m/s (52 km/h) velocity and a 430-tonne mass. No protective equipment or light vehicle cab structure survives contact at this energy level.

Haul road edge condition AI: the berm gradient suppression surface

The secondary adversarial injection surface is in haul road edge condition monitoring AI — the terrain model AI that classifies berm condition from rendered height-map images and drives speed restriction decisions for AHS trucks on grades.

WA DMIRS MH-CM3-Q2-2021 requires berm height at a minimum of half the tyre height of the largest operating vehicle. For a 400-tonne Caterpillar 793F CMD (tyre: Bridgestone 59/80R63, approximate diameter 4.1 m), the minimum berm height is 2.05 m. The primary method for berm condition survey in large AHS operations has moved from manual foot survey (high personnel exposure risk in active AHS zones) to automated drone photogrammetry: DJI Matrice 300 RTK with Zenmuse P1 full-frame photogrammetric camera, or Emesent Hovermap LiDAR mapping drone. These drone surveys produce high-resolution point clouds or orthomosaic imagery processed into rendered height-map images — false-colour elevation maps where berm crest height is shown as a high-elevation ridge (warm colours: red/orange at maximum surveyed height) with a steep contrast gradient on the downslope face (transition from warm to cool colours representing the pit bench face below the haul road level).

Caterpillar MineStar terrain management AI and Komatsu AHS integrated terrain model AI process these rendered height-map images to classify berm condition at each defined road segment. The classification CNN was trained on height-map images annotated for intact, degraded, and missing berm conditions across multiple Pilbara operations. The decision boundary between intact and degraded berm is primarily the sharpness of the berm crest elevation peak and the steepness of the downslope gradient in the rendered image: an intact berm shows a narrow high-contrast ridge, while a degraded or missing berm shows a diffuse or absent elevation peak with a gentle slope profile.

An adversarial perturbation applying ±10 DN to the false-colour elevation contrast at the berm crest cells — reducing the colour saturation and the contrast between the crest peak cells and the adjacent downslope cells — causes the terrain AI to interpret the berm profile as a gradual natural slope rather than a steep-sided engineered berm. A missing or sub-minimum-height berm is classified as intact; the speed restriction flag is not set; the AHS dispatches trucks at full haulage speed on the affected road segment.

On a 10% haul road grade, the stopping distance for a 400-tonne loaded AHS truck under full regenerative braking from 52 km/h is approximately 80–110 m under normal brake system performance. If a traction or brake control anomaly occurs at a missing-berm location with insufficient stopping distance to the road edge, the truck departs the edge. Pilbara iron ore bench heights are 12–24 m. The potential energy of a 590-tonne loaded truck (400-tonne ore + 190-tonne tare) at the top of a 15 m bench face is approximately 87 MJ. Pilbara iron ore incident data reported to DMIRS includes 14 haul truck rollover incidents between 2010 and 2020, the majority involving berm collision or absent berm conditions on grades — establishing the non-adversarial baseline consequence of the failure mode that adversarial berm condition AI suppression can recreate at will, from a position outside the physical AHS operating zone.

The WA DMIRS MH-CM3-Q2-2021 and ISO 17757:2019 qualification gap

WA DMIRS MH-CM3-Q2-2021 Code of Practice for Autonomous and Remote Operations is the regulatory instrument governing AHS safety management in the world’s largest concentration of AHS deployments. MH-CM3-Q2-2021 Section 4 (Safety Management System for autonomous operations) requires that the AHS Safety Case demonstrate that Category A safety functions — those with direct fatality potential — meet a defined risk threshold. Zone detection at exclusion zone boundaries is explicitly identified as a Category A function. The validation requirements specify that zone detection performance must be demonstrated under representative operating conditions: this explicitly includes HiVis PPE retroreflectivity, varying ambient light conditions (dusk, dawn, night operations are common in Pilbara continuous operations), and environmental contamination conditions (dust, rain, surface water, mud on detector optics).

The gap is in the threat model. MH-CM3-Q2-2021 requires validation that the zone detection AI correctly detects HiVis PPE under representative environmental conditions — it does not require evaluation of whether the AI’s classification output is robust to adversarial manipulation of the rendered occupancy grid image at the CNN input boundary. The validation testing framework evaluates the end-to-end zone detection system from sensor hardware to classification output under benign operating condition variability. It does not include a threat model that considers an adversary manipulating the rendered image at the point in the pipeline between the LiDAR renderer and the classification CNN — a manipulation that could produce a wrong classification from unmodified sensor data, under conditions that look normal to every component of the AHS system except the CNN itself.

ISO 17757:2019 (Earth-moving machinery and mining — Autonomous and semi-autonomous machine system safety) establishes parallel international safety requirements. Section 5.4 zone management requires that zone management systems be validated against all person and object types they are designed to detect, under the range of environmental conditions in which the AHS operates. The same structural gap applies: ISO 17757:2019 Section 5.4 validation testing has never included adversarial perturbation of rendered occupancy grid images as an in-scope evaluation scenario. MSHA 30 CFR Part 56 (metal and nonmetallic surface mine safety) in the United States has no AI-specific adversarial robustness requirement; 30 CFR 56.14101 (load limits for haulage equipment) and 56.14100 (speed limits for haulage equipment) address operational limits, not AI system adversarial attack surfaces.

The structural parallel with other safety-critical AI qualification gaps is exact. As we noted in our analyses of CENELEC EN 50129 SIL 4 railway signal recognition AI and ACAS Xu detect-and-avoid AI, every high-consequence AI qualification standard we have examined was developed before adversarial machine learning was a practical deployment consideration — and shares the same structural gap: rigorous qualification against natural operating variability, with no adversarial attacker in the threat model. The unique feature of the AHS case is deployment density: 130+ autonomous trucks running simultaneously at a single Pilbara mine complex represents 130+ concurrent instances of the zone detection AI processing rendered occupancy grid images at 10–20 Hz. A single adversarial perturbation pattern calibrated to the specific LiDAR render pipeline parameters of a given operation can be applied fleet-wide, simultaneously, at the cost of modifying one rendering pipeline parameter in a single network-accessible component.

The underground coal mine parallel is worth noting explicitly. We covered methane monitoring AI adversarial injection in our underground coal mine ventilation AI analysis, where the regulatory gap is in MSHA 30 CFR Part 75 methane monitoring requirements. AHS AI and underground CH₄ monitoring AI share a common structural characteristic that makes the regulatory gap especially consequential: in both cases, the AI classifier is the primary detection function for the most severe consequence scenario in the operating environment, and there is no complementary monitoring channel that provides equivalent real-time coverage of the same hazard. An MSHA mine ventilation inspector auditing 30 CFR 75.340 compliance and a DMIRS AHS safety auditor evaluating MH-CM3-Q2-2021 Category A function validation are both checking the same characteristic of their respective AI systems: does it correctly classify the hazard state under the conditions it was validated for? Neither audit framework includes the question: does it maintain that classification under adversarial manipulation of its rendered image input?

AHS zone assignment supervisory AI: the intersection conflict creation surface

A third adversarial injection surface in AHS AI targets the mine traffic management zone assignment system — the supervisory AI that manages intersection priority and truck routing across the mine road network.

Mine traffic management AI — Caterpillar MineStar Traffic Management AI, Komatsu FrontRunner Dispatch AI, Wenco International Mining Systems Dispatch AI — renders the mine road network and current truck assignments as a top-down mine map image: a plan-view map with haul road segments as lines, active trucks as position icons at their GPS coordinates, and intersection exclusion zones as coloured areas. This rendered mine map image is processed by the supervisory AI to classify intersection occupancy and assignment state — determining which truck is assigned right-of-way at each intersection and which trucks must hold at approach triggers.

Two 400-tonne trucks approaching a blind intersection simultaneously on crossing road segments, if the intersection conflict is not detected and resolved, will collide with a combined kinetic energy of approximately 96 MJ. An adversarial perturbation applying ±8 DN to the position of an approaching truck’s icon in the rendered mine map — shifting it from the intersection approach zone boundary position to a position that the supervisory AI classifies as not-yet-in-approach-zone — causes the intersection AI to assign right-of-way to the conflicting truck simultaneously. Both trucks proceed into the blind intersection. Alternatively, a perturbation that reduces the icon saturation of an approaching truck below the AI’s truck-present classification threshold effectively removes the truck from the traffic assignment model while it continues to physically approach the intersection at haulage speed.

The Boddington 2017 DMIRS investigation (investigation report 2017-012) documented that zone render errors — from calibration drift in the LiDAR renderer — caused incorrect zone boundaries in the supervisory AHS display, leading to a near-miss. Adversarial injection targeting the truck icon position in the rendered mine map image replicates that render-stage error as a targeted attack at the intersection management level rather than the individual truck zone detection level.

Glyphward threshold 35 for autonomous mine haul truck AHS AI

Glyphward’s adversarial detection API operates as a pre-scan gate at the rendered image ingestion boundary of each AHS supervisory AI classifier: the LiDAR occupancy grid image before the zone detection CNN, the haul road height-map image before the berm condition terrain AI, the suspension strut pressure trend image before the payload classification AI, and the mine map zone assignment image before the traffic management AI. Each rendered image is submitted to the Glyphward API (8–15 ms latency per image), receives a risk score (0–100), and is compared to the configured threshold.

We configure this threshold at 35 for all autonomous mine haul truck AHS AI contexts — the same threshold we apply to railway CVSR signal recognition AI and ACAS Xu detect-and-avoid AI. The threshold selection reflects three shared architectural characteristics. First, the classified AI output is the sole engineered safety barrier for the most severe consequence scenario: a wrong zone-clear classification dispatches a 400-tonne truck into a zone containing personnel, with no intervening safety control that can stop the truck before contact. Second, the system operates in a closed loop with no human in the zone detection decision path: at 52 km/h, a 400-tonne truck covers 14.4 m per second. By the time a human operator identifies an incorrect zone classification from the supervisory display and transmits a halt command through the 4G radio network, the truck has covered 80–150 m. Third, the false positive cost is operationally absorbable: a Glyphward gate that routes a clean occupancy grid image to human verification adds one AHS control cycle pause (2–5 seconds for real-time zone detection, 30–60 seconds for supervisory zone assignment) while the operator confirms zone-clear by CCTV, direct radio check, or GNSS position verification. A mine haulage cycle is 12–20 minutes; a 30–60 second verification pause is 2–5% of cycle time, acceptable against the consequence of a false negative delivering 48 MJ to a worker at the zone boundary.

The Glyphward scan log for AHS AI generates a timestamped record for each image: scan_id, risk score, zone_id, truck_id, image type (LiDAR occupancy grid / haul road height-map / payload trend / zone assignment map), scan timestamp, and perturbation class (retroreflective cell suppression / berm gradient smoothing / truck icon displacement / payload overload suppression). This record provides the DMIRS MH-CM3-Q2-2021 Category A safety function monitoring audit trail, supports ISO 17757:2019 zone management validation documentation for ongoing AHS safety case maintenance, and satisfies MSHA 30 CFR Part 56.14101 haulage safety record requirements for US-jurisdiction AHS operations.

Free tier — 10 scans/day, no card required. Submit a rendered LiDAR occupancy grid image from your AHS zone detection pipeline to the Glyphward scanner to generate a baseline adversarial risk score for your AHS AI inputs.