AID-D-004.006: Effective Route & Failover Drift Detection
Pillars: app, infra
Lifecycle: operation
Scope boundary
Owns independent runtime route-drift/visibility detection, complete measured population reconciliation, and verified signal/alert effectiveness. It compares the authorized decision for each attempt to actual observed routing; it does not authorize/block routes or earn protected provenance custody credit. Window observations remain distinct from assurance of the configured detector.
AID-H-033.001: AID-D-004.006 consumes the canonical v3 decision projection, re-derived from the separately verified signed v2 ledger export and independently enumerated ingress; AID-H-033.001 owns immutable complete history, custody, and durable intent acknowledgment. Replaying the ledger does not earn duplicate provenance credit or establish that the detector produces and delivers required signals.
AID-H-033.005: AID-D-004.006 detects authorized-decision versus actual-route drift and missing visibility; AID-H-033.005 owns local final-route eligibility and blocking. Approved requested-versus-effective substitutions are benign when the actual route matches its authorized decision. A confirmed drift observation can invalidate enforcement coverage while correctly demonstrating detector behavior.
AID-D-004.003: AID-D-004.006 owns effective route and failover drift; AID-D-004.003 owns generic runtime configuration and policy drift. Route resolution includes dynamic aliases, retries, and provider readback that a static configuration comparison may not reveal.
Implementation guidance
- AID-D-004.006-G001: Collect independent ingress, per-attempt authorized decisions, actual gateway routing and provider/network readback; close and bind exact independently authenticated source populations without silent sampling.
- AID-D-004.006-G002: Verify signed per-attempt route observations, emit drift or visibility signals, and prove configured detection and delivered-alert effects with adversarial and benign cases.
AID-D-004.006-G001
Capture independently observed routing after resolution
Observe the last actual gateway egress and independent provider or network readback, after aliases, defaults, account translation, retry, and fallback. Pair by (invocation_id, attempt_id), not one row per invocation. Actual fields must come from the observed transport/provider identity; requested model and planned route are context. Approved alias expansion or approved fallback is not itself a finding. Compare the actual route against that attempt's signed authorized decision and policy digest. Independently enumerate ingress and downstream sends so a gateway can neither hide an unrecorded request nor declare its own population complete.
Use v3 window receipts and decision projections over v2 policy and ledger schemas
Keep the common module from H-033.001 installed as a root-controlled shared dependency. Its verify_window captures stable regular files, verifies their exact captured bytes with actual Cosign, and evaluates those same bytes. Supply policy.json, inventory.json, receipt.json, ingress.jsonl, decisions.jsonl, gateway.jsonl, provider.jsonl, and ledger.json, each with a same-basename .bundle.json and a separately authorized <label>.pub under the key root. Do not replace cryptographic verification with a signature_verified boolean or reread mutable live files after verification.
The signed observation policy is aidefend.route-observation-policy.v2: exact application/version/gateway subject; exact expected canonical recorder-policy SHA-256; approved routing-policy version-to-file-digest map; environment-defined freshness, future skew, and minimum invocation count. The signed aidefend.route-architecture-inventory.v2 binds that same subject and actual gateway_present. The signed aidefend.route-window-receipt.v3 binds subject, window ID/start/end, exact policy/inventory hashes, independently enumerated expected invocation IDs, closed status, and each source's exact digest, row count, authenticated collection completeness, and first/next native cursor. The window closer must obtain these cursors and closure from the actual collector/source protocol; counting its own produced rows or setting booleans cannot prove completeness. A missing source, native readback, or applicable observation stays a visibility gap. A signed absence of the gateway architecture can establish inapplicability; collector outage cannot.
Ingress rows use aidefend.trusted-route-ingress.v2 and invocation/requested-model/time. Decision rows come from the ledger's aidefend.route-decision.v3 projection and bind each invocation/attempt/predecessor, exact decision digest, source application/version/gateway and canonical recorder-policy digest, routing-policy file digest, eligibility, eight fields, decision time, and durable_intent_sha256. That field is the exact committed INTENT event digest, or null when no such event exists; never fill it from an eligibility result, gateway claim, record count, or provider observation. Call project_decisions(export, window_id, expected_subject=policy["subject"], expected_recorder_policy_sha256=policy["recorder_policy_sha256"], invocation_ids=receipt["expected_invocation_ids"]) with the independently verified observation policy. A witnessed dispatch with null intent is dispatched_without_durable_intent even when all route fields match. These source identity bindings must exactly match the signed observation policy rather than being re-labeled by the new window receipt. Independent gateway rows bind window/invocation/attempt, outcome, actual eight fields (absent for non-dispatch/unknown), provider request ID, observation time, and decision digest. Provider/network readback rows bind window/invocation/attempt, provider request ID, observation time, and actual provider/account/model/deployment/endpoint/region. Unsupported provider visibility produces a gap; do not fill actual values from the planned decision. The schema and field checks are executable in the shared module, not an assessment engine envelope.
Close, sign, and retain source snapshots
The separately credentialed source collectors export their native bounded, closed populations and sign exact bytes. The route-window closer binds the independent ingress ID set, exact five source hashes/counts/cursors, including the independently attested ledger, application/version/gateway, policy/inventory, and closure in its signed receipt. An offline JSONL export is accepted only when the collector proves the native source boundary and retains the authentication/readback; the local filesystem adapter cannot invent closure. Include denied attempts and observed non-dispatch, multiple fallback/retry attempts with predecessor IDs, UNKNOWN states, provider receipts, and hidden/unbound provider sends. Preserve source-specific acquisition identities and versions. A gateway's self-reported route metadata is one input and never replaces independent actual observation.
Verify source acquisition against the deployed architecture
Exercise source outage, readback blind spots, omitted or extra IDs, truncated cursors, stale or future windows, substituted application/gateway/policy, duplicates, and unsupported fields. Verify that collection produces explicit visibility gaps or processing faults rather than apparently clean observations. Telemetry products listed here provide acquisition, search, or alert transport only; custom final-attempt bindings, source-native closure, signatures, and independent readback are required. An absent gateway is inapplicable; inaccessible collectors for an applicable gateway remain unobserved.
Supply all eight independently authenticated window inputs, including ledger.json and its separately protected recorder-attestation signature. The v3 receipt covers ledger event count/cursors/completeness and exact digest. verify_window rejects reuse of the ledger public key for decisions, receipt, gateway or provider signing. It verifies the complete ledger chain, calls project_decisions(..., invocation_ids=receipt["expected_invocation_ids"]) and compares the canonical projection to decisions.jsonl. A disagreement produces decision_projection_differs_from_protected_ledger, including a fabricated intent hash or a decision hash masquerading as an intent hash. The projection remains v3; ledger events/export remain v2. Missing independent ledger evidence cannot support a clean window.
AID-D-004.006-G002
Run an independent detector over one signed measurement window
Install this runnable wrapper alongside the common module from H-033.001. It verifies signed policy, architecture, receipt, and all five source snapshots, including the independently attested ledger before comparison. It retains confirmed field/dispatch drift even if another source has a coverage gap. A processing fault produces PROCESSING_ERROR; unknown or incomplete observations produce VISIBILITY_INCOMPLETE; complete observations without a discrepancy produce NO_DRIFT_OBSERVED. These are local window observations, not a control assurance score.
# File: detection/verify_route_window.py
from __future__ import annotations
import argparse
from datetime import datetime, timezone
import os
from pathlib import Path
from gateway.route_ledger import canonical, timestamp, verify_window
parser = argparse.ArgumentParser()
parser.add_argument("--input-root", type=Path, required=True)
parser.add_argument("--key-root", type=Path, required=True)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("--verify-timeout-seconds", type=float, required=True)
parser.add_argument("--snapshot-max-bytes", type=int, required=True)
parser.add_argument("--evaluation-time")
args = parser.parse_args()
clock = datetime.now(timezone.utc)
try:
if args.evaluation_time:
clock = timestamp(args.evaluation_time)
result = verify_window(args.input_root, args.key_root,
args.verify_timeout_seconds, args.snapshot_max_bytes, clock)
except Exception as error:
result = {"window_result": "PROCESSING_ERROR", "error_class": type(error).__name__,
"evaluated_at": clock.isoformat(), "findings": [], "coverage_gaps": []}
raw = canonical(result) + b"\n"
args.output.parent.mkdir(parents=True, exist_ok=True)
# Each result is a new artifact; never overwrite a retained measurement.
descriptor = os.open(args.output, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(descriptor, "wb") as output:
output.write(raw); output.flush(); os.fsync(output.fileno())
if args.output.read_bytes() != raw:
raise OSError("result readback differs")
# CLI success means a new result was persisted and read back, never a clean window.
# Detection/alert classification MUST inspect result["window_result"].
raise SystemExit(0)
Execute, sign, and independently replay the exact result
set -euo pipefail
test ! -e "$AIDEFEND_NEW_ROUTE_RESULT_PATH"
python3 -m detection.verify_route_window \
--input-root "$AIDEFEND_ROUTE_WINDOW_ROOT" \
--key-root /etc/aidefend/keys/route-window \
--output "$AIDEFEND_NEW_ROUTE_RESULT_PATH" \
--verify-timeout-seconds "$AIDEFEND_COSIGN_VERIFY_TIMEOUT_SECONDS" \
--snapshot-max-bytes "$AIDEFEND_ROUTE_SNAPSHOT_MAX_BYTES"
# A zero CLI exit proves a new result was written/read back; inspect window_result.
# On write/readback failure set-e stops before signing any prior result.
cosign sign-blob --yes --key env://ROUTE_DETECTOR_SIGNING_KEY \
--bundle "$AIDEFEND_NEW_ROUTE_RESULT_PATH.bundle.json" "$AIDEFEND_NEW_ROUTE_RESULT_PATH"
cosign verify-blob --key /etc/aidefend/keys/route-detector.pub \
--bundle "$AIDEFEND_NEW_ROUTE_RESULT_PATH.bundle.json" "$AIDEFEND_NEW_ROUTE_RESULT_PATH"
A successful CLI exit establishes only that a new result was written and read back. The detector and alert destination classify its authenticated window_result; drift, gaps, and processing faults are retained and signed too. Write/readback failure stops before signing. A separate identity reads back all exact signed inputs/result and keys, replays with --evaluation-time equal to the retained result, and compares canonical bytes. Preserve input hashes, source-native cursors/closure, collector and detector identities/versions, policy, tested environment/profile, and authenticated readback. Missing evidence does not become a clean window.
Verify detection and alert effectiveness independently
In a versioned test profile exercise wrong provider/account/model/deployment/endpoint/region/route/fallback, decision substitution, header override, hidden or denied provider dispatch, alias drift, unauthorized fallback, wrong predecessor, duplicate IDs, unknown crash outcome, source gaps, and stale/substituted signed inputs. Confirm the configured detector emits the correct finding or visibility signal for each applicable case and that the production alert destination receives the exact signal within the environment-defined deadline. Include benign approved aliases/fallbacks and normal transport failure to measure false alerts. Read back alert rule, software/configuration, evidence binding, deduplication, delivery receipt, and retained incident signal under an independent identity. A detector may be effective while finding a route violation; a clean window does not prove the detector is installed or working. H-033.005 owns blocking/route correction and enforcement credit; H-033.001 owns immutable complete route provenance and durability credit.
Supply all eight independently authenticated window inputs, including ledger.json and its separately protected recorder-attestation signature. The v3 receipt covers ledger event count/cursors/completeness and exact digest. verify_window rejects reuse of the ledger public key for decisions, receipt, gateway or provider signing. It verifies the complete ledger chain, calls project_decisions(..., invocation_ids=receipt["expected_invocation_ids"]) and compares the canonical projection to decisions.jsonl. A disagreement produces decision_projection_differs_from_protected_ledger, including a fabricated intent hash or a decision hash masquerading as an intent hash. The projection remains v3; ledger events/export remain v2. Missing independent ledger evidence cannot support a clean window.
The window evaluator also checks attested OBSERVED DISPATCHED and FINAL VIOLATION events for the independently enumerated invocation IDs. A dispatch missing from the gateway snapshot, conflicting observed route/provider identity, or retained violation remains a finding even when the decision projection matches. Events for other invocation populations do not taint this window; a later witness for this population still invalidates a clean claim.
Snapshot capacity: this v2 ledger exporter and verifier require the complete protected segment from sequence 1. Paging bounds each frame, not the total snapshot or verification cost. An export exceeding the signed snapshot ceiling produces a processing/capacity fault; never truncate history, raise the ceiling without reviewing resources, or treat that fault as a clean window. Automatic rotation and cross-segment checkpoint verification are not implemented here. Use an externally qualified segmented recorder/verifier with signed continuity anchors for sustained high-volume service, or bound and close this reference segment within the deployment profile. Keep every unresolved invocation and outbox record recoverable across a segment transition.
Reconcile the latest protected-ledger observation for each attempt and latest FINAL for each invocation in the independently enumerated window population. A ledger NOT_DISPATCHED conflicting with an actual gateway dispatch is a finding. Missing ledger observation, latest UNKNOWN, or latest UNRESOLVED FINAL is a visibility gap even when gateway/provider snapshots report dispatch. A later independently retained resolution replaces an earlier UNKNOWN or UNRESOLVED checkpoint; historical dispatched evidence and violations remain findings. Complete agreement is required for a clean local window observation, and confirmed findings survive other visibility gaps.
Open-source tools
- OpenTelemetry (independent source acquisition and signal export)
- Envoy (actual gateway upstream observation)
- OpenSearch (route drift search and alerting)
- Cosign (exact signed snapshot verification)
Commercial / hosted tools
- Cloudflare AI Gateway (route telemetry input; independent provider/network readback required)
- Prisma AIRS AI Gateway (formerly Portkey; route telemetry input; independent provider/network readback required)
- Splunk Enterprise Security (signal correlation and alert delivery)
- Microsoft Sentinel (signal correlation and alert delivery)
- Google Security Operations (signal correlation and alert delivery)
Threat mappings
MITRE ATLAS
- AML.T0010 AI Supply Chain Compromise (detects an unapproved effective provider, model, endpoint, or fallback)
MAESTRO
- Integration Risks (L7) (detects an unapproved effective provider, model, endpoint, or fallback)
- Data Exfiltration (L2) (detects sensitive requests routed to an unapproved endpoint)
- Supply Chain Attacks (Cross-Layer) (detects an unapproved provider or model fallback)
OWASP LLM Top 10 2026
- LLM02:2026 Sensitive Information Disclosure (route reconciliation detects sensitive requests sent to an unapproved endpoint)
- LLM04:2026 Supply Chain (route reconciliation detects unapproved provider or model fallback)
OWASP ML Top 10 2023
- ML06:2023 AI Supply Chain Attacks (detects an unapproved provider or model fallback)
OWASP Top 10 for Agentic Applications 2026
- ASI04:2026 Agentic Supply Chain Vulnerabilities (detects an unapproved provider or model fallback)
- ASI08:2026 Cascading Failures (detects retry or fallback routes that diverge from the approved decision)
NIST Adversarial Machine Learning 2025
- N/A
Cisco Integrated AI Security and Safety Framework
- AITech-8.2 Data Exfiltration / Exposure (sensitive traffic routed to an unapproved actual model-provider endpoint)
Google Secure AI Framework 2.0 - Risks
- IIC: Insecure Integrated Component (detects an unapproved effective provider or endpoint)
- SDD: Sensitive Data Disclosure (detects sensitive requests routed to an unapproved endpoint)
Databricks AI Security Framework 3.0
- Model Serving - Inference requests 9.10: Accidental exposure of unauthorized data to models (detects sensitive requests routed to an unapproved model or provider)
- Operations 11.1: Lack of MLOps - repeatable enforced standards (detects authorized-decision versus actual-route drift)