Scope boundary

Owns application-layer substitutability rules for fallback models and providers, ensuring required moderation, capability, approval, logging, retention, and compliance properties are preserved. It does not control network reachability.

AID-I-002.002: AID-H-033.002 decides whether an alternate reachable model or provider preserves the original safety envelope; AID-I-002.002 enforces which external destinations can be reached. Network approval does not make two providers policy-equivalent.

AID-H-033.005: AID-H-033.002 determines which fallback substitutions are eligible; AID-H-033.005 resolves the final route and blocks dispatch unless it matches the signed contract. Eligibility policy and last-mile enforcement are separately evidenced.

Implementation guidance

• AID-H-033.002-G001: Define an explicit failover eligibility matrix, evaluate candidates through a deterministic route-profile checker, and fail closed if no fallback preserves the original route's safety and compliance envelope.

AID-H-033.002-G001

Concept:

Fallback is not a generic availability feature. Each primary route needs an explicit policy bundle that declares which fallback routes are eligible and which moderation, tool, logging, retention, and approval constraints must still hold. The gateway should emit one structured failover decision and block silent substitution when no candidate preserves that envelope.

Step 1: Define one failover policy bundle

{
  "routes": {
    "gpt-4.1-primary": {
      "eligible_fallbacks": ["gpt-4.1-backup-us", "gpt-4.1-backup-eu"],
      "required_constraints": {
        "moderation_profile": "strict",
        "tool_profile": "no_direct_shell",
        "log_profile": "forensic",
        "retention_class": "customer-default"
      },
      "override_mode": "explicit_operator_only"
    }
  },
  "route_profiles": {
    "gpt-4.1-backup-us": {
      "moderation_profile": "strict",
      "tool_profile": "no_direct_shell",
      "log_profile": "forensic",
      "retention_class": "customer-default"
    }
  }
}

Step 2: Evaluate candidates through a deterministic checker

# File: routing/failover_policy.py
from __future__ import annotations

import json
from dataclasses import dataclass
from pathlib import Path
from typing import Literal


BUNDLE = json.loads(Path("routing/failover_bundle.json").read_text(encoding="utf-8"))


@dataclass(frozen=True)
class FailoverDecision:
    decision: Literal["allow", "deny", "require_override"]
    chosen_route: str | None
    reason: str


def evaluate_failover(primary_route: str, candidate_routes: list[str]) -> FailoverDecision:
    policy = BUNDLE["routes"][primary_route]
    required = policy["required_constraints"]
    profiles = BUNDLE["route_profiles"]

    for candidate in candidate_routes:
        if candidate not in policy["eligible_fallbacks"]:
            continue
        profile = profiles.get(candidate, {})
        if all(profile.get(key) == value for key, value in required.items()):
            return FailoverDecision(
                decision="allow",
                chosen_route=candidate,
                reason="matched_policy_preserving_fallback",
            )

    if policy.get("override_mode") == "explicit_operator_only":
        return FailoverDecision(
            decision="require_override",
            chosen_route=None,
            reason="no_policy_preserving_fallback",
        )

    return FailoverDecision(
        decision="deny",
        chosen_route=None,
        reason="no_policy_preserving_fallback",
    )

Step 3: Enforce and log at the gateway

# File: routing/gateway_failover.py
from __future__ import annotations

import json
import logging

from routing.failover_policy import evaluate_failover

audit_logger = logging.getLogger("failover")


def select_route(primary_route: str, candidate_routes: list[str]) -> str:
    decision = evaluate_failover(primary_route, candidate_routes)
    audit_logger.warning(
        json.dumps(
            {
                "event_type": "failover_decision",
                "primary_route": primary_route,
                "candidate_routes": candidate_routes,
                "decision": decision.decision,
                "chosen_route": decision.chosen_route,
                "reason": decision.reason,
            }
        )
    )
    if decision.decision != "allow" or not decision.chosen_route:
        raise PermissionError(decision.reason)
    return decision.chosen_route

Operational notes

• Evaluate failover before the request leaves the router, not after a best-effort provider retry has already changed the safety envelope.
• Keep route profiles versioned with the same release controls as your routing bundle so safety-preserving equivalence is reviewable.
• If no preserving fallback exists, queue, deny, or route into an explicit override path rather than silently substituting a weaker destination.

Action: Make every failover decision pass through a deterministic route-profile checker. Availability may degrade, but the gateway must not silently downgrade moderation, tool, logging, or retention guarantees.

Keep the action safely bounded

Reconcile the complete population of every fallback candidate and every retry or failover attempt for each eligible request; reject expired, revoked, or stale policy and asset bindings under the policy-defined freshness window.

Verify safely

A separately credentialed verifier independently replays candidate-profile comparison and effective route with missing field, weaker moderation, expanded tools, changed retention/region, stale health, chained fallback, and no-candidate fixtures, reads back effective state from the enforcement system, exercises positive, negative, boundary, stale-evidence, and dependency-failure fixtures, and reconciles every eligible item.

Open-source tools

• Open Policy Agent (OPA)
• Cedar
• Envoy

Commercial / hosted tools

• Azure API Management
• Cloudflare AI Gateway
• Kong Konnect AI Gateway

Threat mappings

MITRE ATLAS

• N/A

MAESTRO

• Integration Risks (L7)

OWASP LLM Top 10 2026

• LLM02:2026 Sensitive Information Disclosure (failover equivalence preserves downstream confidentiality controls but does not itself redact or authorize data)
• LLM03:2026 Excessive Agency (failover equivalence preserves downstream tool and approval constraints but does not authorize individual actions)

OWASP ML Top 10 2023

• N/A

OWASP Top 10 for Agentic Applications 2026

• ASI08:2026 Cascading Failures

NIST Adversarial Machine Learning 2025

• N/A

Cisco Integrated AI Security and Safety Framework

• AISubtech-2.1.2 Capability and Permission Overreach (rejects fallback routes that expand the approved tool-capability profile)
• AITech-2.1 Excessive Agency (rejects fallback routes that expand the approved tool-capability profile)
• AITech-8.2 Data Exfiltration / Exposure (rejects fallback routes with weaker approved data-handling profiles)
• AITech-8.3 Information Disclosure (rejects fallback routes with weaker approved data-handling profiles)

Google Secure AI Framework 2.0 - Risks

• IIC: Insecure Integrated Component
• SDD: Sensitive Data Disclosure

Databricks AI Security Framework 3.0

• Operations 11.1: Lack of MLOps - repeatable enforced standards