AID-M-009.004: Runtime Trust-State Demotion & Autonomy Narrowing
Key question: what verified trust-state change should downstream enforcement consume when behavior turns risky?
Pillars: app, infra
Lifecycle: operation, response
Scope boundary
Owns the trust-state model, transition rules, signed and freshness-bounded state publication, monotonic transition history, and restoration criteria for Normal, Elevated, Degraded, Restricted, or Quarantined agent states. It does not directly force approval, safe mode, or shutdown.
AID-H-018: AID-M-009.004 publishes the verified current trust state and transition evidence; AID-H-018 consumes that state when deciding whether and how the exact action may proceed. A Restricted state is an enforcement input, not the enforcement action itself.
AID-I-003: AID-M-009.004 defines and publishes trust-state demotion or restoration; AID-I-003 executes quarantine, safe-mode downgrade, and interaction throttling. A signed Quarantined state does not prove that containment was applied, and containment should consume an authoritative state transition rather than invent governance facts.
Implementation guidance
- AID-M-009.004-G001: Integrate runtime trust-state transitions with monitoring signals and auditable demotion rules.
- AID-M-009.004-G002: Record every trust-state transition and enforce clean-window evidence before restoring broader autonomy.
AID-M-009.004-G001
Purpose and boundary
This step defines the signed trust-state decision contract that consumes authenticated monitoring signals and selects a narrower autonomy profile. Detect controls own signal generation; Harden or Isolate controls own effective permission enforcement. This module owns deterministic state selection, release binding, and evidence.
Define the transition-policy interface
# File: agent_trust/schemas/trust-state-policy.schema.yaml
schema_version: aidefend.agent-trust-state-policy-schema.v1
policy_required_fields:
- policy_id
- policy_version
- policy_digest
- ordered_states
- signal_catalog
- demotion_rules
- state_to_control_profile_digest
- conflict_resolution
- freshness_policy
signal_required_fields:
- signal_id
- signal_type
- agent_id
- agent_release_digest
- session_id
- observed_at
- source_control_id
- source_receipt_digest
- source_identity
- signature
decision_required_fields:
- decision_id
- agent_id
- agent_release_digest
- session_id
- from_state
- to_state
- policy_digest
- evaluated_signal_ids
- rejected_signal_ids
- target_control_profile_digest
- decided_at
- result
result_states:
- PASS
- FAIL
- INSUFFICIENT_DATA
- NOT_APPLICABLE
- ERRORRules may demote only from independently authenticated, fresh signals that bind the same agent release and session. Missing catalog entries, conflicting rules, stale signals, or an unavailable effective-capability reader fail closed. The decision must not claim that downstream permission narrowing succeeded.
Action: Sign the decision and pass only its target control-profile digest to the canonical enforcement control. Independently read back effective capabilities and reference that separate receipt as a dependency, not as a self-declared field.
AID-M-009.004-G002
Purpose and boundary
This step records every trust-state transition and determines whether a broader state is eligible for consideration. A component-generated HMAC, approval ID string, alert count, or hard-coded clean window is not sufficient evidence.
Define transition and restoration-evidence contracts
# File: agent_trust/schemas/transition-evidence.schema.yaml
schema_version: aidefend.agent-trust-transition-evidence-schema.v1
transition_event_required_fields:
- transition_id
- decision_receipt_digest
- agent_id
- agent_release_digest
- session_id
- from_state
- to_state
- target_control_profile_digest
- effective_capability_readback_digest
- occurred_at
- external_event_store_receipt
- signer_identity
- detached_signature
restoration_policy_required_fields:
- policy_id
- policy_version
- policy_digest
- state_pair
- clean_window_definition
- required_signal_types
- disqualifying_signal_rules
- required_approver_roles
- evidence_freshness
restoration_evidence_required_fields:
- transition_id
- observation_window_start
- observation_window_end
- expected_partition_count
- observed_partition_count
- signal_population_digest
- disqualifying_signal_ids
- missing_partition_ids
- approval_receipt_digests
- approver_subjects
- effective_capability_readback_digest
- resultWrite transition events under a signer and append-only store identity that the agent and decision component cannot use. Resolve the clean-window algorithm, duration, signal population, and approver roles only from signed policy. Reconstruct the exact observation population from source partitions; zero reported alerts is not proof of clean operation when a partition or signal source is missing.
Action: Emit a signed restoration-eligibility receipt only after independent event reconciliation, approver identity and role verification, and effective-capability readback. Eligibility is input to a separate promotion decision and is never itself permission restoration.
Open-source tools
- Open Policy Agent (OPA)
- Cosign
Commercial / hosted tools
- Cerbos Hub
Threat mappings
MITRE ATLAS
- N/A
MAESTRO
- Compromised Agents (L7) (signed evidence-bound trust-state demotion marks a behaviorally divergent agent for downstream restriction or quarantine)
OWASP LLM Top 10 2026
- N/A
OWASP ML Top 10 2023
- N/A
OWASP Top 10 for Agentic Applications 2026
- ASI10:2026 Rogue Agents (signed evidence-bound trust-state demotion prevents stale or unverifiable agent state from being treated as normal)
NIST Adversarial Machine Learning 2025
- N/A
Cisco Integrated AI Security and Safety Framework
- N/A
Google Secure AI Framework 2.0 - Risks
- N/A
Databricks AI Security Framework 3.0
- Agents - Core 13.13: Rogue Agents in Multi-Agent Systems (signed runtime trust-state demotion marks behaviorally divergent agents for downstream restriction or quarantine)