Scope boundary

Owns production quality tracking based on a real ground-truth feedback loop, including delayed label joins, metric comparison, label coverage, and feedback-population integrity.

AID-D-002.001: AID-D-002.002 measures correctness against trusted labels; AID-D-002.001 measures unlabeled input and output distribution shift. Ground-truth performance is a direct quality claim, while drift statistics are proxy signals that apply when labels are absent or delayed.

Implementation guidance

• AID-D-002.002-G001: Monitor model performance metrics by comparing predictions to ground truth labels.

AID-D-002.002-G001

Reusable prediction-label join

Never inner-join predictions to available labels and report the surviving metric as fleet performance. Preserve unlabeled, late, orphaned, duplicate, abstained, and errored records; evaluate metrics only on predictions whose signed label-delay window has matured.

from __future__ import annotations

# File: detection/labeled_performance.py

from datetime import datetime, timedelta, timezone

import pandas as pd
from sklearn.metrics import accuracy_score, f1_score


def evaluate_labeled_performance(
    predictions: pd.DataFrame,
    labels: pd.DataFrame,
    policy: dict,
    *,
    as_of: datetime,
) -> dict:
    required = {
        "policy_version", "label_delay_seconds", "minimum_labeled_predictions",
        "maximum_unlabeled_rate", "minimum_accuracy", "minimum_f1",
        "positive_label",
    }
    if set(policy) != required:
        raise ValueError("performance policy schema mismatch")
    if predictions["prediction_id"].duplicated().any() or labels["prediction_id"].duplicated().any():
        raise ValueError("duplicate prediction or label identity")
    expected_prediction_columns = {
        "prediction_id", "predicted_label", "predicted_at", "prediction_status",
    }
    expected_label_columns = {"prediction_id", "label", "labeled_at"}
    if not expected_prediction_columns.issubset(predictions) or not expected_label_columns.issubset(labels):
        raise ValueError("prediction or label schema mismatch")
    current = as_of.astimezone(timezone.utc)
    cutoff = current - timedelta(seconds=int(policy["label_delay_seconds"]))
    pred = predictions.copy()
    pred["predicted_at"] = pd.to_datetime(pred["predicted_at"], utc=True, errors="raise")
    lab = labels.copy()
    lab["labeled_at"] = pd.to_datetime(lab["labeled_at"], utc=True, errors="raise")
    eligible = pred[pred["predicted_at"] <= cutoff].copy()
    joined = eligible.merge(lab, on="prediction_id", how="left", validate="one_to_one", indicator=True)
    orphan_labels = sorted(set(lab["prediction_id"]) - set(pred["prediction_id"]))
    scored = joined[
        (joined["_merge"] == "both")
        & (joined["prediction_status"] == "PREDICTED")
        & joined["predicted_label"].notna()
        & joined["label"].notna()
    ]
    eligible_count = len(joined)
    labeled_count = int((joined["_merge"] == "both").sum())
    unlabeled_count = eligible_count - labeled_count
    unlabeled_rate = unlabeled_count / max(1, eligible_count)
    if eligible_count == 0 or len(scored) < int(policy["minimum_labeled_predictions"]):
        return {
            "control_status": "INSUFFICIENT_DATA",
            "finding_status": "UNKNOWN",
            "eligible_predictions": eligible_count,
            "labeled_predictions": labeled_count,
            "scored_predictions": len(scored),
            "unlabeled_predictions": unlabeled_count,
            "orphan_label_ids": orphan_labels,
            "policy_version": policy["policy_version"],
        }
    accuracy = float(accuracy_score(scored["label"], scored["predicted_label"]))
    f1 = float(
        f1_score(
            scored["label"],
            scored["predicted_label"],
            pos_label=policy["positive_label"],
            zero_division=0,
        )
    )
    reasons = []
    if unlabeled_rate > float(policy["maximum_unlabeled_rate"]):
        reasons.append("label_coverage_degraded")
    if accuracy < float(policy["minimum_accuracy"]):
        reasons.append("accuracy_below_policy")
    if f1 < float(policy["minimum_f1"]):
        reasons.append("f1_below_policy")
    if orphan_labels:
        reasons.append("orphan_ground_truth_labels")
    return {
        "schema_version": "aidefend.labeled_performance.v1",
        "control_status": "PASS",
        "finding_status": "FINDING" if reasons else "NO_FINDING",
        "policy_version": policy["policy_version"],
        "as_of": current.isoformat(),
        "eligible_predictions": eligible_count,
        "labeled_predictions": labeled_count,
        "scored_predictions": len(scored),
        "unlabeled_predictions": unlabeled_count,
        "unlabeled_rate": round(unlabeled_rate, 8),
        "orphan_label_ids": orphan_labels,
        "accuracy": round(accuracy, 8),
        "f1": round(f1, 8),
        "reasons": reasons,
    }

Verification: Replay matched, unlabeled, late-label, orphan-label, abstention, prediction-error, duplicate-ID, empty, and degraded-performance fixtures. Rebuild both source populations independently and verify that missing labels never disappear from the denominator.

Action: Run the join only after the signed maturity delay, retain every unmatched state, and route coverage and performance findings without triggering retraining from this detector.

Open-source tools

• scikit-learn
• MLflow
• pandas
• Polars
• Prometheus
• Grafana

Commercial / hosted tools

• Weights & Biases
• Arize AX
• Fiddler AI Observability
• Evidently Cloud
• Datadog

Threat mappings

MITRE ATLAS

• AML.T0031 Erode AI Model Integrity
• AML.T0015 Evade AI Model
• AML.T0020 Training Data Poisoning
• AML.T0076 Corrupt AI Model

MAESTRO

• Manipulation of Evaluation Metrics (L5) (live labeled metrics detect degraded quality)
• Data Poisoning (L2)
• Adversarial Examples (L1)

OWASP LLM Top 10 2026

• LLM05:2026 Data and Model Poisoning (ground-truth degradation can expose poisoning-induced behavior without proving its cause)
• LLM07:2026 Misinformation

OWASP ML Top 10 2023

• ML08:2023 Model Skewing
• ML02:2023 Data Poisoning Attack (labeled performance degradation is a primary poisoning indicator)
• ML10:2023 Model Poisoning (labeled live evaluation catches poisoned-model behavior)
• ML01:2023 Input Manipulation Attack (labeled feedback reveals evasion success)

OWASP Top 10 for Agentic Applications 2026

• N/A

NIST Adversarial Machine Learning 2025

• NISTAML.013 Data Poisoning
• NISTAML.011 Model Poisoning (Availability)
• NISTAML.026 Model Poisoning (Integrity)
• NISTAML.022 Evasion
• NISTAML.027 Misaligned Outputs

Cisco Integrated AI Security and Safety Framework

• N/A

Google Secure AI Framework 2.0 - Risks

• DP: Data Poisoning
• MEV: Model Evasion

Databricks AI Security Framework 3.0

• Algorithms 5.2: Model drift
• Model Serving - Inference response 10.1: Lack of audit and monitoring inference quality