AID-H-006.003: Prediction Response Minimization & Extraction-Resistance Validation

Version 1.20261015 · Actionable sub-technique

Expose only the numeric prediction detail required by an explicit API contract through one versioned response profile. For a minimization transform, gate release on a model-family query suite showing reduced extraction signal within utility and API-contract budgets. If calibrated probabilities are required, explicitly bind the permitted fields and precision to that contract, retain calibration and extraction-risk review, and enforce compensating access and query-abuse controls. Keeping required precision is not evidence that extraction risk was reduced.

Applicability: use only for APIs that expose classifier confidence or continuous predictions. If the API already returns no numeric model signal, this control does not apply. A configured bin, rounding rule, or noise flag alone is insufficient. This control makes no differential-privacy claim and rejects raw logits and probability fields outside the approved response contract.

Pillars: model, app

Lifecycle: building, validation, operation

Scope boundary

Owns contract-bounded response detail for numeric prediction APIs, including explicit calibrated-probability release decisions, and independent validation that rounding, binning, range reduction, or approved noise lowers extraction signal within utility and API-contract budgets. It makes no differential-privacy claim.

AID-H-005.001: AID-H-006.003 empirically reduces externally exposed prediction precision; AID-H-005.001 provides a quantified differential-privacy guarantee through calibrated noise and accounting. Response minimization must not be represented as DP unless the full privacy mechanism and accountant apply.

Implementation guidance

AID-H-006.003-G001

Apply a signed classifier response profile that permits only contract-required labels, confidence bands, or explicitly authorized calibrated probabilities.
Bind the response to the deployment contract

Load the profile from a signature-verified deployment source, never from a request parameter. Use labels or bands when sufficient. Enable calibrated_probabilities only when the contract requires the complete class vector at the model output precision; otherwise select a narrower profile. The release process verifies that the referenced calibration report covers the exact deployed model and class ordering, the risk review covers the endpoint and audience, and the named access/query-abuse controls are enforced. Nonempty reference strings below are structural checks, not evidence verification or calibration. Do not normalize logits and relabel them calibrated probabilities.

# File: hardening/classifier_response_gate.py
from __future__ import annotations

import math
import re
from numbers import Real


def validate_policy(policy: dict) -> None:
    if not isinstance(policy, dict):
        raise ValueError("response policy must be an object")
    keys = {"mode", "sum_tolerance", "confidence_bands"}
    mode = policy.get("mode")
    if mode == "calibrated_probabilities":
        keys.add("probability_release")
    if set(policy) != keys or mode not in {
        "top_class_only", "top_class_with_band", "calibrated_probabilities"
    }:
        raise ValueError("response policy schema or mode differs")
    tolerance = policy["sum_tolerance"]
    if (isinstance(tolerance, bool) or not isinstance(tolerance, Real) or
            not math.isfinite(tolerance) or not 0 < tolerance <= 0.01):
        raise ValueError("sum_tolerance is invalid")
    if mode == "calibrated_probabilities":
        release = policy["probability_release"]
        required = {"contract_id", "calibration_report_sha256", "risk_review_id", "compensating_controls"}
        if not isinstance(release, dict) or set(release) != required:
            raise ValueError("probability release decision is missing")
        if any(not isinstance(release[k], str) or not release[k].strip()
               for k in required - {"compensating_controls"}):
            raise ValueError("probability release binding is missing")
        if not re.fullmatch(r"[a-f0-9]{64}", release["calibration_report_sha256"]):
            raise ValueError("calibration report digest is invalid")
        controls = release["compensating_controls"]
        if (not isinstance(controls, list) or not controls or
                any(not isinstance(c, str) or not c.strip() for c in controls)):
            raise ValueError("compensating controls must be named")


def classifier_response(probabilities: list[float], policy: dict) -> dict:
    validate_policy(policy)
    values = list(probabilities)
    if (len(values) < 2 or any(isinstance(v, bool) or not isinstance(v, Real)
            or not math.isfinite(v) or not 0 <= v <= 1 for v in values)):
        raise ValueError("classifier probabilities must be one finite bounded vector")
    values = [float(v) for v in values]
    if not math.isclose(math.fsum(values), 1.0, abs_tol=policy["sum_tolerance"], rel_tol=0.0):
        raise ValueError("classifier probabilities do not sum to one")
    top_index = max(range(len(values)), key=values.__getitem__)
    output = {"prediction_class": top_index}
    if policy["mode"] == "top_class_only":
        return output
    if policy["mode"] == "calibrated_probabilities":
        output["probabilities"] = values
        return output
    bands = policy["confidence_bands"]
    if not isinstance(bands, list) or not bands:
        raise ValueError("confidence bands are required")
    if any(not isinstance(item, dict) or set(item) != {"minimum", "label"} for item in bands):
        raise ValueError("confidence-band schema differs")
    bounds = [item["minimum"] for item in bands]
    labels = [item["label"] for item in bands]
    if (any(isinstance(v, bool) or not isinstance(v, Real) or not math.isfinite(v)
            or not 0 <= v <= 1 for v in bounds) or
            bounds != sorted(set(bounds), reverse=True) or bounds[-1] != 0 or
            any(not isinstance(label, str) or not label for label in labels) or
            len(labels) != len(set(labels))):
        raise ValueError("confidence bands must be unique, finite, descending, and cover zero")
    for item in bands:
        if values[top_index] >= item["minimum"]:
            output["confidence_band"] = item["label"]
            return output
    raise ValueError("confidence bands do not cover the observed value")


def assert_minimized_response(response: dict, policy: dict) -> None:
    validate_policy(policy)
    allowed = {"prediction_class"}
    if policy["mode"] == "top_class_with_band":
        allowed.add("confidence_band")
    elif policy["mode"] == "calibrated_probabilities":
        allowed.add("probabilities")
    if not isinstance(response, dict) or set(response) != allowed:
        raise PermissionError("response exposes fields outside the signed response profile")
    if policy["mode"] == "calibrated_probabilities":
        expected = classifier_response(response["probabilities"], policy)
        if response != expected:
            raise PermissionError("probability response contradicts its prediction")
Verify safely

Replay label-only, banded, and explicitly authorized probability contracts against the exact model/schema release. Reject an absent or unverified policy, unknown mode, missing release binding, non-finite or out-of-range probabilities, inconsistent class order, and extra response fields (including raw logits). Verify that request input cannot select the profile. Exercise positive, negative, boundary, and dependency-failure fixtures and independent response readback. An unchanged calibrated vector makes no extraction-reduction claim; evaluate that residual risk and the compensating controls separately. For actual minimization transforms, use G003 to verify extraction reduction within the utility budget.

AID-H-006.003-G002

Apply a signed regression rounding or range profile with an exact response-schema gate.
# File: hardening/regression_response_gate.py
from __future__ import annotations

from decimal import Decimal, ROUND_HALF_EVEN


def regression_response(value: float, policy: dict) -> dict:
    if isinstance(value, bool) or not isinstance(value, (int, float, Decimal)):
        raise ValueError("regression value must be numeric and not boolean")
    observed = Decimal(str(value))
    if not observed.is_finite():
        raise ValueError("regression value must be finite")
    mode = policy.get("mode")
    if mode == "decimal_places":
        places = policy["decimal_places"]
        if isinstance(places, bool) or not isinstance(places, int) or places < 0 or places > 12:
            raise ValueError("decimal_places must be non-negative")
        quantum = Decimal(1).scaleb(-places)
        return {"prediction": str(observed.quantize(quantum, rounding=ROUND_HALF_EVEN))}
    if mode != "ranges":
        raise ValueError("unsupported regression minimization mode")

    ranges = policy.get("ranges")
    if not isinstance(ranges, list) or not ranges:
        raise ValueError("ranges are required")
    parsed_ranges = []
    for item in ranges:
        if not isinstance(item, dict) or set(item) != {"minimum", "maximum", "label"}:
            raise ValueError("range schema differs")
        lower, upper = Decimal(str(item["minimum"])), Decimal(str(item["maximum"]))
        if not lower.is_finite() or not upper.is_finite() or lower >= upper or not isinstance(item["label"], str) or not item["label"]:
            raise ValueError("range bounds or label are invalid")
        parsed_ranges.append((lower, upper, item))
    if parsed_ranges != sorted(parsed_ranges, key=lambda row: row[0]) or any(
            parsed_ranges[index - 1][1] > parsed_ranges[index][0]
            for index in range(1, len(parsed_ranges))):
        raise ValueError("ranges overlap or are not ordered")
    matches = [item for lower, upper, item in parsed_ranges if lower <= observed < upper]
    if len(matches) != 1:
        raise ValueError("ranges must cover the observed value exactly once")
    return {"prediction_range": str(matches[0]["label"])}


def assert_minimized_response(response: dict, policy: dict) -> None:
    expected = {"prediction"} if policy["mode"] == "decimal_places" else {"prediction_range"}
    if set(response) != expected:
        raise PermissionError("regression response exposes unapproved precision")
Keep the action safely bounded

Calibrate policy-approved bins or rounding on held-out data, apply it after internal decisions but before serialization, enforce finite domain/range, and deny raw or higher-precision score fields across all aliases and error paths.

Verify safely

A separately credentialed verifier independently reads back or replays every endpoint/model release whose numeric confidence or score precision is bounded by a signed disclosure policy, exercises positive, negative, boundary and dependency-failure fixtures, and reconciles the complete eligible population.

AID-H-006.003-G003

Independently validate the selected output-minimization profile against extraction and utility policy.
Separate a necessary precision exception from an effective transform

This effectiveness test applies to an actual minimization transform. A contract-required calibrated-probability profile that retains precision must not fabricate a passing reduction result: record that no reduction is established, bind the calibration and residual-risk review, and verify the separate access/query-abuse controls. The response contract still limits every exposed field.

Independent effectiveness gate:

The transform is accepted only from an evaluator-produced comparison over the same model digest, query-suite digest, attacker implementation, and utility population. Noise, if selected by policy, is merely one response transform and is not differential privacy.

# File: evaluation/gate_output_minimization.py
from __future__ import annotations

import hashlib
import json
import math
from pathlib import Path


class InsufficientData(RuntimeError):
    pass


def sha256_file(path: Path) -> str:
    return hashlib.sha256(path.read_bytes()).hexdigest()


def load_json(path: Path) -> dict:
    raw = path.read_bytes()
    if not raw or len(raw) > 4 * 1024 * 1024:
        raise ValueError("evaluation input is empty or oversized")
    def unique(pairs):
        value = {}
        for key, item in pairs:
            if key in value:
                raise ValueError(f"duplicate JSON key: {key}")
            value[key] = item
        return value
    def reject_constant(value):
        raise ValueError(f"non-finite JSON number: {value}")
    result = json.loads(raw.decode("utf-8", errors="strict"), object_pairs_hook=unique, parse_constant=reject_constant)
    if not isinstance(result, dict):
        raise ValueError("evaluation JSON root must be an object")
    return result

def finite_number(value: object, name: str) -> float:
    if isinstance(value, bool) or not isinstance(value, (int, float)) or not math.isfinite(value):
        raise ValueError(name + " must be a finite number")
    return float(value)

def gate(policy_path: Path, baseline_path: Path, candidate_path: Path) -> dict:
    policy = load_json(policy_path)
    baseline = load_json(baseline_path)
    candidate = load_json(candidate_path)
    bindings = ("model_sha256", "query_suite_sha256", "attacker_version", "utility_dataset_sha256")
    if any(baseline.get(key) != candidate.get(key) for key in bindings):
        raise InsufficientData("baseline and candidate evaluation bindings differ")
    measured = candidate.get("queries_measured")
    minimum = policy.get("minimum_queries_measured")
    if type(measured) is not int or type(minimum) is not int or minimum <= 0 or measured < minimum:
        raise InsufficientData("candidate query coverage is invalid or insufficient")
    baseline_score = finite_number(baseline.get("extraction_score"), "baseline extraction_score")
    candidate_score = finite_number(candidate.get("extraction_score"), "candidate extraction_score")
    utility_loss = finite_number(candidate.get("utility_loss"), "candidate utility_loss")
    maximum_score = finite_number(policy.get("maximum_extraction_score"), "maximum_extraction_score")
    minimum_reduction = finite_number(policy.get("minimum_extraction_score_reduction"), "minimum_extraction_score_reduction")
    maximum_utility_loss = finite_number(policy.get("maximum_utility_loss"), "maximum_utility_loss")
    if any(value < 0 or value > 1 for value in (baseline_score, candidate_score, utility_loss, maximum_score, minimum_reduction, maximum_utility_loss)):
        raise ValueError("evaluation scores and policy thresholds must be in [0, 1]")

    failures = []
    if candidate_score > maximum_score:
        failures.append("extraction_score")
    reduction = baseline_score - candidate_score
    if reduction < minimum_reduction:
        failures.append("extraction_reduction")
    if utility_loss > maximum_utility_loss:
        failures.append("utility_budget")
    if candidate.get("raw_numeric_signal_exposed") is not False:
        failures.append("raw_signal_exposed")
    return {
        "schema_version": "aidefend.output-minimization-evaluation.v1",
        "status": "fail" if failures else "pass",
        "failures": failures,
        "policy_sha256": sha256_file(policy_path),
        "baseline_evaluation_sha256": sha256_file(baseline_path),
        "candidate_evaluation_sha256": sha256_file(candidate_path),
        "bindings": {key: candidate[key] for key in bindings},
        "extraction_score_reduction": reduction,
    }

Evidence: The independent evaluator signs canonical baseline, candidate, and gate reports. Missing populations, nondeterministic/unreplayable attack inputs, absent utility data, or unsupported output type is insufficient_data; an API with no numeric output signal is explicitly not_applicable.

Keep the action safely bounded

Run the complete seeded extraction and utility population against both releases, bind all responses and metrics to exact artifact/config digests, independently reproduce them, and block promotion unless extraction-risk and utility gates pass.

Open-source tools

Commercial / hosted tools

Threat mappings

MITRE ATLAS

MAESTRO

OWASP LLM Top 10 2026

OWASP ML Top 10 2023

OWASP Top 10 for Agentic Applications 2026

NIST Adversarial Machine Learning 2025

Cisco Integrated AI Security and Safety Framework

Google Secure AI Framework 2.0 - Risks

Databricks AI Security Framework 3.0

AID-H-006.003: Prediction Response Minimization & Extraction-Resistance Validation | AIDEFEND

Loading interactive view…

AIDEFEND™

A Practical Knowledge Base for AI Security Defenses

Version

GitHub MCP Service