AID-H-007.005: Fine-Tuning Safety Alignment Preservation & Erosion Detection

Version 1.20261015 · Actionable sub-technique

Gate fine-tuned, adapter-tuned, instruction-tuned, RLHF/RLAIF-updated, or otherwise customized models against a pre/post safety regression suite before promotion. The control compares the tuned model to an approved baseline on refusal behavior, harmful-content compliance, jailbreak and prompt-injection resistance, tool-use policy adherence, and sensitive-data leakage. Promotion is blocked when the tuned model materially erodes safety alignment even if ordinary task-quality metrics improve.

Pillars: model, app

Lifecycle: validation, improvement

Scope boundary

Owns the pre/post safety-alignment regression gate for fine-tuned, adapter-tuned, instruction-tuned, RLHF or RLAIF-updated, and otherwise customized target models, blocking promotion on policy-defined safety erosion.

AID-H-007.004: AID-H-007.005 evaluates safety erosion caused by model customization; AID-H-007.004 governs integrity and sufficiency of the evaluation population and promotion evidence. The safety delta gate consumes but does not replace evaluation governance.

AID-H-007.006: AID-H-007.005 covers changes that customize model behavior through fine-tuning or alignment training; AID-H-007.006 covers deployment artifacts and serving pairings derived from an already approved source through quantization, pruning, conversion, merge, distillation, or compilation. Choose ownership from the transformation that produced the candidate, not merely from the output format.

AID-H-036: AID-H-007.005 evaluates the safety alignment of the tuned model being promoted; AID-H-036 evaluates the multilingual and locale-stratified performance of prompt-safety classifiers. The protected model and the guardrail classifier are different evaluation subjects.

Implementation guidance

AID-H-007.005-G001

Run a pre/post fine-tune safety regression gate and block model promotion when safety-alignment erosion exceeds approved thresholds.
Concept:

Fine-tuning can improve task performance while quietly weakening refusal behavior or jailbreak resistance. Evaluate the approved baseline model and the tuned candidate on the same signed safety suite, compare category-level regressions, and fail promotion when the candidate erodes safety beyond policy.

Step 1: Load the signed safety-gate policy and evaluator evidence

The safety-policy authority supplies concrete finite rate thresholds in a signed policy. A separately administered evaluator signs the baseline and candidate result populations. The gate below verifies byte snapshots under fixed trust anchors, validates exact schemas and complete category coverage, and never accepts an edit-time threshold.

Step 2: Compare the complete signed baseline and candidate populations
# File: eval/fine_tune_safety_gate.py
from __future__ import annotations
import hashlib, json, math, os, subprocess, tempfile
from dataclasses import asdict, dataclass
from pathlib import Path

POLICY_KEY = Path("/opt/aidefend/trust/fine-tune-safety-policy.pub")
EVALUATOR_KEY = Path("/opt/aidefend/trust/safety-evaluator.pub")
REQUIRED = {"harmful_content", "jailbreak", "prompt_injection", "sensitive_data", "tool_policy"}

def verify_timeout() -> float:
    value = float(os.environ["VERIFIED_H007_FINE_TUNE_GATE_VERIFY_TIMEOUT_SECONDS"])
    if not math.isfinite(value) or value <= 0: raise ValueError("verification timeout must be finite and positive")
    return value

def signed_json(path: str, signature: str, key: Path) -> tuple[dict, str]:
    payload = Path(path).read_bytes()
    with tempfile.NamedTemporaryFile(prefix="aidefend-safety-gate-", delete=False) as handle:
        handle.write(payload); snapshot = Path(handle.name)
    try:
        subprocess.run(["cosign", "verify-blob", "--key", str(key), "--bundle", signature, str(snapshot)], check=True, capture_output=True, text=True, timeout=verify_timeout())
    finally:
        snapshot.unlink(missing_ok=True)
    return json.loads(payload), hashlib.sha256(payload).hexdigest()

def rate(value, label: str) -> float:
    if isinstance(value, bool) or not isinstance(value, (int, float)) or not math.isfinite(value) or not 0 <= value <= 1: raise ValueError(label + " must be a finite rate")
    return float(value)

def validate_policy(policy: dict) -> None:
    fields = {"schema_version", "policy_version", "max_policy_violation_rate_delta", "max_refusal_regression", "max_sensitive_data_leakage_delta", "min_jailbreak_block_rate", "required_categories"}
    if not isinstance(policy, dict) or set(policy) != fields: raise ValueError("safety policy schema mismatch")
    if policy["schema_version"] != "aidefend.fine-tune-safety-policy.v1" or not isinstance(policy["policy_version"], str) or not policy["policy_version"]: raise ValueError("safety policy identity is invalid")
    if not isinstance(policy["required_categories"], list) or set(policy["required_categories"]) != REQUIRED or len(policy["required_categories"]) != len(REQUIRED): raise ValueError("required category population differs")
    for field in fields - {"schema_version", "policy_version", "required_categories"}: rate(policy[field], field)

def validate_results(results: dict, required: list[str]) -> None:
    fields = {"schema_version", "model_id", "model_sha256", "suite_id", "case_set_sha256", "categories"}
    if not isinstance(results, dict) or set(results) != fields or results["schema_version"] != "aidefend.fine-tune-safety-results.v1": raise ValueError("safety result schema mismatch")
    for field in ("model_id", "suite_id"):
        if not isinstance(results[field], str) or not results[field]: raise ValueError(field + " is required")
    for field in ("model_sha256", "case_set_sha256"):
        if not isinstance(results[field], str) or len(results[field]) != 64 or any(c not in "0123456789abcdef" for c in results[field]): raise ValueError(field + " must be lowercase SHA-256")
    if not isinstance(results["categories"], dict) or set(results["categories"]) != set(required): raise ValueError("result category population differs")
    for category, metrics in results["categories"].items():
        expected = {"policy_violation_rate", "refusal_rate"}
        if category == "jailbreak": expected.add("block_rate")
        if category == "sensitive_data": expected.add("leakage_rate")
        if not isinstance(metrics, dict) or set(metrics) != expected: raise ValueError("metric population differs for " + category)
        for metric, value in metrics.items(): rate(value, category + "." + metric)

@dataclass(frozen=True)
class GateFinding:
    category: str; metric: str; baseline: float; candidate: float; threshold: float; passed: bool

def compare_results(policy: dict, baseline: dict, candidate: dict) -> dict:
    validate_policy(policy); required = policy["required_categories"]
    validate_results(baseline, required); validate_results(candidate, required)
    if baseline["suite_id"] != candidate["suite_id"] or baseline["case_set_sha256"] != candidate["case_set_sha256"]: raise ValueError("baseline and candidate populations differ")
    findings = []
    for category in required:
        base, cand = baseline["categories"][category], candidate["categories"][category]
        delta = cand["policy_violation_rate"] - base["policy_violation_rate"]
        findings.append(GateFinding(category, "policy_violation_rate_delta", base["policy_violation_rate"], cand["policy_violation_rate"], policy["max_policy_violation_rate_delta"], delta <= policy["max_policy_violation_rate_delta"]))
        regression = base["refusal_rate"] - cand["refusal_rate"]
        findings.append(GateFinding(category, "refusal_regression", base["refusal_rate"], cand["refusal_rate"], policy["max_refusal_regression"], regression <= policy["max_refusal_regression"]))
        if category == "jailbreak": findings.append(GateFinding(category, "candidate_jailbreak_block_rate", base["block_rate"], cand["block_rate"], policy["min_jailbreak_block_rate"], cand["block_rate"] >= policy["min_jailbreak_block_rate"]))
        if category == "sensitive_data":
            leakage = cand["leakage_rate"] - base["leakage_rate"]
            findings.append(GateFinding(category, "sensitive_data_leakage_delta", base["leakage_rate"], cand["leakage_rate"], policy["max_sensitive_data_leakage_delta"], leakage <= policy["max_sensitive_data_leakage_delta"]))
    return {"decision": "pass" if all(item.passed for item in findings) else "block", "findings": [asdict(item) for item in findings], "baseline_model_sha256": baseline["model_sha256"], "candidate_model_sha256": candidate["model_sha256"], "suite_id": candidate["suite_id"], "case_set_sha256": candidate["case_set_sha256"]}

if __name__ == "__main__":
    policy, policy_sha = signed_json("eval/fine_tune_safety_gate_policy.json", "eval/fine_tune_safety_gate_policy.json.sig", POLICY_KEY)
    baseline, baseline_sha = signed_json("artifacts/baseline_safety_results.json", "artifacts/baseline_safety_results.json.sig", EVALUATOR_KEY)
    candidate, candidate_sha = signed_json("artifacts/candidate_safety_results.json", "artifacts/candidate_safety_results.json.sig", EVALUATOR_KEY)
    report = compare_results(policy, baseline, candidate); report.update({"policy_sha256": policy_sha, "baseline_evidence_sha256": baseline_sha, "candidate_evidence_sha256": candidate_sha})
    Path("artifacts/fine_tune_safety_gate_report.json").write_text(json.dumps(report, indent=2, sort_keys=True) + "\n", encoding="utf-8")
    raise SystemExit(0 if report["decision"] == "pass" else 1)
Step 3: Bind the gate result to promotion

The gate report should be signed or stored as part of the model promotion evidence bundle. Promotion systems should fail closed when the report is missing, stale, unsigned, generated from a different model digest, or produced with an unapproved safety suite.

Action: Run the same safety suite before and after fine-tuning. Promote the tuned model only when it preserves approved safety behavior within threshold and the report is bound to the exact candidate artifact.

Verify safely

A separately credentialed verifier independently reads back or replays matched base and fine-tuned artifacts under one signed safety, security, capability and utility evaluation profile, exercises positive, negative, boundary and dependency-failure fixtures, and reconciles the complete eligible population.

Open-source tools

Source-available / open-weight tools

Commercial / hosted tools

Threat mappings

MITRE ATLAS

MAESTRO

OWASP LLM Top 10 2026

OWASP ML Top 10 2023

OWASP Top 10 for Agentic Applications 2026

NIST Adversarial Machine Learning 2025

Cisco Integrated AI Security and Safety Framework

Google Secure AI Framework 2.0 - Risks

Databricks AI Security Framework 3.0

AID-H-007.005: Fine-Tuning Safety Alignment Preservation & Erosion Detection | AIDEFEND

Loading interactive view…

AIDEFEND™

A Practical Knowledge Base for AI Security Defenses

Version

GitHub MCP Service