wip: phase_z2 evidence 파이프라인 + matching 실험(phase2~26) + 프론트 trace 패널 진행분 스냅샷
- src: phase_z2 composition/mapper/pipeline/placement_planner/retry, ai_fallback(prompts/schema/validate), mdx_text_atoms 신규 - Front: PipelineTracePanel 신규, FramePanel/SlideCanvas/Home/designAgentApi 등 갱신 + 테스트 4종 추가 - templates/phase_z2: catalog(component_expansion_registry, node_slot_mapping 신규), frames, families, slide_base 갱신 - tests/matching: phase2~26 매칭 실험 스크립트·리포트·온톨로지 전체 (미커밋 진행분) - tests: b4_v4 evidence, task5~28.5 시리즈, regression(imp95 baseline) 등 신규 테스트 대량 추가 - docs/reference: MDX 구조 인벤토리, MDX→Frame 구조 계약 문서 - scripts: mdx 계약/parity/coverage/viewport 체크, gitea comment, run sync 유틸 - .gitignore: tmp*.json, chromedriver, .orchestrator, *.pkl, Front_test* 등 임시/스냅샷 제외 미완성 작업의 보존용 스냅샷 커밋 (2026-07-02) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.8
@@ -0,0 +1,48 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"axis": "IMP-95 u8 — final.html SHA baseline captured via FULL run_phase_z2_mvp1 pipeline under PHASE_Z_B4_V4_EVIDENCE=OFF and PHASE_Z_B4_MAPPER_SOURCE=OFF (defaults)",
|
||||
"description": "Frozen SHA-256 of `final.html` bytes (production write site src/phase_z2_pipeline.py:5994-5996) for mdx 01/02/04/05 under PHASE_Z_B4_V4_EVIDENCE OFF. Under flag OFF, IMP-95 (u1~u7) is strictly no-op for final.html (planner branch falls through to legacy _select_frame at u3; u4/u5/u6 additive telemetry confined to placement_trace per the trace-only docstring at src/phase_z2_pipeline.py:86). The u8 regression test asserts SHA equality with these frozen values, so any future code change that drifts the flag-OFF render output produces a mismatch and breaks the test. mdx 03 is excluded per Stage 2 u8 scope (mdx 03 정비 LOCK). Regenerate only when an upstream delta is reviewed and accepted as the new pre-IMP-95 reference.",
|
||||
"captured_at_utc": "2026-05-27T08:23:24Z",
|
||||
"renderer": {
|
||||
"entrypoint": "src.phase_z2_pipeline.run_phase_z2_mvp1",
|
||||
"write_site": "src/phase_z2_pipeline.py:5994-5996",
|
||||
"artifact_relpath": "<RUNS_DIR>/<run_id>/phase_z2/final.html"
|
||||
},
|
||||
"mdx_batch": [
|
||||
"01.mdx",
|
||||
"02.mdx",
|
||||
"04.mdx",
|
||||
"05.mdx"
|
||||
],
|
||||
"mdx_files": {
|
||||
"01.mdx": {
|
||||
"mdx_file": "01.mdx",
|
||||
"run_id": "imp95_baseline_01",
|
||||
"final_html_size_bytes": 29959,
|
||||
"sha256": "62d793166ca4cdd8d8d1d134d8659b43dce95bb958e7efd6ce0e72619b3c335d",
|
||||
"pipeline_exit_code": null
|
||||
},
|
||||
"02.mdx": {
|
||||
"mdx_file": "02.mdx",
|
||||
"run_id": "imp95_baseline_02",
|
||||
"final_html_size_bytes": 38135,
|
||||
"sha256": "d6df786e322bac9b337cd06a62a8a817fd34f5a17dcacc45452a94808fcbf0ed",
|
||||
"pipeline_exit_code": null
|
||||
},
|
||||
"04.mdx": {
|
||||
"mdx_file": "04.mdx",
|
||||
"run_id": "imp95_baseline_04",
|
||||
"final_html_size_bytes": 52646,
|
||||
"sha256": "9f93bedc4ef11ce21fb2c22d8dd7798cfdd7e73f550ba1141a6b5f6fa974fd8f",
|
||||
"pipeline_exit_code": 1
|
||||
},
|
||||
"05.mdx": {
|
||||
"mdx_file": "05.mdx",
|
||||
"run_id": "imp95_baseline_05",
|
||||
"final_html_size_bytes": 31751,
|
||||
"sha256": "f212df273f7761c60631c54be3847a57f9226ed50bfe8b702e166e7a0413babb",
|
||||
"pipeline_exit_code": null
|
||||
}
|
||||
},
|
||||
"total_files": 4
|
||||
}
|
||||
@@ -0,0 +1,190 @@
|
||||
"""IMP-95 u8 — capture final.html SHA baseline via the FULL Phase Z pipeline
|
||||
under PHASE_Z_B4_V4_EVIDENCE=OFF (default) for mdx 01/02/04/05.
|
||||
|
||||
Runs ``src.phase_z2_pipeline.run_phase_z2_mvp1`` end-to-end for each mdx file
|
||||
in the Stage 2 u8 scope (01/02/04/05; mdx 03 is excluded per the ``mdx 03
|
||||
정비`` lock). Each run writes a real ``final.html`` to disk at the production
|
||||
write site ``src/phase_z2_pipeline.py:5994-5996``; bytes are IMP-94-normalized
|
||||
and SHA-256 hashed, then frozen to
|
||||
``tests/regression/fixtures/imp95_pre_baseline_sha.json``.
|
||||
|
||||
The accompanying regression test
|
||||
``tests/regression/test_b4_v4_evidence_off_sha_parity.py`` re-runs the same
|
||||
pipeline shape under flag OFF and asserts SHA equality with the frozen
|
||||
values. Under flag OFF, IMP-95 (u1~u7) is a strict no-op for ``final.html``
|
||||
bytes — see the trace-only docstring at ``src/phase_z2_pipeline.py:86`` —
|
||||
so the captured baseline IS the immediate pre-IMP-95 reference even when
|
||||
captured with IMP-95 code already in tree.
|
||||
|
||||
Reason for a separate baseline (vs. reusing 89a_pre_baseline_sha.json)
|
||||
======================================================================
|
||||
|
||||
The 89a baseline was captured at HEAD ``6e9e3ee`` before subsequent
|
||||
working-tree changes accumulated (e.g. the Emergency P3/P4/P4b verbatim
|
||||
slot_payload builders), so the 89a SHAs no longer match the current
|
||||
flag-OFF pipeline output for mdx 02/04/05 — a pre-existing baseline
|
||||
drift orthogonal to IMP-95. A fresh baseline keyed to the current
|
||||
flag-OFF state isolates IMP-95's u8 regression axis from that upstream
|
||||
drift; the 89a baseline remains the load-bearing guard for the 89-a
|
||||
axis.
|
||||
|
||||
IMP-94 marker normalization
|
||||
===========================
|
||||
|
||||
IMP-94 ``data-region-id`` / ``data-content-unit-id`` tokens (stamped at
|
||||
``src/region_marker_stamper.py:131-135``) are stripped before hashing.
|
||||
The strip is anchored on ``leading space + attr token`` shape and is
|
||||
disjoint from the IMP-96 ``data-frame-slot-id`` axis by attribute name.
|
||||
|
||||
Run from repo root::
|
||||
|
||||
python tests/regression/scripts/capture_imp95_pre_baseline.py
|
||||
|
||||
Idempotent. Re-run only when an upstream mapper/render/template delta is
|
||||
reviewed and accepted as the new pre-IMP-95 reference. Refuses to run
|
||||
with PHASE_Z_B4_V4_EVIDENCE enabled (the flag-ON state is not the
|
||||
baseline axis).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import tempfile
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[3]
|
||||
sys.path.insert(0, str(_REPO_ROOT))
|
||||
sys.path.insert(0, str(_REPO_ROOT / "src"))
|
||||
|
||||
import src.phase_z2_pipeline as pz2 # noqa: E402
|
||||
|
||||
_SAMPLES_DIR = _REPO_ROOT / "samples" / "mdx_batch"
|
||||
_MDX_BATCH = ("01.mdx", "02.mdx", "04.mdx", "05.mdx") # Stage 2 u8 scope
|
||||
_OUT_PATH = (
|
||||
_REPO_ROOT
|
||||
/ "tests"
|
||||
/ "regression"
|
||||
/ "fixtures"
|
||||
/ "imp95_pre_baseline_sha.json"
|
||||
)
|
||||
|
||||
# IMP-94 additive marker strip patterns — must stay in sync with
|
||||
# tests/regression/test_b4_v4_evidence_off_sha_parity.py.
|
||||
_STRIP_REGION_ID_RE = re.compile(rb' data-region-id="[^"]*"')
|
||||
_STRIP_CONTENT_UNIT_ID_RE = re.compile(rb' data-content-unit-id="[^"]*"')
|
||||
|
||||
|
||||
def _strip_imp94_markers(raw_bytes: bytes) -> bytes:
|
||||
stripped = _STRIP_REGION_ID_RE.sub(b"", raw_bytes)
|
||||
stripped = _STRIP_CONTENT_UNIT_ID_RE.sub(b"", stripped)
|
||||
return stripped
|
||||
|
||||
|
||||
def _capture_one(mdx_file: str, runs_root: Path) -> dict:
|
||||
mdx_path = _SAMPLES_DIR / mdx_file
|
||||
assert mdx_path.exists(), f"sample missing: {mdx_path}"
|
||||
|
||||
run_id = f"imp95_baseline_{mdx_path.stem}"
|
||||
pipeline_exit_code: int | None = None
|
||||
try:
|
||||
pz2.run_phase_z2_mvp1(mdx_path, run_id=run_id)
|
||||
except SystemExit as exc:
|
||||
pipeline_exit_code = (
|
||||
int(exc.code) if isinstance(exc.code, int) else 1
|
||||
)
|
||||
|
||||
final_html_path = runs_root / run_id / "phase_z2" / "final.html"
|
||||
assert final_html_path.exists(), (
|
||||
f"final.html not written by pipeline: {final_html_path} "
|
||||
f"(pipeline_exit_code={pipeline_exit_code})"
|
||||
)
|
||||
raw_bytes = final_html_path.read_bytes()
|
||||
assert len(raw_bytes) > 0, f"final.html is empty: {final_html_path}"
|
||||
normalized_bytes = _strip_imp94_markers(raw_bytes)
|
||||
|
||||
return {
|
||||
"mdx_file": mdx_file,
|
||||
"run_id": run_id,
|
||||
"final_html_size_bytes": len(normalized_bytes),
|
||||
"sha256": hashlib.sha256(normalized_bytes).hexdigest(),
|
||||
"pipeline_exit_code": pipeline_exit_code,
|
||||
}
|
||||
|
||||
|
||||
def capture() -> dict:
|
||||
assert os.environ.get("PHASE_Z_B4_V4_EVIDENCE", "") == "", (
|
||||
"PHASE_Z_B4_V4_EVIDENCE must be unset when capturing baseline "
|
||||
"(default-OFF state is the production-equivalent axis for u8). "
|
||||
"Refusing to run with the flag enabled."
|
||||
)
|
||||
assert os.environ.get("PHASE_Z_B4_MAPPER_SOURCE", "") == "", (
|
||||
"PHASE_Z_B4_MAPPER_SOURCE must also be unset — the u8 baseline "
|
||||
"is captured under both flags OFF (full pre-IMP-89/95 path)."
|
||||
)
|
||||
|
||||
_OUT_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix="imp95_baseline_") as tmp:
|
||||
runs_root = Path(tmp)
|
||||
original_runs_dir = pz2.RUNS_DIR
|
||||
pz2.RUNS_DIR = runs_root
|
||||
try:
|
||||
entries = [_capture_one(mf, runs_root) for mf in _MDX_BATCH]
|
||||
finally:
|
||||
pz2.RUNS_DIR = original_runs_dir
|
||||
|
||||
return {
|
||||
"schema_version": 1,
|
||||
"axis": (
|
||||
"IMP-95 u8 — final.html SHA baseline captured via FULL "
|
||||
"run_phase_z2_mvp1 pipeline under PHASE_Z_B4_V4_EVIDENCE=OFF "
|
||||
"and PHASE_Z_B4_MAPPER_SOURCE=OFF (defaults)"
|
||||
),
|
||||
"description": (
|
||||
"Frozen SHA-256 of `final.html` bytes (production write site "
|
||||
"src/phase_z2_pipeline.py:5994-5996) for mdx 01/02/04/05 "
|
||||
"under PHASE_Z_B4_V4_EVIDENCE OFF. Under flag OFF, IMP-95 "
|
||||
"(u1~u7) is strictly no-op for final.html (planner branch "
|
||||
"falls through to legacy _select_frame at u3; u4/u5/u6 "
|
||||
"additive telemetry confined to placement_trace per the "
|
||||
"trace-only docstring at src/phase_z2_pipeline.py:86). The "
|
||||
"u8 regression test asserts SHA equality with these frozen "
|
||||
"values, so any future code change that drifts the flag-OFF "
|
||||
"render output produces a mismatch and breaks the test. "
|
||||
"mdx 03 is excluded per Stage 2 u8 scope (mdx 03 정비 LOCK). "
|
||||
"Regenerate only when an upstream delta is reviewed and "
|
||||
"accepted as the new pre-IMP-95 reference."
|
||||
),
|
||||
"captured_at_utc": (
|
||||
datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
),
|
||||
"renderer": {
|
||||
"entrypoint": "src.phase_z2_pipeline.run_phase_z2_mvp1",
|
||||
"write_site": "src/phase_z2_pipeline.py:5994-5996",
|
||||
"artifact_relpath": "<RUNS_DIR>/<run_id>/phase_z2/final.html",
|
||||
},
|
||||
"mdx_batch": list(_MDX_BATCH),
|
||||
"mdx_files": {entry["mdx_file"]: entry for entry in entries},
|
||||
"total_files": len(entries),
|
||||
}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
data = capture()
|
||||
_OUT_PATH.write_text(
|
||||
json.dumps(data, indent=2, ensure_ascii=False) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
print(
|
||||
f"wrote {_OUT_PATH} ({data['total_files']} files: "
|
||||
f"{', '.join(data['mdx_files'].keys())})"
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,360 @@
|
||||
"""IMP-95 u9 — Flag-ON adapter_needed monotonicity guard with PHASE_Z_B4_GATEKEEPER=1.
|
||||
|
||||
Stage 2 u9 contract (verbatim)::
|
||||
|
||||
Flag-ON adapter_needed count monotone non-increasing regression with
|
||||
PHASE_Z_B4_GATEKEEPER=1.
|
||||
|
||||
Stage 2 axis A11 binding invariant
|
||||
==================================
|
||||
|
||||
When ``PHASE_Z_B4_GATEKEEPER=1`` is held constant, turning
|
||||
``PHASE_Z_B4_V4_EVIDENCE`` from OFF (declaration-order legacy selector)
|
||||
to ON (V4-aware selector) MUST be **monotone non-increasing** on
|
||||
``adapter_needed_count``. Rationale (Stage 1 root_cause line in the
|
||||
exit report): the V4-aware selector prefers eligible contracts whose
|
||||
``template_id`` matches the mapper's V4 rank-1 — therefore
|
||||
``matches_mapper`` is True at least as often under flag ON as under
|
||||
flag OFF, so the gatekeeper trigger ``not matches_mapper`` (the sole
|
||||
``adapter_needed_units.append`` site at ``src/phase_z2_pipeline.py:6747``)
|
||||
fires at most as often.
|
||||
|
||||
The inverse (V4 evidence consumption *increasing* adapter_needed) would
|
||||
mean V4 ranks are systematically picking contracts the mapper rejects —
|
||||
i.e. V4 evidence is *worse* than declaration-order, which violates the
|
||||
"V4 evidence → B4 selection alignment" premise that motivates IMP-95.
|
||||
That would surface here as ``treatment_count > baseline_count`` on at
|
||||
least one mdx file, and the per-file + holistic assertions below would
|
||||
fail with the offending file + counts in the error message.
|
||||
|
||||
Why this guard cannot be a unit test
|
||||
====================================
|
||||
|
||||
The gatekeeper trigger reads from real ``CompositionUnit.v4_candidates``
|
||||
(produced by the composition stage from real V4 evidence YAMLs, not a
|
||||
stub) and from ``placement_plan.selected_template_id`` (resolved by
|
||||
``_select_frame_v4_aware`` against the full ``frame_contracts.yaml``).
|
||||
A unit-level fixture would have to mock both sides and would not detect
|
||||
upstream drift in (a) the V4 evidence shape, (b) the
|
||||
``accepted_content_types`` declarations, (c) the partial_exists callable
|
||||
wiring (u6), (d) the gatekeeper ``not matches_mapper`` site itself, or
|
||||
(e) ``CompositionUnit.v4_candidates`` field semantics. Running the FULL
|
||||
``run_phase_z2_mvp1`` pipeline twice per mdx file (flag OFF → flag ON,
|
||||
both with ``PHASE_Z_B4_GATEKEEPER=1`` pinned) catches all five axes.
|
||||
|
||||
Reading adapter_needed_count via captured stdout
|
||||
================================================
|
||||
|
||||
The Step 20 ``slide_status`` artifact (write site at
|
||||
``src/phase_z2_pipeline.py:8620-8628``) is NOT a reliable read source
|
||||
under ``PHASE_Z_B4_GATEKEEPER=1``: the gatekeeper short-circuit at
|
||||
``src/phase_z2_pipeline.py:6730-6750`` increments
|
||||
``adapter_needed_units`` but — unlike the IMP-86 u1 FitError path at
|
||||
``src/phase_z2_pipeline.py:6914-6997`` which appends an
|
||||
``__empty__`` placeholder to ``zones_data`` — leaves ``zones_data``
|
||||
empty. When every zone of an mdx file traverses the gatekeeper short-
|
||||
circuit (the case on mdx 01 in the rewind report at
|
||||
``Codex #23`` baseline run), the pre-build layout invariant at
|
||||
``src/phase_z2_pipeline.py:7572-7588`` raises ``ValueError`` BEFORE
|
||||
Step 20 writes the artifact, leaving ``adapter_needed_count`` unreadable
|
||||
from disk. (This asymmetry is a real production defect surfaced by u9;
|
||||
see ``follow_up_issue_candidates`` below.)
|
||||
|
||||
The robust read source is captured stdout. Both adapter_needed code
|
||||
paths print a unique trailing literal **immediately after**
|
||||
``adapter_needed_units.append`` and **before** the ``continue``:
|
||||
|
||||
* ``src/phase_z2_pipeline.py:6748-6749`` — V4-B4 gatekeeper short-
|
||||
circuit: ``"... v4_b4_mismatch → adapter_needed (skip render)"``
|
||||
(preceded by ``adapter_needed_units.append`` at line 6747).
|
||||
* ``src/phase_z2_pipeline.py:6998-6999`` — IMP-86 u1 FitError fall-
|
||||
back: ``"... FitError → adapter_needed (skip render)"`` (preceded
|
||||
by ``adapter_needed_units.append`` at line 6922).
|
||||
|
||||
The literal ``adapter_needed (skip render)`` is unique to these two
|
||||
sites (verified via ``rg "adapter_needed \\(skip render\\)" src/``),
|
||||
so counting its occurrences in captured stdout = total
|
||||
``adapter_needed_count`` for the run. Both prints fire **before** the
|
||||
layout invariant at line 7572 and the IMP-87 ``EMPTY_SHELL_NO_CONTENT``
|
||||
``sys.exit(1)`` at line 8782, so the count is authoritative regardless
|
||||
of which terminal state the pipeline reaches.
|
||||
|
||||
mdx 03 exclusion + Stage 2 u9 scope
|
||||
===================================
|
||||
|
||||
Stage 2 u9 scopes the regression to ``mdx 01/02/04/05`` (mdx 03 is
|
||||
under the user-locked ``mdx 03 정비`` axis).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
import src.phase_z2_pipeline as pz2
|
||||
from src.phase_z2_placement_planner import (
|
||||
PHASE_Z_B4_V4_EVIDENCE_ENV,
|
||||
_b4_v4_evidence_enabled,
|
||||
)
|
||||
|
||||
V4_FLAG = PHASE_Z_B4_V4_EVIDENCE_ENV
|
||||
GATEKEEPER_FLAG = "PHASE_Z_B4_GATEKEEPER"
|
||||
MAPPER_FLAG = "PHASE_Z_B4_MAPPER_SOURCE"
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
_SAMPLES_DIR = _REPO_ROOT / "samples" / "mdx_batch"
|
||||
_MDX_BATCH = ("01.mdx", "02.mdx", "04.mdx", "05.mdx") # Stage 2 u9 scope
|
||||
|
||||
# Adapter-needed print literal anchored at src/phase_z2_pipeline.py:6749
|
||||
# (v4_b4_mismatch gatekeeper short-circuit) and :6999 (IMP-86 u1 FitError
|
||||
# fallback). Both sites print **after** ``adapter_needed_units.append``
|
||||
# and **before** ``continue``, so per-occurrence count = adapter signal
|
||||
# count. The literal is unique to these two sites by rg search across
|
||||
# src/ — any future adapter_needed site that omits this exact suffix
|
||||
# would silently shrink the count, so u9 also pins the substring shape
|
||||
# below in ``test_imp95_u9_adapter_print_pattern_is_unique_and_anchored``.
|
||||
_ADAPTER_LITERAL = "adapter_needed (skip render)"
|
||||
|
||||
|
||||
def _run_pipeline_count_adapter_signals(
|
||||
mdx_file: str,
|
||||
runs_root: Path,
|
||||
run_id: str,
|
||||
capfd: pytest.CaptureFixture[str],
|
||||
) -> int:
|
||||
"""Run ``run_phase_z2_mvp1`` once and tally adapter_needed prints.
|
||||
|
||||
Three terminal states are tolerated by design:
|
||||
|
||||
* Normal return — pipeline completed Step 20.
|
||||
* ``SystemExit`` — IMP-87 ``EMPTY_SHELL_NO_CONTENT BLOCKED`` at
|
||||
``src/phase_z2_pipeline.py:8782``. Fires AFTER both adapter print
|
||||
sites (6749, 6999).
|
||||
* ``ValueError`` — pre-build layout invariant at
|
||||
``src/phase_z2_pipeline.py:7572-7588``. Fires AFTER both adapter
|
||||
print sites but BEFORE Step 20 write. The rewind harness
|
||||
(``[Codex #23]``) hits this state on every mdx under
|
||||
``PHASE_Z_B4_GATEKEEPER=1`` baseline because the gatekeeper
|
||||
short-circuit at line 6747 leaves ``zones_data`` empty.
|
||||
|
||||
In all three states ``capfd`` has already captured the adapter
|
||||
print lines emitted at 6748-6749 and 6998-6999. The captured stream
|
||||
is drained at entry so the count reflects only this run's prints.
|
||||
"""
|
||||
mdx_path = _SAMPLES_DIR / mdx_file
|
||||
assert mdx_path.exists(), f"sample missing: {mdx_path}"
|
||||
# Drain any prior buffered capture so the post-run readouterr is
|
||||
# bounded to this pipeline invocation only.
|
||||
capfd.readouterr()
|
||||
try:
|
||||
pz2.run_phase_z2_mvp1(mdx_path, run_id=run_id)
|
||||
except (SystemExit, ValueError):
|
||||
# SystemExit = IMP-87 BLOCKED (line 8782); ValueError = pre-build
|
||||
# layout invariant (line 7578). Both fire AFTER the adapter print
|
||||
# sites, so captured stdout is authoritative for the count.
|
||||
pass
|
||||
captured = capfd.readouterr()
|
||||
return captured.out.count(_ADAPTER_LITERAL)
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
@pytest.mark.parametrize("mdx_file", _MDX_BATCH)
|
||||
def test_imp95_flag_on_adapter_needed_monotone_non_increasing(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
capfd: pytest.CaptureFixture[str],
|
||||
mdx_file: str,
|
||||
) -> None:
|
||||
"""Per-mdx-file Stage 2 A11 monotonicity guard.
|
||||
|
||||
Runs ``run_phase_z2_mvp1`` twice with ``PHASE_Z_B4_GATEKEEPER=1``
|
||||
pinned (gatekeeper trigger held constant), flipping
|
||||
``PHASE_Z_B4_V4_EVIDENCE`` OFF → ON between runs. Asserts treatment
|
||||
``adapter_needed_count`` ≤ baseline ``adapter_needed_count``.
|
||||
|
||||
Counts are read from captured stdout (see module docstring rationale
|
||||
for why Step 20 ``slide_status`` is not the read source under
|
||||
``PHASE_Z_B4_GATEKEEPER=1``).
|
||||
"""
|
||||
monkeypatch.delenv(MAPPER_FLAG, raising=False)
|
||||
monkeypatch.setenv(GATEKEEPER_FLAG, "1")
|
||||
runs_root = tmp_path / "runs"
|
||||
runs_root.mkdir(parents=True, exist_ok=True)
|
||||
monkeypatch.setattr(pz2, "RUNS_DIR", runs_root)
|
||||
|
||||
monkeypatch.delenv(V4_FLAG, raising=False)
|
||||
assert _b4_v4_evidence_enabled() is False
|
||||
baseline_count = _run_pipeline_count_adapter_signals(
|
||||
mdx_file, runs_root, f"imp95_u9_off_{Path(mdx_file).stem}", capfd
|
||||
)
|
||||
|
||||
monkeypatch.setenv(V4_FLAG, "1")
|
||||
assert _b4_v4_evidence_enabled() is True
|
||||
treatment_count = _run_pipeline_count_adapter_signals(
|
||||
mdx_file, runs_root, f"imp95_u9_on_{Path(mdx_file).stem}", capfd
|
||||
)
|
||||
|
||||
assert treatment_count <= baseline_count, (
|
||||
f"IMP-95 Stage 2 A11 monotonicity broken for {mdx_file}:\n"
|
||||
f" PHASE_Z_B4_V4_EVIDENCE OFF (baseline): {baseline_count}\n"
|
||||
f" PHASE_Z_B4_V4_EVIDENCE ON (treatment): {treatment_count}\n"
|
||||
f" (PHASE_Z_B4_GATEKEEPER=1 held constant; counts read from "
|
||||
f"captured stdout literal {_ADAPTER_LITERAL!r})\n"
|
||||
f" -> V4 evidence consumption *increased* adapter_needed. V4-aware "
|
||||
f"selector is picking contracts the mapper rejects — V4 evidence is "
|
||||
f"systematically worse than declaration order for this mdx file. "
|
||||
f"Inspect placement_trace[v4_evidence_consumed / v4_rank_used / "
|
||||
f"v4_b4_frame_match] in debug.json and the v4_candidates feeding "
|
||||
f"_select_frame_v4_aware."
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_imp95_flag_on_adapter_needed_holistic_sweep(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
capfd: pytest.CaptureFixture[str],
|
||||
) -> None:
|
||||
"""Aggregate Stage 2 A11 guard across mdx 01/02/04/05.
|
||||
|
||||
Guards against parametrize zero-iteration silent pass and against a
|
||||
case where individual files are equal but the holistic sum diverges
|
||||
(impossible given per-file monotonicity, but the explicit aggregate
|
||||
is the contract the exit report names).
|
||||
"""
|
||||
monkeypatch.delenv(MAPPER_FLAG, raising=False)
|
||||
monkeypatch.setenv(GATEKEEPER_FLAG, "1")
|
||||
runs_root = tmp_path / "runs"
|
||||
runs_root.mkdir(parents=True, exist_ok=True)
|
||||
monkeypatch.setattr(pz2, "RUNS_DIR", runs_root)
|
||||
|
||||
baseline_sum = 0
|
||||
treatment_sum = 0
|
||||
matched = 0
|
||||
for mdx_file in _MDX_BATCH:
|
||||
stem = Path(mdx_file).stem
|
||||
monkeypatch.delenv(V4_FLAG, raising=False)
|
||||
baseline_count = _run_pipeline_count_adapter_signals(
|
||||
mdx_file, runs_root, f"imp95_u9_sweep_off_{stem}", capfd
|
||||
)
|
||||
monkeypatch.setenv(V4_FLAG, "1")
|
||||
treatment_count = _run_pipeline_count_adapter_signals(
|
||||
mdx_file, runs_root, f"imp95_u9_sweep_on_{stem}", capfd
|
||||
)
|
||||
assert treatment_count <= baseline_count, (
|
||||
f"sweep monotonicity mismatch on {mdx_file}: "
|
||||
f"baseline={baseline_count} treatment={treatment_count}"
|
||||
)
|
||||
baseline_sum += baseline_count
|
||||
treatment_sum += treatment_count
|
||||
matched += 1
|
||||
|
||||
assert matched == len(_MDX_BATCH), (
|
||||
f"sweep covered only {matched}/{len(_MDX_BATCH)} mdx files — "
|
||||
f"coverage too shallow to guard A11 monotonicity."
|
||||
)
|
||||
assert treatment_sum <= baseline_sum, (
|
||||
f"aggregate adapter_needed_count rose under flag ON: "
|
||||
f"baseline_sum={baseline_sum} treatment_sum={treatment_sum} "
|
||||
f"(PHASE_Z_B4_GATEKEEPER=1 held constant)"
|
||||
)
|
||||
|
||||
|
||||
def test_imp95_u9_flag_independence_preconditions(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Lock the env precondition shape u9 depends on.
|
||||
|
||||
Both flags are read by independent readers
|
||||
(``_b4_v4_evidence_enabled`` and the inline gatekeeper check at
|
||||
``src/phase_z2_pipeline.py:6730``). u9 holds the gatekeeper ON and
|
||||
flips V4_EVIDENCE — the test below pins the truthy-value contract
|
||||
so a future change to either reader (e.g. accepting only ``'on'``)
|
||||
surfaces here instead of as a silent monotonicity false-pass.
|
||||
"""
|
||||
monkeypatch.delenv(V4_FLAG, raising=False)
|
||||
assert _b4_v4_evidence_enabled() is False
|
||||
monkeypatch.setenv(V4_FLAG, "1")
|
||||
assert _b4_v4_evidence_enabled() is True
|
||||
monkeypatch.setenv(V4_FLAG, "true")
|
||||
assert _b4_v4_evidence_enabled() is True
|
||||
monkeypatch.setenv(V4_FLAG, "yes")
|
||||
assert _b4_v4_evidence_enabled() is True
|
||||
monkeypatch.setenv(V4_FLAG, "0")
|
||||
assert _b4_v4_evidence_enabled() is False
|
||||
monkeypatch.setenv(V4_FLAG, "")
|
||||
assert _b4_v4_evidence_enabled() is False
|
||||
|
||||
# PHASE_Z_B4_GATEKEEPER is the inline reader at pipeline.py:6730 —
|
||||
# we cannot import a helper, so this test fixes the truthy values
|
||||
# u9 sets ("1") and the env-key constant so renaming surfaces here.
|
||||
monkeypatch.setenv(GATEKEEPER_FLAG, "1")
|
||||
assert os.environ.get(GATEKEEPER_FLAG, "").strip().lower() in {
|
||||
"1", "true", "yes",
|
||||
}
|
||||
|
||||
|
||||
def test_imp95_u9_mdx_batch_covers_stage2_scope() -> None:
|
||||
"""Guard the Stage 2 u9 scope literal (mdx 01/02/04/05).
|
||||
|
||||
If the literal silently shrinks (e.g. someone removes 04.mdx
|
||||
because it ``SystemExit``s) the sweep coverage assertion would
|
||||
still pass with a thinner batch and the monotonicity guard would
|
||||
weaken without notice. This test pins the exact tuple.
|
||||
"""
|
||||
assert _MDX_BATCH == ("01.mdx", "02.mdx", "04.mdx", "05.mdx"), (
|
||||
f"u9 mdx batch drifted from Stage 2 scope: {_MDX_BATCH}"
|
||||
)
|
||||
for mdx_file in _MDX_BATCH:
|
||||
assert (_SAMPLES_DIR / mdx_file).exists(), (
|
||||
f"u9 scope file missing on disk: {_SAMPLES_DIR / mdx_file}"
|
||||
)
|
||||
|
||||
|
||||
def test_imp95_u9_adapter_print_pattern_is_unique_and_anchored() -> None:
|
||||
"""Lock the captured-stdout read contract.
|
||||
|
||||
``_run_pipeline_count_adapter_signals`` reads
|
||||
``adapter_needed_count`` by counting occurrences of
|
||||
``_ADAPTER_LITERAL`` in captured stdout. The literal MUST be:
|
||||
|
||||
* Unique to the two adapter-needed print sites at
|
||||
``src/phase_z2_pipeline.py:6749`` (v4_b4_mismatch gatekeeper
|
||||
short-circuit) and ``:6999`` (IMP-86 u1 FitError fallback).
|
||||
* Present in src/ at exactly those two sites.
|
||||
|
||||
A future adapter_needed site that omits the trailing
|
||||
``(skip render)`` would silently shrink the count and make the
|
||||
monotonicity guard appear to pass on a real regression. This test
|
||||
pins the literal at its canonical anchor so any rename surfaces as
|
||||
a u9 failure with an obvious diagnostic instead of as a silent
|
||||
false pass in the parametrized assertions above.
|
||||
"""
|
||||
pipeline_src = (
|
||||
_REPO_ROOT / "src" / "phase_z2_pipeline.py"
|
||||
).read_text(encoding="utf-8")
|
||||
# Both print sites must include the literal verbatim.
|
||||
occurrences = pipeline_src.count(_ADAPTER_LITERAL)
|
||||
assert occurrences == 2, (
|
||||
f"expected exactly 2 occurrences of {_ADAPTER_LITERAL!r} in "
|
||||
f"src/phase_z2_pipeline.py (lines 6749 / 6999); got {occurrences}. "
|
||||
f"If a new adapter_needed site was added, update u9's stdout-based "
|
||||
f"counter contract OR ensure the new site also emits "
|
||||
f"{_ADAPTER_LITERAL!r}. If a site was removed, u9's contract is "
|
||||
f"unsound and must be re-keyed."
|
||||
)
|
||||
# Anchor each print-site's reason prefix so a future refactor that
|
||||
# renames "v4_b4_mismatch" / "FitError" while keeping the trailing
|
||||
# literal also surfaces here. The "→" U+2192 character is part of
|
||||
# the literal at both sites (lines 6749 / 6999).
|
||||
assert "v4_b4_mismatch → adapter_needed (skip render)" in pipeline_src, (
|
||||
"gatekeeper short-circuit print at src/phase_z2_pipeline.py:6749 "
|
||||
"no longer carries the 'v4_b4_mismatch → adapter_needed "
|
||||
"(skip render)' form — u9 stdout counter contract is broken."
|
||||
)
|
||||
assert "FitError → adapter_needed (skip render)" in pipeline_src, (
|
||||
"FitError fallback print at src/phase_z2_pipeline.py:6999 "
|
||||
"no longer carries the 'FitError → adapter_needed (skip render)' "
|
||||
"form — u9 stdout counter contract is broken."
|
||||
)
|
||||
@@ -0,0 +1,262 @@
|
||||
"""IMP-95 u8 — Flag-OFF final.html SHA parity guard for mdx 01/02/04/05.
|
||||
|
||||
Stage 2 u8 contract (verbatim)::
|
||||
|
||||
Flag-OFF final.html SHA parity regression for mdx 01/02/04/05.
|
||||
|
||||
Why this is a genuine pre-IMP-95 baseline guard under flag OFF
|
||||
==============================================================
|
||||
|
||||
Under ``PHASE_Z_B4_V4_EVIDENCE`` OFF (default) the planner branch in
|
||||
``plan_placement`` falls through to the legacy ``_select_frame``
|
||||
declaration-order path (u3 contract). The u4 Step 11 wiring still
|
||||
attaches ``v4_candidates`` to the call, but the legacy branch never
|
||||
inspects them. The u4 additive top-level keys land on
|
||||
``placement_trace`` (a ``debug_zones`` field) and the u5 ``v4_short_circuit``
|
||||
field is also confined to ``placement_trace`` / ``adapter_record`` — none
|
||||
of these flow into ``final.html``. The trace-only contract is anchored
|
||||
verbatim at ``src/phase_z2_pipeline.py:86``::
|
||||
|
||||
# trace-only runtime 연결 v0 — B1 → B4 chain.
|
||||
# final.html / mapper / render path 미영향. debug_zones[i].placement_trace 만 기록.
|
||||
|
||||
The u6 ``partial_exists`` callable is forwarded to ``_select_frame_v4_aware``
|
||||
ONLY inside the flag-ON branch (planner u6 contract), so it is unreachable
|
||||
under flag OFF and cannot perturb final.html.
|
||||
|
||||
Therefore, under flag OFF, IMP-95 (u1~u7) is a strict no-op for the
|
||||
``final.html`` bytes on disk. The pre-IMP-95 reference SHA is frozen at
|
||||
``tests/regression/fixtures/imp95_pre_baseline_sha.json`` (captured via
|
||||
``tests/regression/scripts/capture_imp95_pre_baseline.py`` with both
|
||||
``PHASE_Z_B4_V4_EVIDENCE`` and ``PHASE_Z_B4_MAPPER_SOURCE`` unset). The
|
||||
89-a baseline (``89a_pre_baseline_sha.json``) is NOT reused here because
|
||||
working-tree drift unrelated to IMP-95 (e.g. the Emergency P3/P4/P4b
|
||||
verbatim slot_payload builders accumulated between the 89-a baseline
|
||||
capture at HEAD ``6e9e3ee`` and the current flag-OFF pipeline state)
|
||||
already breaks 89-a SHA parity for mdx 02/04/05. Keying u8 to its own
|
||||
baseline isolates IMP-95's regression axis from that upstream drift —
|
||||
the 89-a baseline continues to guard its own axis via
|
||||
``test_b4_mapper_source_sha_parity.py``.
|
||||
|
||||
IMP-94 additive marker normalization
|
||||
====================================
|
||||
|
||||
IMP-94 stamps ``data-region-id`` and ``data-content-unit-id`` onto each
|
||||
family-partial root ``<div>`` after the ``_stamp_zone_html`` (IMP-56)
|
||||
call in ``render_slide``. The stamper emission shape is anchored at
|
||||
``src/region_marker_stamper.py:131-135``. The bytes are normalized by
|
||||
stripping the two attribute tokens (with their leading single space)
|
||||
before SHA-256 hashing — same shape used by the IMP-89 89-a u4 regression
|
||||
test and by the u8 capture script. The strip is disjoint from the
|
||||
IMP-96 (89-d) ``data-frame-slot-id`` axis by attribute name, so any
|
||||
drift in that axis surfaces here as a SHA mismatch.
|
||||
|
||||
mdx 03 exclusion
|
||||
================
|
||||
|
||||
Stage 2 plan u8 scopes the regression to ``mdx 01/02/04/05`` (per the
|
||||
binding plan). mdx 03 is under the user-locked ``mdx 03 정비`` axis and
|
||||
is intentionally excluded.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
import src.phase_z2_pipeline as pz2
|
||||
from src.phase_z2_placement_planner import (
|
||||
PHASE_Z_B4_V4_EVIDENCE_ENV,
|
||||
_b4_v4_evidence_enabled,
|
||||
)
|
||||
|
||||
V4_FLAG = PHASE_Z_B4_V4_EVIDENCE_ENV
|
||||
MAPPER_FLAG = "PHASE_Z_B4_MAPPER_SOURCE"
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
_SAMPLES_DIR = _REPO_ROOT / "samples" / "mdx_batch"
|
||||
_MDX_BATCH = ("01.mdx", "02.mdx", "04.mdx", "05.mdx") # Stage 2 u8 scope
|
||||
_BASELINE_PATH = (
|
||||
Path(__file__).parent / "fixtures" / "imp95_pre_baseline_sha.json"
|
||||
)
|
||||
|
||||
# IMP-94 additive marker strip patterns (byte-level, leading single space
|
||||
# matches the exact emission shape at src/region_marker_stamper.py:131-135).
|
||||
# Disjoint from the IMP-96 data-frame-slot-id axis by attribute name.
|
||||
_STRIP_REGION_ID_RE = re.compile(rb' data-region-id="[^"]*"')
|
||||
_STRIP_CONTENT_UNIT_ID_RE = re.compile(rb' data-content-unit-id="[^"]*"')
|
||||
|
||||
|
||||
def _strip_imp94_markers(raw_bytes: bytes) -> bytes:
|
||||
"""Return ``raw_bytes`` with IMP-94 additive marker tokens removed.
|
||||
|
||||
Same normalization shape as
|
||||
``tests/regression/test_b4_mapper_source_sha_parity.py:_strip_imp94_markers``
|
||||
so the SHA comparison is anchored against the identical pre-stamper
|
||||
byte form.
|
||||
"""
|
||||
stripped = _STRIP_REGION_ID_RE.sub(b"", raw_bytes)
|
||||
stripped = _STRIP_CONTENT_UNIT_ID_RE.sub(b"", stripped)
|
||||
return stripped
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def baseline() -> dict:
|
||||
"""Load the frozen pre-IMP-95 final.html SHA baseline."""
|
||||
assert _BASELINE_PATH.exists(), (
|
||||
f"baseline fixture missing: {_BASELINE_PATH}\n"
|
||||
f"regenerate with: python tests/regression/scripts/"
|
||||
f"capture_imp95_pre_baseline.py"
|
||||
)
|
||||
with _BASELINE_PATH.open("r", encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
assert data.get("schema_version") == 1, (
|
||||
f"unexpected baseline schema_version: {data.get('schema_version')} "
|
||||
f"(expected 1 for imp95_pre_baseline_sha.json)"
|
||||
)
|
||||
return data
|
||||
|
||||
|
||||
def _live_final_html_sha_under_flag_off(
|
||||
mdx_file: str, runs_root: Path
|
||||
) -> tuple[str, int, int | None]:
|
||||
"""Run the full pipeline once under flag OFF and SHA the on-disk final.html.
|
||||
|
||||
Returns ``(sha256_hex, normalized_size_bytes, pipeline_exit_code)``.
|
||||
``SystemExit`` is caught to mirror the 89-a regression test: the
|
||||
IMP-87 ``EMPTY_SHELL_NO_CONTENT BLOCKED`` exit on mdx 05 fires AFTER
|
||||
the final.html write site so the artifact still exists on disk.
|
||||
"""
|
||||
mdx_path = _SAMPLES_DIR / mdx_file
|
||||
assert mdx_path.exists(), f"sample missing: {mdx_path}"
|
||||
|
||||
run_id = f"imp95_u8_live_{mdx_path.stem}"
|
||||
pipeline_exit_code: int | None = None
|
||||
try:
|
||||
pz2.run_phase_z2_mvp1(mdx_path, run_id=run_id)
|
||||
except SystemExit as exc:
|
||||
pipeline_exit_code = (
|
||||
int(exc.code) if isinstance(exc.code, int) else 1
|
||||
)
|
||||
|
||||
final_html_path = runs_root / run_id / "phase_z2" / "final.html"
|
||||
assert final_html_path.exists(), (
|
||||
f"final.html not written by pipeline: {final_html_path} "
|
||||
f"(pipeline_exit_code={pipeline_exit_code})"
|
||||
)
|
||||
raw_bytes = final_html_path.read_bytes()
|
||||
normalized_bytes = _strip_imp94_markers(raw_bytes)
|
||||
return (
|
||||
hashlib.sha256(normalized_bytes).hexdigest(),
|
||||
len(normalized_bytes),
|
||||
pipeline_exit_code,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
@pytest.mark.parametrize("mdx_file", _MDX_BATCH)
|
||||
def test_imp95_flag_off_final_html_sha_matches_pre_imp95_baseline(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
mdx_file: str,
|
||||
baseline: dict,
|
||||
) -> None:
|
||||
"""Per-mdx-file final.html SHA parity guard under PHASE_Z_B4_V4_EVIDENCE OFF.
|
||||
|
||||
Both ``PHASE_Z_B4_V4_EVIDENCE`` (IMP-95) and ``PHASE_Z_B4_MAPPER_SOURCE``
|
||||
(IMP-89 89-a) are explicitly unset so the live pipeline exercises the
|
||||
pre-IMP-95 / pre-89-a code path that produced the frozen baseline.
|
||||
Any drift in the planner flag-OFF branch, the legacy ``_select_frame``,
|
||||
Step 11 wiring, gatekeeper telemetry, mapper, render_slide,
|
||||
slide_base.html, or family partials surfaces as a SHA mismatch here.
|
||||
"""
|
||||
monkeypatch.delenv(V4_FLAG, raising=False)
|
||||
monkeypatch.delenv(MAPPER_FLAG, raising=False)
|
||||
assert _b4_v4_evidence_enabled() is False
|
||||
|
||||
runs_root = tmp_path / "runs"
|
||||
runs_root.mkdir(parents=True, exist_ok=True)
|
||||
monkeypatch.setattr(pz2, "RUNS_DIR", runs_root)
|
||||
|
||||
expected = baseline["mdx_files"][mdx_file]
|
||||
expected_sha = expected["sha256"]
|
||||
expected_size = expected["final_html_size_bytes"]
|
||||
expected_exit = expected.get("pipeline_exit_code")
|
||||
|
||||
live_sha, live_size, live_exit = _live_final_html_sha_under_flag_off(
|
||||
mdx_file, runs_root
|
||||
)
|
||||
|
||||
assert live_sha == expected_sha, (
|
||||
f"IMP-95 flag-OFF final.html SHA parity broken for {mdx_file}:\n"
|
||||
f" pre-IMP-95 baseline: {expected_sha} ({expected_size} bytes)\n"
|
||||
f" live IMP-95 flag-OFF: {live_sha} ({live_size} bytes)\n"
|
||||
f" -> flag-OFF code path drifted vs declaration-order legacy. "
|
||||
f"Either u3/u4/u5/u6 leaked into the flag-OFF render path, an "
|
||||
f"upstream selector/mapper/render_slide/slide_base/partial drift "
|
||||
f"occurred, or the additive IMP-94 marker normalization is no "
|
||||
f"longer disjoint from a newly-stamped attribute. If the delta "
|
||||
f"is intentional, regenerate via "
|
||||
f"tests/regression/scripts/capture_imp95_pre_baseline.py."
|
||||
)
|
||||
assert live_exit == expected_exit, (
|
||||
f"pipeline exit-code parity broken for {mdx_file}: "
|
||||
f"baseline={expected_exit} live={live_exit} — SHA matches but "
|
||||
f"the pipeline terminal state diverged (IMP-87 BLOCKED axis)."
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_imp95_flag_off_final_html_sha_holistic_sweep(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
baseline: dict,
|
||||
) -> None:
|
||||
"""Single-pass sweep guarding against a parametrize zero-iteration silent pass."""
|
||||
monkeypatch.delenv(V4_FLAG, raising=False)
|
||||
monkeypatch.delenv(MAPPER_FLAG, raising=False)
|
||||
assert _b4_v4_evidence_enabled() is False
|
||||
|
||||
runs_root = tmp_path / "runs"
|
||||
runs_root.mkdir(parents=True, exist_ok=True)
|
||||
monkeypatch.setattr(pz2, "RUNS_DIR", runs_root)
|
||||
|
||||
matched = 0
|
||||
for mdx_file in _MDX_BATCH:
|
||||
expected = baseline["mdx_files"][mdx_file]
|
||||
live_sha, live_size, _live_exit = _live_final_html_sha_under_flag_off(
|
||||
mdx_file, runs_root
|
||||
)
|
||||
assert live_sha == expected["sha256"], (
|
||||
f"sweep mismatch on {mdx_file}: baseline {expected['sha256']} "
|
||||
f"≠ live {live_sha} (size baseline={expected['final_html_size_bytes']} "
|
||||
f"live={live_size})"
|
||||
)
|
||||
matched += 1
|
||||
|
||||
assert matched == len(_MDX_BATCH), (
|
||||
f"sweep covered only {matched}/{len(_MDX_BATCH)} mdx files — "
|
||||
f"coverage too shallow to guard final.html SHA parity."
|
||||
)
|
||||
|
||||
|
||||
def test_imp95_flag_off_default_state_locks_gate_to_false() -> None:
|
||||
"""Default-OFF contract precondition for SHA parity."""
|
||||
assert os.environ.get(V4_FLAG, "") == ""
|
||||
assert _b4_v4_evidence_enabled() is False
|
||||
|
||||
|
||||
def test_imp95_u8_baseline_covers_all_scoped_mdx_files(baseline: dict) -> None:
|
||||
"""Guard against silent zero-iteration: every Stage 2 u8 mdx must be
|
||||
present in the baseline fixture."""
|
||||
baseline_files = set(baseline["mdx_files"].keys())
|
||||
missing = set(_MDX_BATCH) - baseline_files
|
||||
assert not missing, (
|
||||
f"baseline missing required mdx coverage for u8: {missing}. "
|
||||
f"Baseline contains {baseline_files}; regenerate via "
|
||||
f"tests/regression/scripts/capture_imp95_pre_baseline.py."
|
||||
)
|
||||
@@ -0,0 +1,474 @@
|
||||
"""IMP-95 u10 — Flag-ON placement_trace field presence/type regression.
|
||||
|
||||
Stage 2 u10 contract (verbatim)::
|
||||
|
||||
Flag-ON trace field presence/type regression on mdx 01/02/04/05.
|
||||
|
||||
Stage 2 axis A12 binding invariant
|
||||
==================================
|
||||
|
||||
When ``PHASE_Z_B4_V4_EVIDENCE`` is ON, every renderable / IMP-86 u1
|
||||
``adapter_needed`` zone in ``debug.json`` MUST carry the six u1
|
||||
``TRACE_KEY_*`` fields plus the u5 ``v4_short_circuit`` field at the
|
||||
top level of ``placement_trace`` with the expected types and (for the
|
||||
``frame_selection_basis`` enum) the expected value set. The fields are
|
||||
hoisted at ``src/phase_z2_pipeline.py:6685-6706`` (u4 hoist) and
|
||||
``:6726`` (u5 ``v4_short_circuit``); ``debug.json`` persists them via
|
||||
``write_debug_json`` at ``src/phase_z2_pipeline.py:5076-5114``.
|
||||
|
||||
Why this is a live-pipeline regression, not a unit test
|
||||
=======================================================
|
||||
|
||||
The u4 ``_replay_trace_assembly`` mirror in
|
||||
``tests/phase_z2/test_phase_z2_b4_v4_evidence_pipeline_trace.py`` only
|
||||
verifies the assembly shape for fixture-built ``PlacementPlan``
|
||||
instances. A4 / A5 / A6 wiring drift downstream — Step 11 silently
|
||||
omitting a hoisted key, ``write_debug_json`` losing ``zones``, the
|
||||
u5 ``v4_short_circuit`` derivation skipping the adapter record path —
|
||||
is undetectable at that fixture layer. Running the full
|
||||
``run_phase_z2_mvp1`` pipeline and reading the hoisted fields off
|
||||
``debug.json`` exercises the entire chain (planner → Step 11 → Step 21
|
||||
``write_debug_json``) and catches any drift in the persisted shape.
|
||||
|
||||
Read source: ``run_dir/debug.json``
|
||||
===================================
|
||||
|
||||
``debug.json`` is the authoritative on-disk artifact for
|
||||
``placement_trace`` (anchored at ``src/phase_z2_pipeline.py:86``::
|
||||
|
||||
# final.html / mapper / render path 미영향. debug_zones[i].placement_trace 만 기록.
|
||||
|
||||
The write site at ``src/phase_z2_pipeline.py:8674`` runs BEFORE the
|
||||
IMP-87 ``EMPTY_SHELL_NO_CONTENT BLOCKED`` ``sys.exit(1)`` at line
|
||||
8782, so the artifact exists on disk even when the pipeline terminates
|
||||
with ``SystemExit`` on mdx 05. Under flag ON without
|
||||
``PHASE_Z_B4_GATEKEEPER``, zones never get diverted en masse to the
|
||||
gatekeeper short-circuit, so the pre-build layout invariant
|
||||
``ValueError`` at line 7572-7588 (which fires BEFORE Step 21 and
|
||||
blocked u9's Step 20 read) cannot trip here.
|
||||
|
||||
Zone-shape tolerance
|
||||
====================
|
||||
|
||||
``debug_zones`` carries five shapes (Step 11 control flow). Three carry
|
||||
the hoisted trace surface and are the unit-under-test for u10; two
|
||||
structurally lack ``placement_trace`` because their code path never
|
||||
calls (or never persists) ``plan_placement``, and u10 SKIPS them:
|
||||
|
||||
Shapes WITH ``placement_trace`` (u10 verifies):
|
||||
* Renderable path (``src/phase_z2_pipeline.py:7092``).
|
||||
* IMP-86 u1 ``adapter_needed`` FitError path
|
||||
(``src/phase_z2_pipeline.py:6987``).
|
||||
|
||||
Shapes WITHOUT ``placement_trace`` (u10 skips):
|
||||
* IMP-30 u4 empty-shell path — ``placement_trace: None`` literal at
|
||||
``src/phase_z2_pipeline.py:6606``. ``plan_placement`` is never
|
||||
called for these units (the ``continue`` at line 6615 short-
|
||||
circuits Step 11 before the planner).
|
||||
* Emergency P4b verbatim recovery path at
|
||||
``src/phase_z2_pipeline.py:6853-6863``. ``plan_placement`` IS
|
||||
called upstream, but the debug_zone appended by this branch has
|
||||
no ``placement_trace`` key (the local trace is dropped on the
|
||||
verbatim recovery fork). Mdx 05 routes through this path under
|
||||
the current Emergency P3 ``no_rank_1_V4_evidence_in_any_section``
|
||||
pipeline state — every zone surfaces as a verbatim-built shape
|
||||
with no trace. This is a real structural gap (see
|
||||
``follow_up_issue_candidates``) but outside u10's scope-lock:
|
||||
Stage 2 A12 is a trace-field-shape regression on the hoisted
|
||||
surface, not an audit of whether every code path persists the
|
||||
trace.
|
||||
* Emergency P4 inline-AI recovery path
|
||||
(``src/phase_z2_pipeline.py:6900-6911``) — same shape as P4b.
|
||||
|
||||
The per-mdx test therefore tolerates an empty trace surface on a given
|
||||
mdx file (every zone routed to one of the no-trace shapes above). The
|
||||
**aggregate** sweep is the load-bearing guard against silent regression:
|
||||
it asserts that across mdx 01/02/04/05 at least one zone actually
|
||||
consumed V4 evidence (``v4_evidence_consumed=True``). If a future
|
||||
change shifts every renderable / IMP-86 u1 zone in the entire batch
|
||||
into a no-trace shape, the aggregate would fail with "no zone reported
|
||||
v4_evidence_consumed=True" — the only failure mode that catches "flag
|
||||
ON silently behaves like flag OFF" across the whole batch.
|
||||
|
||||
mdx 03 exclusion + Stage 2 u10 scope
|
||||
====================================
|
||||
|
||||
Stage 2 u10 scopes the regression to ``mdx 01/02/04/05`` (mdx 03 is
|
||||
under the user-locked ``mdx 03 정비`` axis).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
import pytest
|
||||
|
||||
import src.phase_z2_pipeline as pz2
|
||||
from src.phase_z2_placement_planner import (
|
||||
FRAME_SELECTION_BASIS_DECLARATION_ORDER,
|
||||
FRAME_SELECTION_BASIS_V4_RANKED,
|
||||
PHASE_Z_B4_V4_EVIDENCE_ENV,
|
||||
TRACE_KEY_B4_PARTIAL_MISSING_SKIP,
|
||||
TRACE_KEY_B4_V0_FALLBACK_REASON,
|
||||
TRACE_KEY_FRAME_SELECTION_BASIS,
|
||||
TRACE_KEY_V4_B4_FRAME_MATCH,
|
||||
TRACE_KEY_V4_EVIDENCE_CONSUMED,
|
||||
TRACE_KEY_V4_RANK_USED,
|
||||
_b4_v4_evidence_enabled,
|
||||
)
|
||||
|
||||
V4_FLAG = PHASE_Z_B4_V4_EVIDENCE_ENV
|
||||
GATEKEEPER_FLAG = "PHASE_Z_B4_GATEKEEPER"
|
||||
MAPPER_FLAG = "PHASE_Z_B4_MAPPER_SOURCE"
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
_SAMPLES_DIR = _REPO_ROOT / "samples" / "mdx_batch"
|
||||
_MDX_BATCH = ("01.mdx", "02.mdx", "04.mdx", "05.mdx") # Stage 2 u10 scope
|
||||
|
||||
# u5 telemetry field (derived at src/phase_z2_pipeline.py:6722-6726). It
|
||||
# is hoisted onto every ``placement_trace`` carried by a renderable /
|
||||
# IMP-86 u1 zone, not just on adapter records.
|
||||
_TRACE_KEY_V4_SHORT_CIRCUIT = "v4_short_circuit"
|
||||
|
||||
# Frozen u1 enum domain for ``frame_selection_basis``. A value outside
|
||||
# this set means a new basis was introduced without updating u10 (e.g.
|
||||
# a ``v4_evidence_pinned`` follow-up axis); the test fails loudly with
|
||||
# the offending value so the regression surface is updated in tandem.
|
||||
_BASIS_DOMAIN = frozenset(
|
||||
{FRAME_SELECTION_BASIS_DECLARATION_ORDER, FRAME_SELECTION_BASIS_V4_RANKED}
|
||||
)
|
||||
|
||||
# Frozen ``b4_v0_fallback_reason`` enum domain (planner u2/u3 outputs):
|
||||
# * ``v4_evidence_empty`` — ``v4_candidates`` was empty / None.
|
||||
# * ``no_v4_rank_eligible`` — no V4 rank satisfied the
|
||||
# accepted_content_types ⊇ constraint.
|
||||
# Both are str. ``None`` is also valid (set on the V4-ranked basis where
|
||||
# no fallback was needed).
|
||||
_FALLBACK_REASON_DOMAIN = frozenset({"v4_evidence_empty", "no_v4_rank_eligible"})
|
||||
|
||||
|
||||
def _run_pipeline_and_read_debug_json(
|
||||
mdx_file: str, runs_root: Path, run_id: str
|
||||
) -> dict:
|
||||
"""Run ``run_phase_z2_mvp1`` once under flag ON and read ``debug.json``.
|
||||
|
||||
Catches the IMP-87 ``EMPTY_SHELL_NO_CONTENT BLOCKED`` ``SystemExit``
|
||||
(line 8782) since the ``write_debug_json`` site at line 8674 fires
|
||||
BEFORE it; the on-disk artifact is always available for the read.
|
||||
``ValueError`` (pre-build layout invariant at line 7572-7588) is NOT
|
||||
caught — under flag ON without ``PHASE_Z_B4_GATEKEEPER`` it cannot
|
||||
fire (no en-masse zone divert), so its appearance here would be a
|
||||
genuine regression worth surfacing as an unhandled exception.
|
||||
"""
|
||||
mdx_path = _SAMPLES_DIR / mdx_file
|
||||
assert mdx_path.exists(), f"sample missing: {mdx_path}"
|
||||
try:
|
||||
pz2.run_phase_z2_mvp1(mdx_path, run_id=run_id)
|
||||
except SystemExit:
|
||||
# IMP-87 BLOCKED exit at src/phase_z2_pipeline.py:8782 fires
|
||||
# AFTER write_debug_json at line 8674, so debug.json is on disk.
|
||||
pass
|
||||
debug_path = runs_root / run_id / "phase_z2" / "debug.json"
|
||||
assert debug_path.exists(), (
|
||||
f"debug.json not written by pipeline: {debug_path}"
|
||||
)
|
||||
return json.loads(debug_path.read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
def _assert_placement_trace_shape(
|
||||
placement_trace: dict, zone_index: int, mdx_file: str
|
||||
) -> None:
|
||||
"""Assert the six u1 + one u5 hoisted fields on a single placement_trace.
|
||||
|
||||
Field-by-field type and (for ``frame_selection_basis`` /
|
||||
``b4_v0_fallback_reason``) enum checks. The diagnostic message names
|
||||
the mdx file + zone index + offending value so a failure is
|
||||
actionable without re-running.
|
||||
"""
|
||||
ctx = f"[{mdx_file} zone#{zone_index}]"
|
||||
|
||||
# ── frame_selection_basis — str within fixed enum domain ──
|
||||
assert TRACE_KEY_FRAME_SELECTION_BASIS in placement_trace, (
|
||||
f"{ctx} missing top-level key '{TRACE_KEY_FRAME_SELECTION_BASIS}' — "
|
||||
f"u4 hoist at src/phase_z2_pipeline.py:6690-6691 dropped."
|
||||
)
|
||||
basis = placement_trace[TRACE_KEY_FRAME_SELECTION_BASIS]
|
||||
assert isinstance(basis, str), (
|
||||
f"{ctx} {TRACE_KEY_FRAME_SELECTION_BASIS} type={type(basis).__name__} "
|
||||
f"value={basis!r}, expected str."
|
||||
)
|
||||
assert basis in _BASIS_DOMAIN, (
|
||||
f"{ctx} {TRACE_KEY_FRAME_SELECTION_BASIS}={basis!r} outside u1 enum "
|
||||
f"domain {_BASIS_DOMAIN}. If a new basis was added, update _BASIS_DOMAIN."
|
||||
)
|
||||
|
||||
# ── v4_evidence_consumed — bool ──
|
||||
assert TRACE_KEY_V4_EVIDENCE_CONSUMED in placement_trace, (
|
||||
f"{ctx} missing top-level key '{TRACE_KEY_V4_EVIDENCE_CONSUMED}' — "
|
||||
f"u4 hoist at src/phase_z2_pipeline.py:6693-6695 dropped."
|
||||
)
|
||||
consumed = placement_trace[TRACE_KEY_V4_EVIDENCE_CONSUMED]
|
||||
assert isinstance(consumed, bool), (
|
||||
f"{ctx} {TRACE_KEY_V4_EVIDENCE_CONSUMED} type={type(consumed).__name__} "
|
||||
f"value={consumed!r}, expected bool."
|
||||
)
|
||||
|
||||
# ── v4_rank_used — int or None (0-based rank index) ──
|
||||
assert TRACE_KEY_V4_RANK_USED in placement_trace, (
|
||||
f"{ctx} missing top-level key '{TRACE_KEY_V4_RANK_USED}' — "
|
||||
f"u4 hoist at src/phase_z2_pipeline.py:6696 dropped."
|
||||
)
|
||||
rank_used = placement_trace[TRACE_KEY_V4_RANK_USED]
|
||||
# ``bool`` is a subclass of ``int`` in Python — reject it explicitly so
|
||||
# a future planner edit that returns True/False does not pass as int.
|
||||
assert rank_used is None or (
|
||||
isinstance(rank_used, int) and not isinstance(rank_used, bool)
|
||||
), (
|
||||
f"{ctx} {TRACE_KEY_V4_RANK_USED} type={type(rank_used).__name__} "
|
||||
f"value={rank_used!r}, expected int|None."
|
||||
)
|
||||
if isinstance(rank_used, int):
|
||||
assert rank_used >= 0, (
|
||||
f"{ctx} {TRACE_KEY_V4_RANK_USED}={rank_used} negative; "
|
||||
f"planner contract is 0-based rank index."
|
||||
)
|
||||
|
||||
# ── v4_b4_frame_match — bool ──
|
||||
assert TRACE_KEY_V4_B4_FRAME_MATCH in placement_trace, (
|
||||
f"{ctx} missing top-level key '{TRACE_KEY_V4_B4_FRAME_MATCH}' — "
|
||||
f"u4 hoist at src/phase_z2_pipeline.py:6697-6699 dropped."
|
||||
)
|
||||
frame_match = placement_trace[TRACE_KEY_V4_B4_FRAME_MATCH]
|
||||
assert isinstance(frame_match, bool), (
|
||||
f"{ctx} {TRACE_KEY_V4_B4_FRAME_MATCH} type={type(frame_match).__name__} "
|
||||
f"value={frame_match!r}, expected bool."
|
||||
)
|
||||
|
||||
# ── b4_v0_fallback_reason — str (in fallback enum) or None ──
|
||||
assert TRACE_KEY_B4_V0_FALLBACK_REASON in placement_trace, (
|
||||
f"{ctx} missing top-level key '{TRACE_KEY_B4_V0_FALLBACK_REASON}' — "
|
||||
f"u4 hoist at src/phase_z2_pipeline.py:6700-6702 dropped."
|
||||
)
|
||||
fallback_reason = placement_trace[TRACE_KEY_B4_V0_FALLBACK_REASON]
|
||||
assert fallback_reason is None or isinstance(fallback_reason, str), (
|
||||
f"{ctx} {TRACE_KEY_B4_V0_FALLBACK_REASON} "
|
||||
f"type={type(fallback_reason).__name__} value={fallback_reason!r}, "
|
||||
f"expected str|None."
|
||||
)
|
||||
if isinstance(fallback_reason, str):
|
||||
assert fallback_reason in _FALLBACK_REASON_DOMAIN, (
|
||||
f"{ctx} {TRACE_KEY_B4_V0_FALLBACK_REASON}={fallback_reason!r} "
|
||||
f"outside planner enum {_FALLBACK_REASON_DOMAIN}. If a new "
|
||||
f"fallback reason was added, update _FALLBACK_REASON_DOMAIN."
|
||||
)
|
||||
|
||||
# ── b4_partial_missing_skip — list (u6) ──
|
||||
assert TRACE_KEY_B4_PARTIAL_MISSING_SKIP in placement_trace, (
|
||||
f"{ctx} missing top-level key '{TRACE_KEY_B4_PARTIAL_MISSING_SKIP}' — "
|
||||
f"u4 hoist at src/phase_z2_pipeline.py:6703-6705 dropped."
|
||||
)
|
||||
skip = placement_trace[TRACE_KEY_B4_PARTIAL_MISSING_SKIP]
|
||||
assert isinstance(skip, list), (
|
||||
f"{ctx} {TRACE_KEY_B4_PARTIAL_MISSING_SKIP} type={type(skip).__name__} "
|
||||
f"value={skip!r}, expected list."
|
||||
)
|
||||
|
||||
# ── v4_short_circuit — bool (u5 derived telemetry) ──
|
||||
assert _TRACE_KEY_V4_SHORT_CIRCUIT in placement_trace, (
|
||||
f"{ctx} missing top-level key '{_TRACE_KEY_V4_SHORT_CIRCUIT}' — "
|
||||
f"u5 derivation at src/phase_z2_pipeline.py:6722-6726 dropped."
|
||||
)
|
||||
short_circuit = placement_trace[_TRACE_KEY_V4_SHORT_CIRCUIT]
|
||||
assert isinstance(short_circuit, bool), (
|
||||
f"{ctx} {_TRACE_KEY_V4_SHORT_CIRCUIT} "
|
||||
f"type={type(short_circuit).__name__} value={short_circuit!r}, "
|
||||
f"expected bool."
|
||||
)
|
||||
|
||||
|
||||
def _iter_trace_zones(debug: dict) -> list[tuple[int, dict]]:
|
||||
"""Return ``[(zone_index, placement_trace), ...]`` for non-None traces.
|
||||
|
||||
IMP-30 u4 empty-shell zones (``placement_trace: None``) are filtered
|
||||
out — they short-circuit Step 11 before ``plan_placement`` runs and
|
||||
structurally cannot carry the hoisted shape (see module docstring
|
||||
"Zone-shape tolerance").
|
||||
"""
|
||||
zones = debug.get("zones") or []
|
||||
out: list[tuple[int, dict]] = []
|
||||
for idx, zone in enumerate(zones):
|
||||
trace = zone.get("placement_trace")
|
||||
if trace is not None:
|
||||
out.append((idx, trace))
|
||||
return out
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
@pytest.mark.parametrize("mdx_file", _MDX_BATCH)
|
||||
def test_imp95_u10_flag_on_trace_fields_present_and_typed(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
mdx_file: str,
|
||||
) -> None:
|
||||
"""Per-mdx-file Stage 2 A12 trace field presence/type guard.
|
||||
|
||||
Runs the pipeline once under ``PHASE_Z_B4_V4_EVIDENCE=1`` (no
|
||||
gatekeeper — see module docstring for why) and asserts every
|
||||
non-None ``placement_trace`` carries the six u1 + one u5 hoisted
|
||||
fields with the expected types and enum values.
|
||||
"""
|
||||
monkeypatch.delenv(MAPPER_FLAG, raising=False)
|
||||
monkeypatch.delenv(GATEKEEPER_FLAG, raising=False)
|
||||
monkeypatch.setenv(V4_FLAG, "1")
|
||||
assert _b4_v4_evidence_enabled() is True
|
||||
|
||||
runs_root = tmp_path / "runs"
|
||||
runs_root.mkdir(parents=True, exist_ok=True)
|
||||
monkeypatch.setattr(pz2, "RUNS_DIR", runs_root)
|
||||
|
||||
run_id = f"imp95_u10_on_{Path(mdx_file).stem}"
|
||||
debug = _run_pipeline_and_read_debug_json(mdx_file, runs_root, run_id)
|
||||
|
||||
trace_zones = _iter_trace_zones(debug)
|
||||
# Per Stage 2 scope-lock: u10 is a TRACE-FIELD presence/type guard on
|
||||
# the hoisted surface. If every zone in this mdx file routed to a
|
||||
# no-trace shape (IMP-30 u4 empty_shell, Emergency P4b verbatim, or
|
||||
# Emergency P4 inline-AI — see module docstring "Zone-shape
|
||||
# tolerance"), the per-mdx surface is empty for legitimate upstream
|
||||
# reasons and the field-shape contract has nothing to assert here.
|
||||
# The aggregate sweep in
|
||||
# ``test_imp95_u10_flag_on_v4_evidence_consumed_at_least_once`` is
|
||||
# the load-bearing guard against the "flag ON silently behaves like
|
||||
# flag OFF across the whole batch" silent-regression class.
|
||||
for zone_idx, placement_trace in trace_zones:
|
||||
_assert_placement_trace_shape(placement_trace, zone_idx, mdx_file)
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_imp95_u10_flag_on_v4_evidence_consumed_at_least_once(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""Aggregate sanity guard: across mdx 01/02/04/05 under flag ON, at
|
||||
least one zone MUST have ``v4_evidence_consumed=True``.
|
||||
|
||||
If every zone reports ``v4_evidence_consumed=False`` then either
|
||||
(a) ``unit.v4_candidates`` is empty everywhere (composition stage
|
||||
drift), (b) the Step 11 wiring at line 6650 is broken, or (c) the
|
||||
planner selector is silently returning declaration-order on every
|
||||
rank. Per-mdx parametrized tests would all pass with default types
|
||||
in that state, so this aggregate is the only place that catches the
|
||||
"flag ON behaves like flag OFF" silent-regression class.
|
||||
"""
|
||||
monkeypatch.delenv(MAPPER_FLAG, raising=False)
|
||||
monkeypatch.delenv(GATEKEEPER_FLAG, raising=False)
|
||||
monkeypatch.setenv(V4_FLAG, "1")
|
||||
assert _b4_v4_evidence_enabled() is True
|
||||
|
||||
runs_root = tmp_path / "runs"
|
||||
runs_root.mkdir(parents=True, exist_ok=True)
|
||||
monkeypatch.setattr(pz2, "RUNS_DIR", runs_root)
|
||||
|
||||
consumed_any = False
|
||||
covered: list[str] = []
|
||||
first_consumed: Optional[tuple[str, int]] = None
|
||||
for mdx_file in _MDX_BATCH:
|
||||
run_id = f"imp95_u10_sweep_{Path(mdx_file).stem}"
|
||||
debug = _run_pipeline_and_read_debug_json(mdx_file, runs_root, run_id)
|
||||
covered.append(mdx_file)
|
||||
for zone_idx, trace in _iter_trace_zones(debug):
|
||||
if trace.get(TRACE_KEY_V4_EVIDENCE_CONSUMED) is True:
|
||||
consumed_any = True
|
||||
if first_consumed is None:
|
||||
first_consumed = (mdx_file, zone_idx)
|
||||
# Cross-field invariant: when v4_evidence_consumed is True,
|
||||
# the basis MUST be 'v4_ranked', v4_rank_used MUST be a
|
||||
# non-None int, and v4_b4_frame_match MUST be True. These
|
||||
# are the four fields the planner sets together at
|
||||
# src/phase_z2_placement_planner.py:265-271 — any drift
|
||||
# breaks the contract that downstream gatekeeper /
|
||||
# adapter consumers depend on.
|
||||
assert (
|
||||
trace[TRACE_KEY_FRAME_SELECTION_BASIS]
|
||||
== FRAME_SELECTION_BASIS_V4_RANKED
|
||||
), (
|
||||
f"[{mdx_file} zone#{zone_idx}] v4_evidence_consumed=True "
|
||||
f"but frame_selection_basis="
|
||||
f"{trace[TRACE_KEY_FRAME_SELECTION_BASIS]!r}; planner "
|
||||
f"contract at src/phase_z2_placement_planner.py:265-271 "
|
||||
f"requires 'v4_ranked' when consumed."
|
||||
)
|
||||
assert trace[TRACE_KEY_V4_RANK_USED] is not None and isinstance(
|
||||
trace[TRACE_KEY_V4_RANK_USED], int
|
||||
), (
|
||||
f"[{mdx_file} zone#{zone_idx}] v4_evidence_consumed=True "
|
||||
f"but v4_rank_used={trace[TRACE_KEY_V4_RANK_USED]!r}; "
|
||||
f"planner must populate a 0-based int rank."
|
||||
)
|
||||
assert trace[TRACE_KEY_V4_B4_FRAME_MATCH] is True, (
|
||||
f"[{mdx_file} zone#{zone_idx}] v4_evidence_consumed=True "
|
||||
f"but v4_b4_frame_match=False; planner sets both True "
|
||||
f"together when V4 rank resolved."
|
||||
)
|
||||
assert trace[TRACE_KEY_B4_V0_FALLBACK_REASON] is None, (
|
||||
f"[{mdx_file} zone#{zone_idx}] v4_evidence_consumed=True "
|
||||
f"but b4_v0_fallback_reason="
|
||||
f"{trace[TRACE_KEY_B4_V0_FALLBACK_REASON]!r}; planner "
|
||||
f"clears the fallback reason on the V4-ranked branch."
|
||||
)
|
||||
|
||||
assert covered == list(_MDX_BATCH), (
|
||||
f"u10 sweep coverage drift: covered={covered}, expected={_MDX_BATCH}."
|
||||
)
|
||||
assert consumed_any, (
|
||||
f"IMP-95 u10: across mdx 01/02/04/05 under PHASE_Z_B4_V4_EVIDENCE=1, "
|
||||
f"no zone reported v4_evidence_consumed=True. Flag ON is behaving "
|
||||
f"identically to flag OFF — either composition is not producing "
|
||||
f"v4_candidates, the Step 11 wiring at src/phase_z2_pipeline.py:6650 "
|
||||
f"is broken, or _select_frame_v4_aware is returning the declaration-"
|
||||
f"order fallback on every rank. Per-mdx field presence tests would "
|
||||
f"all silently pass in this state; only the aggregate catches it. "
|
||||
f"(first_consumed={first_consumed})"
|
||||
)
|
||||
|
||||
|
||||
def test_imp95_u10_mdx_batch_covers_stage2_scope() -> None:
|
||||
"""Guard the Stage 2 u10 scope literal (mdx 01/02/04/05) against silent
|
||||
drift; mirrors u8/u9 scope-guard idiom."""
|
||||
assert _MDX_BATCH == ("01.mdx", "02.mdx", "04.mdx", "05.mdx"), (
|
||||
f"u10 mdx batch drifted from Stage 2 scope: {_MDX_BATCH}"
|
||||
)
|
||||
for mdx_file in _MDX_BATCH:
|
||||
assert (_SAMPLES_DIR / mdx_file).exists(), (
|
||||
f"u10 scope file missing on disk: {_SAMPLES_DIR / mdx_file}"
|
||||
)
|
||||
|
||||
|
||||
def test_imp95_u10_hoisted_key_set_is_anchored_to_u1_plus_u5() -> None:
|
||||
"""Pin the exact key set u10 verifies.
|
||||
|
||||
If a new top-level trace key is hoisted by a future unit (e.g. a u5b
|
||||
extension or a 89-d marker hoist) without updating u10, the
|
||||
regression would silently skip the new field. This test fails loudly
|
||||
when the imported u1 ``TRACE_KEY_*`` set diverges from the keys u10
|
||||
asserts — forcing the new field through the same presence/type
|
||||
verification path.
|
||||
"""
|
||||
u1_keys = {
|
||||
TRACE_KEY_FRAME_SELECTION_BASIS,
|
||||
TRACE_KEY_V4_EVIDENCE_CONSUMED,
|
||||
TRACE_KEY_V4_RANK_USED,
|
||||
TRACE_KEY_V4_B4_FRAME_MATCH,
|
||||
TRACE_KEY_B4_V0_FALLBACK_REASON,
|
||||
TRACE_KEY_B4_PARTIAL_MISSING_SKIP,
|
||||
}
|
||||
assert len(u1_keys) == 6, (
|
||||
f"u1 TRACE_KEY_* set changed: {u1_keys}. u10 verifies these 6 keys "
|
||||
f"plus the u5 v4_short_circuit; update _assert_placement_trace_shape "
|
||||
f"if a new u1 key was added."
|
||||
)
|
||||
assert _TRACE_KEY_V4_SHORT_CIRCUIT == "v4_short_circuit", (
|
||||
f"u5 short-circuit key drifted from canonical literal "
|
||||
f"'v4_short_circuit': {_TRACE_KEY_V4_SHORT_CIRCUIT!r}."
|
||||
)
|
||||
Reference in New Issue
Block a user