test: 통합 스냅샷 재생성 스크립트 추가 + 회귀 테스트 갱신 (#29 후속 작업분)
Multi-MDX Regression (IMP-91) / multi-mdx-regression (push) Failing after 9m49s

- tests/integration/scripts/regenerate_snapshots.py: 스냅샷 일괄 재생성 도구
- 통합 스냅샷 9종 갱신, 회귀/유닛 테스트 7건 보강
- ISSUE_DRAFTS_2026-07-02.md: 7/2 전수 검토 이슈 초안 보존

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-12 11:13:44 +09:00
co-authored by Claude Fable 5
parent 9b79bf7538
commit 4d7c401779
19 changed files with 2274 additions and 851 deletions
+25 -33
View File
@@ -113,40 +113,29 @@ def test_non_vp_smoke_runs_clean(mdx_name: str, prefix: str) -> None:
)
def test_mdx05_blocked_exit_empty_shell_no_content() -> None:
"""mdx05 must exit 1 (BLOCKED) with ``overall=EMPTY_SHELL_NO_CONTENT``.
def test_mdx05_renders_clean_post_emergency_p3() -> None:
"""mdx05 renders successfully (exit 0) — IMP-87 BLOCKED 계약의 후속.
IMP-#87 u5 mdx05 is the canonical Case B fixture (zero V4
evidence for any aligned section per Stage 1; ``judgments_full32 = 0``
in step05). The pre-IMP-#87 pipeline mislabelled this state as
``overall=PASS`` + ``full_mdx_coverage=True`` because the only
rendered unit was an IMP-#30 u4 EMPTY-SHELL placeholder
(``frame_template_id="__empty__"``) which trivially passes the
Selenium overflow check. IMP-#87 u1 splits content-rendered
coverage from legacy ``covered_section_ids``, u2 elevates the
overall enum to ``EMPTY_SHELL_NO_CONTENT`` before the legacy
ladder, and u3 routes that enum to a BLOCKED CLI exit (1).
This smoke pins the post-IMP-#87 contract on the real mdx05
pipeline run:
* subprocess returncode == 1 (BLOCKED, u3 axis A4).
* ``step20_slide_status.json`` ``overall`` ==
``"EMPTY_SHELL_NO_CONTENT"`` (u2 axis A3 precedence over the
legacy 4-way ladder).
* ``step20_slide_status.json`` ``full_mdx_coverage`` is False
(u1 axis A2 content-rendered coverage split).
* The IMP-#85 original crash marker
(``PAYLOAD_BUILDERS has no such entry``) is absent from both
stdout and stderr — the IMP-#85 crash-marker guard is
preserved on the mdx05 path even though mdx05 itself no
longer exits 0.
IMP-#87 u5 계약(mdx05 = canonical empty-shell Case B → exit 1)은
Emergency P3 (generic_fallback mandatory, GitHub #9) + GitHub #17
(05-1/05-2 V4 evidence 정식 평가)로 전제가 소멸했다. 본 smoke 는
현 계약(렌더 성공 + empty-shell terminal 부재 + full coverage)과
IMP-#85 crash-marker 가드 존속을 함께 잠근다.
"""
# ── 2026-07 갱신 (GitHub #29) — IMP-87 전제 소멸 ──────────────────
# 구 계약: mdx05 = canonical Case B (V4 evidence 0) → EMPTY_SHELL_NO_CONTENT
# + BLOCKED exit 1. 이후 두 개선으로 전제 자체가 사라짐:
# 1. Emergency P3 (2026-05-26): raw 후보 0 → generic_fallback mandatory
# (empty_shell terminal 제거 — GitHub #9 검증)
# 2. GitHub #17 (2026-07-07): 05-1/05-2 V4 evidence 를 pipeline_17b 로
# 정식 평가·병합 — "judgments_full32 = 0" 이 더 이상 사실 아님
# 현 계약: mdx05 는 렌더 성공 (exit 0) + 전 섹션 커버 + 텍스트 무손실.
# IMP-85 crash-marker 가드는 그대로 존속.
run_id = _unique_run_id("mdx05")
cp = _run_pipeline("05.mdx", run_id)
assert cp.returncode == 1, (
f"mdx05 expected BLOCKED exit 1, got {cp.returncode}\n"
assert cp.returncode == 0, (
f"mdx05 expected rendered exit 0 (post Emergency-P3/#17), got {cp.returncode}\n"
f"--- stderr tail ---\n{cp.stderr[-1500:]}\n"
f"--- stdout tail ---\n{cp.stdout[-1500:]}"
)
@@ -166,12 +155,15 @@ def test_mdx05_blocked_exit_empty_shell_no_content() -> None:
)
status_payload = json.loads(status_path.read_text(encoding="utf-8"))
status_data = status_payload.get("data") or {}
assert status_data.get("overall") == "EMPTY_SHELL_NO_CONTENT", (
f"mdx05 overall expected EMPTY_SHELL_NO_CONTENT, got "
assert status_data.get("overall") != "EMPTY_SHELL_NO_CONTENT", (
"mdx05 empty-shell 회귀 — Emergency P3/#17 이후 금지된 terminal"
)
assert status_data.get("overall") in {"PASS", "PARTIAL_COVERAGE"}, (
f"mdx05 overall expected PASS/PARTIAL_COVERAGE, got "
f"{status_data.get('overall')!r}"
)
assert status_data.get("full_mdx_coverage") is False, (
f"mdx05 full_mdx_coverage expected False, got "
assert status_data.get("full_mdx_coverage") is True, (
f"mdx05 full_mdx_coverage expected True, got "
f"{status_data.get('full_mdx_coverage')!r}"
)