kyeongmin
cacc5b30db
- u1: BuilderMissingError(FitError) — narrow exception aligned with pipeline catch - u2: load_frame_contracts catalog invariant + VP skip + CatalogInvariantError - u3a: audit CLI I1~I3 (partial existence / declared builder / registry membership) - u3b: audit CLI I4 (slot_payload refs vs declared/generated payload keys) - u4: lookup_v4_candidates VP filter (lookup_v4_all_judgments raw telemetry untouched) - u5: catalog invariant regression coverage + temp non-VP failure fixtures - u6: mdx04 VP routing fixture tests (sw_dependency_four_problems excluded from live) - u7: tests/conftest.py env isolation + mdx03/mdx04/mdx05 subprocess smoke Targeted 74 PASS (12.31s). Full regression 1063 PASS (87.70s). Audit CLI clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
113 lines
4.8 KiB
Python
113 lines
4.8 KiB
Python
"""IMP-#85 u7 — pytest env isolation for src.config defaults.
|
|
|
|
This conftest.py runs BEFORE any test module is imported by pytest.
|
|
Setting ``os.environ["AI_FALLBACK_*"]`` here overrides values that the
|
|
live ``.env`` file would otherwise inject through ``pydantic-settings``
|
|
(priority: init args > os.environ > env_file). The ``src.config``
|
|
module-level ``settings = Settings()`` singleton is therefore built
|
|
against the test-clean environment when src.config is first imported
|
|
during test collection.
|
|
|
|
Scope (per Stage 2 plan u7):
|
|
* Restore the default-OFF contract for ``ai_fallback_enabled`` so
|
|
``tests/test_phase_z2_ai_fallback_config.py`` and
|
|
``tests/test_imp47b_step12_ai_wiring.py`` (which lock the
|
|
flag-off short-circuit) match the source-of-truth default in
|
|
``src/config.py``.
|
|
* Restore the default-OFF contract for ``ai_fallback_auto_cache``.
|
|
|
|
Out of scope:
|
|
* Touching ``ANTHROPIC_API_KEY`` / ``KEI_API_URL`` / ``LOG_LEVEL``.
|
|
* Resetting the ``src.config.settings`` singleton mid-session.
|
|
Tests that need to flip ``settings.ai_fallback_enabled`` at
|
|
runtime mutate the singleton directly (mirrors the production
|
|
``--auto-cache`` CLI path in ``src/phase_z2_pipeline.py``).
|
|
|
|
IMP-35 baseline-red invariance carve-out
|
|
========================================
|
|
The IMP-35 baseline-red invariance gate at
|
|
``tests/phase_z2/test_imp35_baseline_red_invariance.py`` spawns a child
|
|
pytest subprocess that targets ONLY the two baseline-area files:
|
|
|
|
tests/test_imp47b_step12_ai_wiring.py
|
|
tests/test_phase_z2_ai_fallback_config.py
|
|
|
|
That gate's binding contract (Stage 2 u11 lock) is that those four
|
|
registered known-red tests STAY RED until a follow-up issue
|
|
deregisters them. If this conftest blindly forces
|
|
``AI_FALLBACK_ENABLED=false`` in the gate's subprocess, the
|
|
``test_ai_fallback_master_flag_default_off`` registered red flips
|
|
green and the invariance gate trips — a real cross-issue contract
|
|
conflict (see Codex #8 Stage 3 verification of IMP-#85 u7).
|
|
|
|
The carve-out below detects that exact subprocess signature
|
|
(positional ``.py`` targets are entirely baseline-area files) and
|
|
skips env isolation, leaving the gate's child process in its native
|
|
``.env``-loaded state. Every other pytest invocation — full-suite
|
|
``pytest -q tests``, the IMP-#85 smoke targets, single-file dev runs
|
|
on non-baseline files — still gets the default-OFF isolation.
|
|
|
|
Per ``feedback_demo_env_toggle_policy``: demo activation belongs in
|
|
``.env`` only. The override below is test-scoped (lives under
|
|
``tests/``) and never propagates into ``src/`` or ``vite.config``.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import os
|
|
import sys
|
|
|
|
# File suffixes (basenames) of the IMP-35 baseline-red area files.
|
|
# The IMP-35 gate spawns its subprocess with these as the sole positional
|
|
# pytest targets. Suffix matching is used so the detection is robust
|
|
# across Windows/POSIX path separators and absolute/relative cwd.
|
|
_IMP35_BASELINE_AREA_FILE_SUFFIXES: tuple[str, ...] = (
|
|
"test_imp47b_step12_ai_wiring.py",
|
|
"test_phase_z2_ai_fallback_config.py",
|
|
)
|
|
|
|
|
|
def _is_imp35_baseline_subprocess() -> bool:
|
|
"""True iff the current pytest argv targets ONLY IMP-35 baseline-area files.
|
|
|
|
The IMP-35 baseline-red invariance gate
|
|
(``tests/phase_z2/test_imp35_baseline_red_invariance.py``) runs:
|
|
|
|
python -m pytest -q --tb=no -p no:cacheprovider \\
|
|
tests/test_imp47b_step12_ai_wiring.py \\
|
|
tests/test_phase_z2_ai_fallback_config.py
|
|
|
|
The two trailing positional ``.py`` arguments are the signature.
|
|
We compare on basename suffix so the check is path-separator and
|
|
cwd agnostic.
|
|
|
|
Returning True here suppresses the ``AI_FALLBACK_*`` env override
|
|
so the baseline-red registry contract (Stage 2 u11 lock) holds for
|
|
the gate's child process while every other invocation
|
|
(full-suite, IMP-#85 smokes, mixed-target dev runs) still gets the
|
|
default-OFF isolation.
|
|
"""
|
|
file_targets = [arg for arg in sys.argv[1:] if arg.endswith(".py")]
|
|
if not file_targets:
|
|
return False
|
|
return all(
|
|
any(
|
|
arg.replace("\\", "/").endswith(suffix)
|
|
for suffix in _IMP35_BASELINE_AREA_FILE_SUFFIXES
|
|
)
|
|
for arg in file_targets
|
|
)
|
|
|
|
|
|
if _is_imp35_baseline_subprocess():
|
|
# Drop any inherited AI_FALLBACK_* values so the gate's child process
|
|
# falls back to the live ``.env`` (AI_FALLBACK_ENABLED=true) — the
|
|
# exact precondition under which the four registered baseline-red
|
|
# tests are red. ``pop`` is no-op when the key is absent, so a
|
|
# developer running the gate manually with a clean environment is
|
|
# unaffected.
|
|
os.environ.pop("AI_FALLBACK_ENABLED", None)
|
|
os.environ.pop("AI_FALLBACK_AUTO_CACHE", None)
|
|
else:
|
|
os.environ["AI_FALLBACK_ENABLED"] = "false"
|
|
os.environ["AI_FALLBACK_AUTO_CACHE"] = "false"
|