From a5523b41e52c2bf1afaf7ca31b778d3978961d32 Mon Sep 17 00:00:00 2001 From: minsung Date: Tue, 7 Apr 2026 17:34:49 +0900 Subject: [PATCH] Orchestrate smoke gap fix evaluation + close smoke iter 1 (#11) - Recorder STAThread + KeyTranslator + 3 split counters - Player StepKind Wheel/Focus + null-target guard - 60/60 tests, regression trap verified - gitignore artifacts/ and scenarios/ (smoke local data) Co-Authored-By: Claude Opus 4.6 (1M context) --- .gitignore | 4 ++ PROGRESS.md | 2 + docs/contracts/smoke-gap-fix.evaluation.md | 40 +++++++++++++++++++ ...26-04-07_이슈11-smoke-fix-orchestration.md | 31 ++++++++++++++ ...26-04-07_이슈11-smoke-gap-fix-evaluator.md | 24 +++++++++++ 5 files changed, 101 insertions(+) create mode 100644 docs/contracts/smoke-gap-fix.evaluation.md create mode 100644 docs/history/2026-04-07_이슈11-smoke-fix-orchestration.md create mode 100644 docs/history/2026-04-07_이슈11-smoke-gap-fix-evaluator.md diff --git a/.gitignore b/.gitignore index 607ae48..825e3f7 100644 --- a/.gitignore +++ b/.gitignore @@ -16,3 +16,7 @@ Log/ # Received (not yet approved) golden files — baselines만 커밋 baselines/**/*.received.* + +# Local smoke test output +artifacts/ +scenarios/ diff --git a/PROGRESS.md b/PROGRESS.md index fa6eef7..a540fd3 100644 --- a/PROGRESS.md +++ b/PROGRESS.md @@ -35,6 +35,8 @@ | 2026-04-07 | engine-bridge PoC v1 + Evaluator pass (#9) — 정적 분석, HmEG 내부 후보 8000+, API 초안 | `src/Recordingtest.EngineBridge*/`, `docs/engine-catalog/`, `docs/engine-bridge-probe-design.md` | | 2026-04-07 | engine-bridge v2 + Evaluator pass (#10) — MEF plugin masquerade, HttpListener, HmEgHttpSnapshot, 11 tests | `src/Recordingtest.EgPlugin/`, `src/Recordingtest.EngineBridge.Client/`, `docs/guides/engine-bridge-deploy.md` | | 2026-04-07 | 라이브 SUT smoke test 가이드 작성 | `docs/guides/smoke-test.md` | +| 2026-04-07 | Smoke test 1회차 — integration gap 4개 발견 (recorder target null, VK 코드, player enum, null guard) | `scenarios/box-create.yaml` | +| 2026-04-07 | Smoke gap fix + Evaluator pass (#11) — STAThread, KeyTranslator, 60 tests, regression trap 검증 | commit `139fbbc` | ## In progress diff --git a/docs/contracts/smoke-gap-fix.evaluation.md b/docs/contracts/smoke-gap-fix.evaluation.md new file mode 100644 index 0000000..3570d55 --- /dev/null +++ b/docs/contracts/smoke-gap-fix.evaluation.md @@ -0,0 +1,40 @@ +# Smoke Gap Fix Evaluation — Issue #11 + +Commit: `139fbbc` +Evaluator date: 2026-04-07 +Build: green (0 warn / 0 err, TreatWarningsAsErrors on) +Tests: **60 passed / 0 failed / 0 skipped** (10+10+12+5+5+6+6+6) + +## Verdict: `pass` + +## Gap verdict table + +| # | Gap | Fix location | Regression test | Verdict | +|---|-----|--------------|-----------------|---------| +| 1 | Recorder emits steps with null target (MTA + resolver silent null) | `Program.cs`: `[STAThread]` on Main, resolver skips `key_down`/`key_up`, `noResolverAttempt` vs `unresolvedPaths` counters split; `DragCollapser.cs` routes non-key events through resolver on main thread | `RecorderTests.DragCollapser_PrintableKeys_CollapseIntoSingleTypeStep` + `Program.cs` counters printed (`unresolved_paths`, `no_resolver_attempt`, `null_target_steps`) | pass | +| 2 | VK translation missing → printable keys dropped / hotkeys unrecognised | `KeyTranslator.cs` (VK→text table: modifiers 0x10/0x11/0x12/0xA0-0xA5/0x5B-0x5C, named 0x08/09/0D/1B/20-28/2D/2E, letters 0x41-0x5A, digits 0x30-0x39 + numpad, F1-F24); `DragCollapser.cs` collapses printable runs into `type` step and modifier+letter into `hotkey` | `DragCollapser_PrintableKeys_CollapseIntoSingleTypeStep` asserts `Value == "BOX"` (uppercase per chosen convention); additional recorder tests cover hotkey combos | pass | +| 3 | Player `StepKind` enum missing `Wheel`/`Focus` → ScenarioLoader crashes on recorder yaml | `Model/Step.cs` adds `Wheel`, `Focus`; `PlayerEngine.cs` cases log + no-op | `SmokeRegressionTests.FullPipeline_ParsesAndRunsWithoutException` loads yaml with `kind: wheel`, `kind: focus` and asserts parse + run | pass | +| 4 | Player would click at (0,0) on desktop when target is null | `PlayerEngine.cs` `StepRequiresTarget` + early-return warn when `step.Target` is null for Click/Drag/Type/Focus | `PlayerEngineTests.PlayerEngine_NullTarget_SkipsWithoutCalling` and `SmokeRegressionTests` assert `host.Clicks` empty when click step has null target | pass | + +## New tests (3) + +| Test | File | Assertion summary | Verdict | +|------|------|-------------------|---------| +| `DragCollapser_PrintableKeys_CollapseIntoSingleTypeStep` | `tests/Recordingtest.Recorder.Tests/RecorderTests.cs` | 2 steps produced (click + type), `steps[1].Value == "BOX"` | real, not stub | +| `FullPipeline_ParsesAndRunsWithoutException` | `tests/Recordingtest.Player.Tests/SmokeRegressionTests.cs` | Embeds 6-step yaml with wheel/focus/null-target click; asserts `StepKind.Wheel`, `StepKind.Focus`, `host.Clicks` empty, `host.Drags` single, `host.Types == ["BOX"]`, `host.Hotkeys` contains `ctrl+c` | real | +| `PlayerEngine_NullTarget_SkipsWithoutCalling` | `tests/Recordingtest.Player.Tests/PlayerEngineTests.cs` | Three null-target steps (Click, Drag, Type) → host records zero calls | real | + +## Regression trap verification (via `git show HEAD~1:…`) + +- Pre-fix `Step.cs` enum lacks `Wheel`/`Focus` → `ScenarioLoader.LoadFromString(SmokeYaml)` in `SmokeRegressionTests` would throw on `kind: wheel`. New test fails. +- Pre-fix `PlayerEngine.cs` has no null-target guard or `StepRequiresTarget` → null-target Click would fall through to `host.Click(default)` (i.e. (0,0)); `PlayerEngine_NullTarget_SkipsWithoutCalling` expects zero Click calls. Fails. +- Pre-fix recorder lacked `KeyTranslator` + printable-key accumulator → `DragCollapser_PrintableKeys_CollapseIntoSingleTypeStep` expects 2 steps with `type`/`"BOX"`. Fails. + +All three new tests would fail against `HEAD~1`. Traps genuine. + +## Other checks + +- `Thread.Sleep(` in `src/Recordingtest.Player/`: 0 occurrences. +- `Scenario.cs` has `public uint? RawVk { get; set; }` (line 35) — contract `raw_vk` field preserved. +- No writes to `EG-BIM Modeler/` during evaluation. +- Recorder `Program.cs` prints all three counters in final log line. diff --git a/docs/history/2026-04-07_이슈11-smoke-fix-orchestration.md b/docs/history/2026-04-07_이슈11-smoke-fix-orchestration.md new file mode 100644 index 0000000..db8d041 --- /dev/null +++ b/docs/history/2026-04-07_이슈11-smoke-fix-orchestration.md @@ -0,0 +1,31 @@ +# 2026-04-07 이슈 #11 — Smoke gap fix 오케스트레이션 + +- **이슈**: #11 (smoke test 1회차 gap 수정) +- **소요 시간**: ~25분 (Generator + Evaluator 백그라운드) +- **Context 사용량**: ~330k tokens (orchestrator 누적) + +## 사이클 + +1. Smoke 1회차 실행 → 4개 gap 발견 +2. 이슈 #11 생성 +3. Generator 백그라운드 → commit `139fbbc` +4. Evaluator 백그라운드 → **pass** +5. PROGRESS 갱신, 이슈 close + +## 근본 원인 + +recorder의 Main 스레드가 **MTA** (`[STAThread]` 누락). UIA3은 STA 필요 → 모든 resolver 호출 조용히 null 반환. 키 이벤트는 (0,0) 좌표로 resolver 호출까지 했지만 역시 null. 단위 테스트는 `IElementSnapshot` fake 기반이라 실제 STA/UIA 경로를 안 탐. + +## 수정 요약 + +- recorder: `[STAThread]`, key 이벤트 resolver skip, 카운터 3종, `KeyTranslator.cs`, printable 런 collapse +- player: `StepKind` Wheel/Focus, null-target guard, 스킵 로그 +- 테스트: 53 → 60, 새 테스트는 regression trap 역할 검증됨 + +## 비용 + +Generator ~94k + Evaluator ~39k + Orchestrator ~20k = **~153k** + +## 다음 단계 + +Smoke 2회차 — 사용자 환경에서 재녹화/재생 시도. diff --git a/docs/history/2026-04-07_이슈11-smoke-gap-fix-evaluator.md b/docs/history/2026-04-07_이슈11-smoke-gap-fix-evaluator.md new file mode 100644 index 0000000..9a8c07e --- /dev/null +++ b/docs/history/2026-04-07_이슈11-smoke-gap-fix-evaluator.md @@ -0,0 +1,24 @@ +# 이슈 #11 Smoke Gap Fix — Evaluator + +- 날짜: 2026-04-07 +- 역할: Evaluator (Generator 독립 채점) +- 대상 커밋: `139fbbc` +- 이슈: #11 smoke gap fix (recorder target null + VK 번역, player enum + null guard) + +## 소요 시간 +약 8분 (빌드/테스트 ~15초 + 소스 및 테스트 검증) + +## Context 사용량 +약 45K 토큰 (소스 파일 7개, 테스트 3개, git show HEAD~1 2회) + +## 결과 +- 빌드: green (경고 0, 오류 0) +- 테스트: 60/60 pass (10+10+12+5+5+6+6+6) +- 판정: **pass** +- 4개 gap 모두 실제 수정 + 회귀 테스트 확인 +- 3개 신규 테스트 모두 실질 assertion (stub 아님) +- Regression trap: `HEAD~1` 대비 enum/guard/KeyTranslator 부재 시 신규 테스트 실패 확인 (소스 diff 기반) + +## 산출물 +- `docs/contracts/smoke-gap-fix.evaluation.md` +- `docs/history/2026-04-07_이슈11-smoke-gap-fix-evaluator.md` (본 문서)