Files
recordingtest/docs/contracts/smoke-gap-fix.evaluation.md
minsung a5523b41e5 Orchestrate smoke gap fix evaluation + close smoke iter 1 (#11)
- Recorder STAThread + KeyTranslator + 3 split counters
- Player StepKind Wheel/Focus + null-target guard
- 60/60 tests, regression trap verified
- gitignore artifacts/ and scenarios/ (smoke local data)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 17:34:49 +09:00

3.7 KiB

Smoke Gap Fix Evaluation — Issue #11

Commit: 139fbbc Evaluator date: 2026-04-07 Build: green (0 warn / 0 err, TreatWarningsAsErrors on) Tests: 60 passed / 0 failed / 0 skipped (10+10+12+5+5+6+6+6)

Verdict: pass

Gap verdict table

# Gap Fix location Regression test Verdict
1 Recorder emits steps with null target (MTA + resolver silent null) Program.cs: [STAThread] on Main, resolver skips key_down/key_up, noResolverAttempt vs unresolvedPaths counters split; DragCollapser.cs routes non-key events through resolver on main thread RecorderTests.DragCollapser_PrintableKeys_CollapseIntoSingleTypeStep + Program.cs counters printed (unresolved_paths, no_resolver_attempt, null_target_steps) pass
2 VK translation missing → printable keys dropped / hotkeys unrecognised KeyTranslator.cs (VK→text table: modifiers 0x10/0x11/0x12/0xA0-0xA5/0x5B-0x5C, named 0x08/09/0D/1B/20-28/2D/2E, letters 0x41-0x5A, digits 0x30-0x39 + numpad, F1-F24); DragCollapser.cs collapses printable runs into type step and modifier+letter into hotkey DragCollapser_PrintableKeys_CollapseIntoSingleTypeStep asserts Value == "BOX" (uppercase per chosen convention); additional recorder tests cover hotkey combos pass
3 Player StepKind enum missing Wheel/Focus → ScenarioLoader crashes on recorder yaml Model/Step.cs adds Wheel, Focus; PlayerEngine.cs cases log + no-op SmokeRegressionTests.FullPipeline_ParsesAndRunsWithoutException loads yaml with kind: wheel, kind: focus and asserts parse + run pass
4 Player would click at (0,0) on desktop when target is null PlayerEngine.cs StepRequiresTarget + early-return warn when step.Target is null for Click/Drag/Type/Focus PlayerEngineTests.PlayerEngine_NullTarget_SkipsWithoutCalling and SmokeRegressionTests assert host.Clicks empty when click step has null target pass

New tests (3)

Test File Assertion summary Verdict
DragCollapser_PrintableKeys_CollapseIntoSingleTypeStep tests/Recordingtest.Recorder.Tests/RecorderTests.cs 2 steps produced (click + type), steps[1].Value == "BOX" real, not stub
FullPipeline_ParsesAndRunsWithoutException tests/Recordingtest.Player.Tests/SmokeRegressionTests.cs Embeds 6-step yaml with wheel/focus/null-target click; asserts StepKind.Wheel, StepKind.Focus, host.Clicks empty, host.Drags single, host.Types == ["BOX"], host.Hotkeys contains ctrl+c real
PlayerEngine_NullTarget_SkipsWithoutCalling tests/Recordingtest.Player.Tests/PlayerEngineTests.cs Three null-target steps (Click, Drag, Type) → host records zero calls real

Regression trap verification (via git show HEAD~1:…)

  • Pre-fix Step.cs enum lacks Wheel/FocusScenarioLoader.LoadFromString(SmokeYaml) in SmokeRegressionTests would throw on kind: wheel. New test fails.
  • Pre-fix PlayerEngine.cs has no null-target guard or StepRequiresTarget → null-target Click would fall through to host.Click(default) (i.e. (0,0)); PlayerEngine_NullTarget_SkipsWithoutCalling expects zero Click calls. Fails.
  • Pre-fix recorder lacked KeyTranslator + printable-key accumulator → DragCollapser_PrintableKeys_CollapseIntoSingleTypeStep expects 2 steps with type/"BOX". Fails.

All three new tests would fail against HEAD~1. Traps genuine.

Other checks

  • Thread.Sleep( in src/Recordingtest.Player/: 0 occurrences.
  • Scenario.cs has public uint? RawVk { get; set; } (line 35) — contract raw_vk field preserved.
  • No writes to EG-BIM Modeler/ during evaluation.
  • Recorder Program.cs prints all three counters in final log line.