diff --git a/PLAN.md b/PLAN.md index 31212d2..d41ec1a 100644 --- a/PLAN.md +++ b/PLAN.md @@ -10,8 +10,9 @@ ## P1 — 라이브 검증 (사용자 환경 필요) -4. **라이브 SUT smoke test 실행** — `docs/guides/smoke-test.md` 따라 수동 수행 -5. **engine-bridge v3** — ReflectionEngineStateProvider 실매핑 (smoke test 이후) +4. **이슈 #14 재생 안정화** — 첫 시도 BOX 타이핑 누락 문제. foreground settle 600ms→능동 대기 전환 또는 첫 type 이전 `Keyboard.Type` warm-up. `/contract player-foreground-stabilize`. +5. **recorder Gap I-1** — type 스텝 target을 `Automation.FocusedElement`로 직접 채워 null_target_steps=0 달성. `/contract recorder-focused-element-target`. +6. **engine-bridge v3** — ReflectionEngineStateProvider 실매핑 (smoke test 이후) ## Follow-ups (non-blocking) diff --git a/PROGRESS.md b/PROGRESS.md index 4f1222a..bf52c59 100644 --- a/PROGRESS.md +++ b/PROGRESS.md @@ -43,10 +43,7 @@ | 2026-04-07 | normalizer follow-ups + Evaluator pass — float epsilon 구성화 + JSON-path 마스크 스코핑, 77 tests | commit `eeee3c2` | | 2026-04-08 | **Smoke test 2회차 — 첫 E2E 성공** 🎉 Box geometry 생성 확인 | `docs/history/2026-04-08_smoke-2회차-첫-e2e-성공.md`, `scenarios/box-v5*.yaml` | | 2026-04-08 | 이슈 #13 Gap E/F/G fix — HotkeyParseTests + FocusEventFilter + WindowPointResolver, 94 tests | `docs/history/2026-04-08_이슈13-smoke3-fix-generator.md` | - -## In progress - -_(없음 — Smoke 2회차 라이브 검증 대기)_ +| 2026-04-08 | **이슈 #14 Raw 시나리오 E2E 성공** 🎉 수동 cleanup 없이 box-v6.yaml 재생으로 Box 생성 | player: null-target fallback + foreground switch + leading alt+tab strip + timing preservation, 24 player tests | ## In progress @@ -62,6 +59,8 @@ _(없음)_ - [ ] player: `wait_for` UIA 이벤트 매핑 강화 (현재 host passthrough). - [ ] player: `UiaPlayerHost` uia_path resolver가 마지막 `@AutomationId`만 사용 — 전체 ancestor chain 지원 필요. - [ ] recorder: IME 조합 키 처리 (contract risks). +- [ ] player: foreground settle이 경계선(600ms) — 1차 재생이 가끔 BOX 타이핑 누락. 능동 대기(focus-ready polling)로 전환 고려. (issue #14 후속) +- [ ] recorder: null_target 이벤트 자체를 줄이기 — `Automation.FocusedElement` 직접 조회해 type 스텝 target 채우기 (issue #14 Gap I-1). 현재는 player fallback으로 우회. ## Blocked diff --git a/docs/history/2026-04-08_gitea-mcp-access-qa.md b/docs/history/2026-04-08_gitea-mcp-access-qa.md new file mode 100644 index 0000000..ff39c32 --- /dev/null +++ b/docs/history/2026-04-08_gitea-mcp-access-qa.md @@ -0,0 +1,20 @@ +# Gitea MCP 접근 경로 Q&A + +**소요 시간**: 5분 +**Context 사용량**: input ~30k / output ~1k tokens +**이슈**: #0 + +## 요약 + +- 사용자 질문: "gitea 엑세스 어떻게 하고 있지. 다른 프로젝트에서는 gitea mcp가 안 된다." +- 확인 결과: Gitea MCP는 글로벌 user scope (`C:\Users\nbright\.claude.json`의 최상위 `mcpServers.gitea`)에 등록됨. + - command: `C:/Users/nbright/bin/gitea-mcp.exe -t stdio` + - env: `GITEA_HOST=https://gitea.hmac.kr`, `GITEA_ACCESS_TOKEN=6f6147...` +- 본 프로젝트에서 바로 쓸 수 있었던 이유: `.claude/settings.json`의 `permissions.allow`에 `mcp__gitea__*` 명시. + +## 다른 프로젝트에서 안 되는 원인 후보 + +1. 프로젝트별 MCP 승인 미처리 (`/mcp`로 enable 필요) +2. `disabledMcpjsonServers`에 gitea 포함 또는 `enableAllProjectMcpServers: false` +3. 프로젝트 로컬 `.mcp.json`이 잘못된 gitea 정의로 덮어씀 +4. `permissions.allow`에 `mcp__gitea__*` 누락 → 매번 허용 프롬프트 diff --git a/docs/history/2026-04-08_이슈14-raw-시나리오-e2e-성공.md b/docs/history/2026-04-08_이슈14-raw-시나리오-e2e-성공.md new file mode 100644 index 0000000..d25f11b --- /dev/null +++ b/docs/history/2026-04-08_이슈14-raw-시나리오-e2e-성공.md @@ -0,0 +1,70 @@ +# 2026-04-08 — 이슈 #14 Raw 시나리오 E2E 성공 + +**이슈**: #14 Raw 레코딩 시나리오를 수동 cleanup 없이 재생 가능하게 +**소요 시간**: ~90분 +**Context 사용량**: ~60k tokens (Opus 4.6) + +## 결과 + +🎉 **`scenarios/box-v6.yaml` 원본(AI 후처리 없음)** → Player 재생 → **SUT에 Box geometry 생성 확인**. + +이전 box-v5-clean.yaml E2E 성공은 AI가 focus 이벤트 40+개 제거, target UIA 경로 리타기팅(ItemsControl→CommandBox), 뷰포트 offset 박음질 등 수작업 후편집의 결과였다. 이번 작업으로 **수작업 없이 recorder 원본을 그대로 재생**하는 경로가 처음으로 열렸다. + +## 문제 분해 + +레코더 원본은 다음 4가지 이유로 재생 불가였다: + +1. **Type/Click null target** — recorder가 key/mouse 이벤트 발생 시 focused element를 resolve 못 해 `target: null`로 저장. Player는 "#11에서 null skip" 정책이라 아예 실행 안 함. +2. **Player 실행 시 포커스** — `dotnet run` 한 PowerShell이 foreground라 첫 type("BOX")이 PowerShell로 들어감. +3. **선두 alt+tab 노이즈** — 녹화 시작 시 사용자가 에디터→SUT로 전환하려던 alt+tab 2개가 재생 시 SUT를 오히려 off-foreground로 보냄. +4. **스텝 간 타이밍 없음** — 엔진이 즉시 연속 실행 → SUT가 BOX 명령 → 모서리 픽 전환할 틈 없음. + +## 구현 (Player 쪽 포스트프로세싱) + +### 1. Null-target fallback (PlayerEngine) +- `Type + null target` → 현재 포커스로 그대로 `host.Type()` (SUT CommandBox 포커스 가정) +- `Click + null target + raw_coord` → screen-absolute 좌표로 직접 `host.Click()` +- 기타 null target → 여전히 skip (안전) +- `Step.RawCoord: int[]?` 추가, YAML `raw_coord` 자동 매핑 + +### 2. SUT foreground 강제 (UiaPlayerHost.BringSutToForeground) +- `_app.GetMainWindow().SetForeground() + Focus()` +- 600ms settle (150 → 600으로 상향; 초기 char 드롭 관찰 후) +- Program.cs가 engine.Run 이전에 1회 호출 + +### 3. 선두 alt+tab 자동 skip (PlayerEngine.Run) +- 녹화 startup 노이즈 제거. SUT가 이미 foreground인 상태에서 alt+tab 실행은 오히려 유해. + +### 4. 스텝 간 타이밍 복원 (PlayerEngine + IPlayerHost.Delay) +- `Step.Ts: long?` 추가 +- `PlayerEngineOptions.PreserveTiming` (기본 on) +- `ts_i - ts_{i-1}` 를 150ms~3s로 클램프해 host.Delay 호출 +- **엔진 내부 `Thread.Sleep` 금지 DoD를 유지하기 위해** 딜레이는 `IPlayerHost.Delay`로 위임. `UiaPlayerHost`만 실제 sleep, `FakePlayerHost`는 기록만. +- 첫 스텝도 MinStepDelay 받도록 prevTs 시드 + +### 5. 스텝 로그 +- `[player] step {i} kind={kind} value={value}` — 라이브 디버깅용 + +## 테스트 + +- `PlayerEngine_NullTarget_SkipsWithoutCalling` → `PlayerEngine_NullTarget_Fallback_Issue14`로 교체 + - click(null+raw_coord) → clicks[0] 검증, type(null+value) → types[0] 검증 + - click(null, no raw_coord), drag(null) → 여전히 skip +- 전체 suite green: 94+ tests (Player 24, Runner 6, Recorder 26, Normalizer 16, ...) + +## 라이브 검증 (사용자 환경) + +``` +dotnet run --project src\Recordingtest.Player -- --scenario scenarios\box-v6.yaml --output-dir artifacts\replay-v6-raw --no-launch +``` + +첫 시도는 BOX 타이핑이 누락 (`[player] step 2 kind=Type value=BOX` 로그는 찍혔지만 SUT command box에 안 들어감). 두 번째 시도에서 Box geometry 생성 성공. + +## 남은 과제 (PLAN.md P1에 등록) + +- **foreground settle 경계선 문제** — 600ms가 가끔 부족. `SetForegroundWindow` 후 능동 대기(`GetForegroundWindow == sut_hwnd` polling) 또는 첫 type 이전에 Keyboard warm-up 필요. +- **recorder Gap I-1** — null_target_steps를 근본적으로 줄이려면 recorder가 key_down 시점에 `Automation.FocusedElement`를 직접 쿼리해서 typeRes를 채워야 함. 현재는 player fallback으로 우회 중. + +## 관련 커밋 + +- (pending) Player null-target fallback + foreground + alt+tab strip + timing preservation diff --git a/src/Recordingtest.Player/IPlayerHost.cs b/src/Recordingtest.Player/IPlayerHost.cs index ebea70c..9b88dfa 100644 --- a/src/Recordingtest.Player/IPlayerHost.cs +++ b/src/Recordingtest.Player/IPlayerHost.cs @@ -25,4 +25,9 @@ public interface IPlayerHost void CaptureCheckpoint(int afterStep, string saveAs); void CaptureFailureArtifacts(int stepIndex, string reason); + + // Issue #14: delay between steps. Kept on the host (not in the engine) + // because PlayerEngine contract forbids fixed sleeps; the host is free + // to implement real time or a virtual clock for tests. + void Delay(TimeSpan duration); } diff --git a/src/Recordingtest.Player/Model/Step.cs b/src/Recordingtest.Player/Model/Step.cs index 2206217..4598b56 100644 --- a/src/Recordingtest.Player/Model/Step.cs +++ b/src/Recordingtest.Player/Model/Step.cs @@ -22,6 +22,12 @@ public sealed class Step public string? WaitFor { get; set; } public int? AfterStep { get; set; } public string? SaveAs { get; set; } + // Issue #14: recorder-captured screen-absolute coordinates used as + // fallback when Target is null (Click). Optional; null for non-mouse steps. + public int[]? RawCoord { get; set; } + // Issue #14: recorder-captured absolute timestamp (ms). Used by the + // engine to preserve inter-step pacing during playback. + public long? Ts { get; set; } } public sealed class Target diff --git a/src/Recordingtest.Player/PlayerEngine.cs b/src/Recordingtest.Player/PlayerEngine.cs index cbb7686..f5bad98 100644 --- a/src/Recordingtest.Player/PlayerEngine.cs +++ b/src/Recordingtest.Player/PlayerEngine.cs @@ -6,6 +6,12 @@ public sealed class PlayerEngineOptions { public TimeSpan ResolveTimeout { get; set; } = TimeSpan.FromSeconds(10); public TimeSpan WaitForTimeout { get; set; } = TimeSpan.FromSeconds(15); + + // Issue #14: preserve recorded inter-step delays (clamped). When true the + // engine sleeps step.Ts - prevStep.Ts between steps, bounded by Min/Max. + public bool PreserveTiming { get; set; } = true; + public TimeSpan MinStepDelay { get; set; } = TimeSpan.FromMilliseconds(150); + public TimeSpan MaxStepDelay { get; set; } = TimeSpan.FromSeconds(3); } public sealed class PlayerEngine @@ -22,9 +28,51 @@ public sealed class PlayerEngine ArgumentNullException.ThrowIfNull(scenario); ArgumentNullException.ThrowIfNull(host); - for (int i = 0; i < scenario.Steps.Count; i++) + // Issue #14: strip leading alt+tab hotkey steps. These are recording + // startup noise (user tabbing from their editor into the SUT at the + // start of the session). At replay time the player already puts the + // SUT in the foreground, so re-running alt+tab here just switches + // focus AWAY from the SUT and breaks subsequent Type steps. + int start = 0; + while (start < scenario.Steps.Count) + { + var s = scenario.Steps[start]; + if (s.Kind == StepKind.Hotkey && + string.Equals(s.Value, "alt+tab", StringComparison.OrdinalIgnoreCase)) + { + Console.WriteLine( + $"[player] info: skipping leading alt+tab step {start} (issue #14)"); + start++; + continue; + } + break; + } + + // Seed prevTs so the FIRST executed step also gets a pre-delay + // (MinStepDelay). Without this, step 2's Type can fire before the + // SUT has fully settled after foreground switch. + long? prevTs = start < scenario.Steps.Count && scenario.Steps[start].Ts is long firstTs + ? firstTs - (long)_options.MinStepDelay.TotalMilliseconds + : null; + for (int i = start; i < scenario.Steps.Count; i++) { var step = scenario.Steps[i]; + + if (_options.PreserveTiming && step.Ts is long ts) + { + if (prevTs is long p) + { + var delta = ts - p; + if (delta < _options.MinStepDelay.TotalMilliseconds) + delta = (long)_options.MinStepDelay.TotalMilliseconds; + if (delta > _options.MaxStepDelay.TotalMilliseconds) + delta = (long)_options.MaxStepDelay.TotalMilliseconds; + host.Delay(TimeSpan.FromMilliseconds(delta)); + } + prevTs = ts; + } + + Console.WriteLine($"[player] step {i} kind={step.Kind} value={step.Value ?? ""}"); try { ExecuteStep(i, step, host); @@ -70,12 +118,29 @@ public sealed class PlayerEngine } else if (StepRequiresTarget(step.Kind)) { - // Issue #11: recorder may emit Click/Drag/Type/Focus steps with - // null target. Never click/drag/type at (0,0) on the desktop — - // skip with a warning instead. - Console.WriteLine( - $"[player] warn: skipping step {index} kind={step.Kind} — target is null (issue #11)"); - return; + // Issue #14: recorder emits Type/Click with null target when the + // focused element / UIA path at record time could not be resolved + // (e.g. typing into a CommandBox before any mouse click, clicks on + // canvas children that don't expose AutomationId). Fall back to: + // - Type → send keystrokes to whatever currently has focus + // - Click → use recorded raw_coord (screen-absolute) directly + // This mirrors the manual cleanup that produced box-v5-clean.yaml. + if (step.Kind == StepKind.Type) + { + } + else if (step.Kind == StepKind.Click + && step.RawCoord is { Length: >= 2 }) + { + point = new ScreenPoint(step.RawCoord[0], step.RawCoord[1]); + Console.WriteLine( + $"[player] info: step {index} kind=Click null target — using raw_coord ({point.X},{point.Y}) (issue #14)"); + } + else + { + Console.WriteLine( + $"[player] warn: skipping step {index} kind={step.Kind} — target is null and no fallback (issue #14)"); + return; + } } switch (step.Kind) diff --git a/src/Recordingtest.Player/Program.cs b/src/Recordingtest.Player/Program.cs index 0813a9f..ba5c7db 100644 --- a/src/Recordingtest.Player/Program.cs +++ b/src/Recordingtest.Player/Program.cs @@ -59,6 +59,13 @@ else } using var host = new UiaPlayerHost(app, artifactDir); + +// Issue #14: ensure SUT is the foreground window before playback so that +// keystrokes (Type/Hotkey) land on the SUT instead of whatever shell the +// player was launched from (PowerShell, VS Code, etc.). Without this, the +// very first "BOX" type step gets typed into the launching terminal. +host.BringSutToForeground(); + var engine = new PlayerEngine(); try { diff --git a/src/Recordingtest.Player/UiaPlayerHost.cs b/src/Recordingtest.Player/UiaPlayerHost.cs index 22f8a9c..2a4c2a8 100644 --- a/src/Recordingtest.Player/UiaPlayerHost.cs +++ b/src/Recordingtest.Player/UiaPlayerHost.cs @@ -206,6 +206,41 @@ public sealed class UiaPlayerHost : IPlayerHost, IDisposable } } + /// + /// Issue #14 — force the SUT main window to foreground and give it keyboard + /// focus before playback starts. Handles the common case where the player + /// was launched from a shell that still has focus when the first Type/Hotkey + /// step fires. + /// + public void BringSutToForeground() + { + try + { + var w = _app?.GetMainWindow(_automation, TimeSpan.FromSeconds(5)); + if (w is null) return; + try { w.SetForeground(); } catch { /* best-effort */ } + try { w.Focus(); } catch { /* best-effort */ } + // Small settle so the OS-level focus switch takes effect before + // the first SendInput. 150ms is enough in practice on Win10. + // Increased from 150→600ms because FlaUI Keyboard.Type drops the + // first couple of characters if SendInput fires before the OS + // finishes the focus transition (observed: "BOX" lost on first + // step, "10" succeeded later once the app had settled). + System.Threading.Thread.Sleep(600); + } + catch + { + // best-effort; if this fails the user will see the BOX text land + // in the wrong window and can re-run with the SUT focused manually. + } + } + + public void Delay(TimeSpan duration) + { + if (duration > TimeSpan.Zero) + System.Threading.Thread.Sleep(duration); + } + public void Dispose() { _automation.Dispose(); diff --git a/tests/Recordingtest.Player.Tests/FakePlayerHost.cs b/tests/Recordingtest.Player.Tests/FakePlayerHost.cs index 67c387a..6913b06 100644 --- a/tests/Recordingtest.Player.Tests/FakePlayerHost.cs +++ b/tests/Recordingtest.Player.Tests/FakePlayerHost.cs @@ -35,4 +35,6 @@ internal sealed class FakePlayerHost : IPlayerHost Checkpoints.Add((afterStep, saveAs)); public void CaptureFailureArtifacts(int stepIndex, string reason) => Failures.Add((stepIndex, reason)); + public List Delays { get; } = new(); + public void Delay(TimeSpan duration) => Delays.Add(duration); } diff --git a/tests/Recordingtest.Player.Tests/PlayerEngineTests.cs b/tests/Recordingtest.Player.Tests/PlayerEngineTests.cs index 16d47e0..9940285 100644 --- a/tests/Recordingtest.Player.Tests/PlayerEngineTests.cs +++ b/tests/Recordingtest.Player.Tests/PlayerEngineTests.cs @@ -154,25 +154,33 @@ baselines: } [Fact] - public void PlayerEngine_NullTarget_SkipsWithoutCalling() + public void PlayerEngine_NullTarget_Fallback_Issue14() { + // Issue #14: null-target fallbacks. + // - Type → send keystrokes to current focus (no target required) + // - Click w/ raw_coord → click at raw_coord screen-absolute + // - Click w/o raw_coord → still skipped (no way to click safely) + // - Drag → still skipped (no raw_coord pair handling yet) var engine = new PlayerEngine(); var host = new FakePlayerHost(); var scenario = new Scenario { Steps = { - new Step { Kind = StepKind.Click, Target = null }, - new Step { Kind = StepKind.Drag, Target = null }, + new Step { Kind = StepKind.Click, Target = null }, // skip + new Step { Kind = StepKind.Click, Target = null, RawCoord = new[] { 123, 456 } }, + new Step { Kind = StepKind.Drag, Target = null }, // skip new Step { Kind = StepKind.Type, Target = null, Value = "hello" }, }, }; engine.Run(scenario, host); - Assert.Empty(host.Clicks); + Assert.Single(host.Clicks); + Assert.Equal(new ScreenPoint(123, 456), host.Clicks[0]); Assert.Empty(host.Drags); - Assert.Empty(host.Types); + Assert.Single(host.Types); + Assert.Equal("hello", host.Types[0]); Assert.Empty(host.Failures); } diff --git a/tests/Recordingtest.Runner.Tests/Fakes.cs b/tests/Recordingtest.Runner.Tests/Fakes.cs index 7bcd0a9..a52e1d1 100644 --- a/tests/Recordingtest.Runner.Tests/Fakes.cs +++ b/tests/Recordingtest.Runner.Tests/Fakes.cs @@ -38,6 +38,7 @@ public sealed class FakePlayerHost : IPlayerHost } public void CaptureCheckpoint(int afterStep, string saveAs) { } public void CaptureFailureArtifacts(int stepIndex, string reason) { } + public void Delay(TimeSpan duration) { } } public sealed class FakeHostFactory : IRunnerHostFactory