recordingtest/docs/contracts/recorder.evaluation.v1.md

Recorder — Evaluation

• Generator commit: d486cbb
• Build: dotnet build recordingtest.sln → green (0 warnings, 0 errors)
• Tests: dotnet test tests/Recordingtest.Recorder.Tests → 5 passed / 0 failed / 0 skipped
• Evaluator: independent reading of source + test artifacts

Verdict table

#	DoD item	Verdict	Evidence
1	Console attach to SUT + 입력 캡처 시작	partial	Program.TryAttach uses Application.Attach(pid) or window-title scan; never Launch(). LowLevelHook installs WH_KEYBOARD_LL + WH_MOUSE_LL on dedicated STA thread. Wired but cannot be exercised in this sandbox (no SUT).
2	캡처 이벤트: 키 down/up, 클릭/드래그/휠, 포커스 변경	partial	LowLevelHook.KeyboardProc emits key_down/key_up; MouseProc emits L/R/M down+up, wheel, move. Drag is NOT collapsed into a single drag step (only down/up are recorded; Program.IsInterestingForStep only keeps mouse_down_l/r and key_down). Focus-change events are NOT captured (no UIA focus listener).
3	Event shape {ts, kind, uia_path, offset_norm, raw_coord, value}	partial	RawEvent carries TimestampMs, Kind, X, Y, Code, WheelDelta; ScenarioStep/ScenarioTarget carry kind, uia_path, offset, value. There is no persistent per-event log with all six fields — raw_coord is consumed for resolution but not stored on the emitted step.
4	3D viewport offset_norm ∈ [0..1]	pass	OffsetNormalizer.Normalize divides by width/height, clamps each axis to [0,1], returns (0,0) for zero-sized rects. Unit test OffsetNormalizer_ClicksInsideElement_ReturnsZeroToOne covers center, top-left, and out-of-bounds clamp.
5	Yaml schema 준수 (name, description, sut{exe, startup_timeout_ms}, steps[{kind, target{uia_path, offset}, value, wait_for}])	pass	Scenario.cs matches the schema; ScenarioWriter uses UnderscoredNamingConvention so casing matches contract (startup_timeout_ms, uia_path, wait_for). Test YamlSerializer_RoundtripsScenario round-trips both a click and a masked-type step.
6	비밀번호/토큰 마스킹 (PasswordBox → <MASKED>)	pass	MaskPolicy.Apply returns <MASKED> when IsPassword or ClassName == "PasswordBox". Program.ConsumeAsync sets step.Value = MaskPolicy.MaskedValue on masked targets. Test FocusedElementIsPassword_ReturnsMasked covers masked + plain paths.
7	60 FPS 영향 없음	untestable	Requires running SUT + perf measurement; not possible in sandbox. Architecture (separate STA hook thread + Channel) is consistent with the requirement.
8	종료 시 요약(이벤트 수, 소요 시간, 미결 건수)	pass (source-only)	Program.Run writes [recorder] done. events={count} elapsed={sw.Elapsed} unresolved_paths={unresolved} on Ctrl+C exit.

Additional checks:

• Program.ParseArgs returns null when --attach is missing → Main prints usage to stderr and returns 2. Verified by Cli_MissingAttach_ExitTwo.
• ElementPathBuilder.Build produces ClassName[@AutomationId='...']/... walking from topmost ancestor down, falling back to @Name and then bare ClassName. Verified by ElementPathBuilder_WithNestedElements_ReturnsFullPath.
• IME (한글 조합) handling: not implemented (acknowledged in generator notes; listed as a Risk in the contract, not a DoD item).

Gaps / required follow-ups

1. Drag collapse — mouse_down_l + movement + mouse_up_l should produce a single kind: drag step with start/end offsets. Today the recorder records only the down event as click. Blocks the contract evaluation step "Box 생성 드래그".
2. Focus-change events — No UIA FocusChangedEventHandler registration. Required by DoD #2.
3. Per-event log shape — Steps drop ts and raw_coord; the contract requires every event to be recorded in the {ts, kind, uia_path, offset_norm, raw_coord, value} shape. Either keep a sidecar event log or extend ScenarioStep with these fields.
4. Manual SUT verification — DoD #1 and the perf check (#7) require attaching to EG-BIM Modeler on a real workstation. This evaluator cannot perform that step.

Overall verdict

fail — blocks release until manual SUT run + drag/focus implementation.

Rationale (per CLAUDE.md): overall pass requires every DoD item pass. Items 1 and 2 are concretely incomplete (drag collapse + focus events missing; not merely untestable). Item 7 is structurally untestable in the sandbox and is treated as partial. Items 3 is partial because ts/raw_coord are not persisted in the output. The honest call is fail with the following release gates:

• Implement drag collapse and focus-change capture, add unit tests for the drag state machine.
• Persist ts and raw_coord on each emitted step (or sidecar log).
• Manual smoke on EG-BIM Modeler: attach by pid, click Box command, drag a box, type into a PasswordBox, Ctrl+C, verify yaml + summary.
• Re-run evaluator after the above.