recorder: focus poller PoC for Gap I-1 (deferred, #14)
Adds a background focus poller that periodically calls
Automation.FocusedElement() and stamps the path onto key_down RawEvents,
so DragCollapser can fill type-step targets without relying on the stale
post-hoc Resolve() pass. Plumbing:
Program.cs — focus poller Task + diagnostic counters
LowLevelHook — volatile CurrentFocusedPath, stamped on key_down
RawEvent — FocusedElementPath already existed (focus_change)
DragCollapser— typeFocusPath captured at first printable key_down,
takes precedence over lastFocusPath/lastMousePath
Result on box-v7.yaml live recording: null_target_steps unchanged (13).
Root cause: EG-BIM Modeler's CommandBox and similar input controls lack
AutomationPeer, so UIA-based focus tracking — from any external process —
cannot see them. The WPF-internal Keyboard.FocusedElement is in-process
only and unreachable from the recorder.
Deferred. The plumbing stays in place because the same stamping path can
be reused by a future generic WPF DLL-injection probe. Player's existing
null-target fallback (Type→OS focus, Click→raw_coord) remains the official
strategy and successfully replays box-v7 end-to-end.
See docs/history/2026-04-08_gap-i1-deferred.md for analysis and future
options (generic WPF injection / AutomationPeer AI attachment).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -60,7 +60,7 @@ _(없음)_
|
||||
- [ ] player: `UiaPlayerHost` uia_path resolver가 마지막 `@AutomationId`만 사용 — 전체 ancestor chain 지원 필요.
|
||||
- [ ] recorder: IME 조합 키 처리 (contract risks).
|
||||
- [x] ~~player: foreground settle 안정화~~ — 능동 대기(`GetForegroundWindow` polling 2s + 100ms settle)로 전환, 1차 재생 성공 확인
|
||||
- [ ] recorder: null_target 이벤트 자체를 줄이기 — `Automation.FocusedElement` 직접 조회해 type 스텝 target 채우기 (issue #14 Gap I-1). 현재는 player fallback으로 우회.
|
||||
- [~] recorder Gap I-1 — UIA `Automation.FocusedElement` 폴링 PoC 시도(commit pending). 결과: SUT의 CommandBox 등 AutomationPeer 미부착 컨트롤은 UIA 외부에서 본질적으로 못 봄. **deferred** — generic WPF DLL injection 또는 SUT-side AutomationPeer 부착 PoC가 필요. 현재는 Player fallback(null target → OS 키 입력 / raw_coord 클릭)이 공식 전략.
|
||||
|
||||
## Blocked
|
||||
|
||||
|
||||
Reference in New Issue
Block a user