At the end of each scenario playback the runner now fetches a state
snapshot from the engine-bridge HTTP server and diffs it against an
approved engine-state baseline. This closes the engine-bridge v3 loop
and adds a semantic-state axis to the golden-file regression strategy —
diffs can now catch "camera moved" or "wrong selection after command"
even when the main saved-file diff still passes.
New:
src/Recordingtest.Runner/IEngineStateSnapshotClient.cs
IEngineStateSnapshotClient.TryCapture() -> string? (never throws)
HttpEngineStateSnapshotClient — GETs /scene /camera /selection off a
base URL (default http://localhost:38080), composes them into
{ "scene": {...}, "camera": {...}, "selection": {...} }
with stable ordering so the downstream differ stays friendly.
Runner is Generic tier, so this client carries zero HmEG knowledge;
it forwards raw JSON strings.
TestRunner.RunAll now takes an optional IEngineStateSnapshotClient.
After engine.Run() completes, CaptureAndDiffSidecar():
- null client -> SidecarStatus = "skipped"
- TryCapture null/throw -> "unavailable" (main result still evaluated)
- success -> writes engine-state.received.json
- baseline found -> normalize + diff + "pass"/"fail"
- baseline missing -> "missing_baseline" (first run convention)
A sidecar "fail" promotes the overall scenario Status to "fail" so
exit code reflects semantic divergence even when the save-file diff
agrees.
ScenarioResult: SidecarCaptured / SidecarHunks / SidecarStatus.
Markdown report grows Sidecar + Sidecar Hunks columns; JSON report
picks up the new fields automatically via camelCase serialization.
Program.cs: --sidecar-url <url> (default localhost:38080) and
--no-sidecar. Default behaviour is sidecar-on so that a loaded
bridge plugin is picked up automatically; when the plugin is not
deployed the client silently reports "unavailable" and CI still runs.
Baseline lookup (new):
<baselinesDir>/<scenario>.engine-state.approved.json
<baselinesDir>/<scenario>.engine-state.json
Tests (Recordingtest.Runner.Tests, +6):
- Sidecar_NullClient_SkippedStatus
- Sidecar_ClientReturnsNull_UnavailableStatus
- Sidecar_Throws_UnavailableStatus_MainStillPasses
- Sidecar_Captured_NoBaseline_MissingBaseline_And_WritesReceivedFile
- Sidecar_Captured_BaselineIdentical_PassPass
- Sidecar_Captured_BaselineDivergent_PromotesScenarioToFail
Full suite 126 -> 132, 0 failures.
Follow-ups (PLAN.md):
- Live loop: first run writes received, user approves, rerun passes.
- Normalizer profile for engine-state (float epsilon for camera
coords, selected_ids sort, document_path masking). Currently
runs through the default profile as identity, so false fails are
possible for sensitive camera moves until this lands.
Ref: #10 engine-bridge v3 final integration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Live end-to-end verification against EG-BIM Modeler succeeded on the
second attempt:
/health -> {"status":"ok","port":38080}
/scene -> {"object_count":4,"document_path":"NewSpace0"}
/camera -> {"eye":[192.97,-328.52,170.72],
"target":[33.03,-72.61,10.78],
"up":[0,0,1],"fov":45}
/selection -> {"selected_ids":["ac0380a2-...","d9a287ee-..."]}
1st attempt returned default-zero camera. Root cause: the viewport lambda
used EditorPlugin.View, which is only populated when the plugin is
actually Run() by a user trigger; our bridge plugin just boots an HTTP
server from its constructor and never runs a command, so View stayed
null. Space access worked because RootSpace goes through AppManager,
which is populated for the whole app.
Fix (HmEgBridgePlugin.BuildProvider):
Before: viewportProvider = () => View;
After: viewportProvider = () => {
var vm = AppManager?.ViewportManager;
if (vm is null) return null;
return vm.FocusedViewport ?? vm.Viewports.FirstOrDefault();
};
Confirmed against read-only view of
HmEGApplicationManagementLibrary/SubManager/ViewportManager.cs
which exposes FocusedViewport and Viewports. EGViewport : HmEGViewport
so the lambda matches the Func<HmEGViewport?> contract directly.
Plus: scripts/deploy-egbim-plugin.bat for one-click deploy. Checks for
a running SUT, builds Debug, purges the legacy Recordingtest.EgPlugin
folder, cleans the destination, copies 3 DLLs (+ PDBs) into
EG-BIM Modeler/Plugins/Recordingtest.Sut.EgBim.PluginHost/
and prints the curl commands for verification. HmEG.dll and the
Editor*.dll assemblies are deliberately NOT copied — the SUT already
supplies them.
PROGRESS.md: engine-bridge v3 row finalized; the long-running "라이브
검증 대기" item is done. PLAN.md P1 advances to the Runner <-> sidecar
integration (snapshot /scene /camera /selection at scenario end and
include in the golden baseline).
Follow-up (noted in history): document_path returned "NewSpace0" for
an unsaved scratch document — need to retest with a saved .hmeg file
to confirm the real FileManager.CurrentFile round-trip.
Ref: #10 follow-up, engine-bridge-v3 contract DoD D7 satisfied.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- 3-tier dependency direction diagram + architecture rule
- Module table reorganized by tier (Generic / HmEG-aware / EgBim)
- Milestones: first E2E, raw scenario E2E (#14), 3-tier split, 126 tests
- Gap status (A~H done, Gap I deferred with Player fallback strategy)
- Directory tree updated to reflect post-refactor layout
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Completes the Generic / HmEG-aware / App-specific separation started in
f6b6e74. The legacy EgPlugin / EngineBridge / EngineBridge.Client /
EngineBridge.Probe modules are moved into their proper tiers, namespaces
and csproj/sln entries are renamed, and the HmegDirectStateProvider
lambdas are finally populated with real handles from the EgBim plugin
host. A new Recordingtest.Architecture.Tests project enforces the tier
rule at build time.
Moves (git mv + csproj/RootNamespace/AssemblyName rename + sln):
src/Recordingtest.EgPlugin
-> src/Sut/EgBim/Recordingtest.Sut.EgBim.PluginHost
src/Recordingtest.EngineBridge
-> src/Hmeg/Recordingtest.Hmeg.Catalog
src/Recordingtest.EngineBridge.Client
-> src/Hmeg/Recordingtest.Hmeg.Bridge.Client
src/Recordingtest.EngineBridge.Probe
-> src/Hmeg/Recordingtest.Hmeg.Catalog.Probe
tests/Recordingtest.EgPlugin.Tests
-> tests/Sut/EgBim/Recordingtest.Sut.EgBim.PluginHost.Tests
tests/Recordingtest.EngineBridge.Tests
-> tests/Hmeg/Recordingtest.Hmeg.Catalog.Tests
tests/Recordingtest.EngineBridge.IntegrationTests
-> tests/Hmeg/Recordingtest.Hmeg.Catalog.IntegrationTests
Namespace rename applied across all .cs files and csproj RootNamespace:
Recordingtest.EgPlugin -> Recordingtest.Sut.EgBim.PluginHost
Recordingtest.EngineBridge -> Recordingtest.Hmeg.Catalog
Recordingtest.EngineBridge.Client -> Recordingtest.Hmeg.Bridge.Client
Recordingtest.EngineBridge.Probe -> Recordingtest.Hmeg.Catalog.Probe
New: tests/Recordingtest.Architecture.Tests/
DependencyGraphTests walks Assembly.GetReferencedAssemblies() for each
tier and fails if a forbidden reference leaks in:
- Generic modules must not reference HmEG or any app-specific DLL
- HmEG-aware modules must not reference app-specific DLLs
- Recordingtest.Hmeg.Bridge must reference HmEG (positive check)
11 tests, all passing. Prevents future drift from CLAUDE.md §8.1.
Engine-bridge v3 wire-up (HmEgBridgePlugin.BuildProvider):
Previously the HmegDirectStateProvider lambdas returned null and the
chain fell through to reflection. They now call directly into the
EditorPlugin base class that HmEgBridgePlugin inherits:
spaceProvider = () => RootSpace
// AppManager.ViewportManager.RootSpace
viewportProvider = () => View
// EGViewport : Control, HmEGViewport
documentPathProvider = () => AppManager?.FileManager?.CurrentFile
Every lambda is wrapped in try/catch so plugin construction still
cannot throw back into the SUT. Editor02.HmEGAppManager.dll added as
a reference on Recordingtest.Sut.EgBim.PluginHost.csproj — app-
specific tier, which is allowed by the architecture tests.
Entry points were confirmed from read-only review of the SUT sources at
D:\GiteaAll\EG-BIM_Modeler\EditorPluginInterface\EditorPlugin.cs
D:\GiteaAll\EG-BIM_Modeler\HmEGApplicationManagementLibrary\HmEGAppManager.cs
D:\GiteaAll\EG-BIM_Modeler\HmEGApplicationManagementLibrary\SubManager\FileManager.cs
closing out Q1/Q2/Q6/Q7 from docs/hmeg-api-survey.md.
Tests: 115 -> 126 (+11 Architecture), 0 failures.
Next step: live verification of /scene /camera /selection with a real
SUT session; any discrepancy in HmegDirectStateProvider reflection will
be tightened after observing real HmEG camera field names.
Ref: #10 follow-up, #14 follow-up, docs/contracts/generic-sut-split.md.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Lays down the Generic / HmEG-aware / App-specific separation that lets us
target other HmEG-hosting WPF applications later, and lands the v3 engine
state provider on top of it.
Architecture rule (CLAUDE.md §8.1, new): every module belongs to exactly one
of three tiers — Generic / HmEG-aware / App-specific (e.g. EgBim). Dependency
direction is strictly App-specific → HmEG-aware → Generic. Generic must not
reference HmEG.dll; HmEG-aware must not reference any per-app assembly.
This commit is the first incremental step:
+ src/Recordingtest.Bridge.Abstractions/ (Generic, new csproj)
IEngineStateProvider, CameraSnapshot, SceneSnapshot,
NullEngineStateProvider — extracted from EgPlugin so the generic core
owns the contract. Zero SUT references.
+ src/Hmeg/Recordingtest.Hmeg.Bridge/ (HmEG-aware, new csproj)
HmegDirectStateProvider — IEngineStateProvider implemented against
the HmEG public API (Space, HmEGViewport, ISelectable, ModelBase.Uid).
Decoupled from any specific host app via Func<Space?>/Func<HmEGViewport?>
lambdas; the EgBim plugin host supplies them. Reusable for any other
WPF application that hosts HmEG.
Selection traversal walks Space.Children and collects ModelBase.Uid
for nodes whose ISelectable.IsSelected is true. We deliberately type
nodes as object + late-bound Uid lookup to avoid pulling MemoryPack
into the dependency graph.
+ tests/Hmeg/Recordingtest.Hmeg.Bridge.Tests/
5 unit tests covering null lambdas, throwing lambdas, document path
provider, and constructor null arg validation.
+ src/Recordingtest.EgPlugin/ChainedEngineStateProvider.cs
Wraps two providers; falls back from Hmeg.Direct to the existing
Reflection accessor when the primary returns empty/default. Lets us
land the new wire-up before the EgBim adapter Q1~Q7 lookups are
filled in. 7 new tests.
+ src/Recordingtest.EgPlugin/IAppManagerAccessor.cs
Reflection accessor abstraction (preserved as the v3 fallback). Looks
up Editor.AppManager.AppManager via well-known Instance/Current
property names. Unit-testable through a fake.
~ src/Recordingtest.EgPlugin/IEngineStateProvider.cs
Type definitions removed (now in Bridge.Abstractions); only the
reflection-based provider remains. ReflectionEngineStateProvider
delegates everything to IAppManagerAccessor.
~ src/Recordingtest.EgPlugin/HmEgBridgePlugin.cs
BuildProvider() picks ChainedEngineStateProvider(Hmeg.Direct,
Reflection). The HmEG-aware lambdas are stubs (return null) until the
next step wires the EgBim adapter; the chain falls through to the
reflection path so behaviour matches v2 for now.
+ docs/contracts/engine-bridge-v3.md — Sprint Contract
+ docs/contracts/generic-sut-split.md — Sprint Contract for the
remaining mass-rename / folder move (step 2, deferred).
+ docs/hmeg-api-survey.md — Read-only survey of the HmEG
public API (Space, ModelBase, HmEGViewport, IHmCamera, IPlugin) used
to design HmegDirectStateProvider. Open Q1~Q7 listed.
Tests: 94 → 115 passing, 0 failing. The new HmEG-aware test project copies
HmEG.dll next to its output (Private=true) since it runs out-of-process.
Step 2 (deferred to next session): mass-rename
src/Recordingtest.EgPlugin → src/Sut/EgBim/Recordingtest.Sut.EgBim.PluginHost + .Adapter
src/Recordingtest.EngineBridge → src/Hmeg/Recordingtest.Hmeg.Catalog
src/Recordingtest.EngineBridge.Client → split (Generic + Hmeg)
plus Recordingtest.Architecture.Tests to enforce the §8.1 dependency rule.
Ref: #10 follow-up, #14 follow-up.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds a background focus poller that periodically calls
Automation.FocusedElement() and stamps the path onto key_down RawEvents,
so DragCollapser can fill type-step targets without relying on the stale
post-hoc Resolve() pass. Plumbing:
Program.cs — focus poller Task + diagnostic counters
LowLevelHook — volatile CurrentFocusedPath, stamped on key_down
RawEvent — FocusedElementPath already existed (focus_change)
DragCollapser— typeFocusPath captured at first printable key_down,
takes precedence over lastFocusPath/lastMousePath
Result on box-v7.yaml live recording: null_target_steps unchanged (13).
Root cause: EG-BIM Modeler's CommandBox and similar input controls lack
AutomationPeer, so UIA-based focus tracking — from any external process —
cannot see them. The WPF-internal Keyboard.FocusedElement is in-process
only and unreachable from the recorder.
Deferred. The plumbing stays in place because the same stamping path can
be reused by a future generic WPF DLL-injection probe. Player's existing
null-target fallback (Type→OS focus, Click→raw_coord) remains the official
strategy and successfully replays box-v7 end-to-end.
See docs/history/2026-04-08_gap-i1-deferred.md for analysis and future
options (generic WPF injection / AutomationPeer AI attachment).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
First time box-v6.yaml (raw recorder output, 676 lines) replays end-to-end
and actually creates a Box in the SUT — no AI post-editing of target paths
or offsets required. This is the counterpart to #13's recorder-side fixes:
the player now absorbs the remaining record→replay gaps instead of demanding
a hand-cleaned scenario.
Changes (all in Recordingtest.Player):
- PlayerEngine: null-target fallbacks
- Type with null target → host.Type() against current focus
- Click with null target + raw_coord → click at screen-absolute raw_coord
- Other null targets still skipped
- PlayerEngine: strip leading alt+tab hotkey steps (recording-startup noise
that fights the player's own foreground switch)
- PlayerEngine: preserve recorded inter-step timing, clamped 150ms–3s,
routed through new IPlayerHost.Delay so the engine itself stays Sleep-free
(keeps the existing "no fixed sleep" DoD test passing)
- PlayerEngine: per-step console log for live debugging
- UiaPlayerHost: BringSutToForeground() — SetForeground + Focus + 600ms
settle, called from Program.cs before engine.Run
- Step model: add RawCoord (int[]) and Ts (long?) fields, auto-mapped from
YAML raw_coord / ts keys
Tests updated:
- PlayerEngine_NullTarget_SkipsWithoutCalling → _Fallback_Issue14
(verifies the new Click-with-raw_coord and Type behavior)
- FakePlayerHost (both player.tests and runner.tests) implement Delay
Live smoke: box-v6.yaml raw replay produced the expected Box geometry on
the 2nd attempt; 1st attempt dropped the initial "BOX" keystrokes, tracked
as a follow-up (foreground settle is still threshold-sensitive at 600ms).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Gap E/F/G evaluated: pass with caveat (G honest partial)
- 94/94 tests, Anthropic API 529 mid-session recovery demonstrated
- Smoke 3회차 라이브 검증 대기
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- UiaPlayerHost.Hotkey now recognizes enter/tab/esc/space/arrows/F1-F9
- Live run generated an actual Box geometry in EG-BIM Modeler (milestone)
- Gap F/G/H documented for follow-up (issue #13)
- PROGRESS: first E2E success row
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Follow-ups to #4 normalizer PoC v2:
- Profile.float_decimals (default 6) flows into Rules.RoundFloatsInNode.
- mask_volatile_settings switches from name-only HashSet to a JSONPath-lite
allowlist ($.a.b.c) so same-named fields in unrelated subtrees stay intact.
- default.yaml migrated; 6 new tests including a regression trap for the
unrelated-subtree case. 16/16 normalizer tests, 77/77 solution tests.
Refs #2
- Live SUT smoke surfaced 4 structural gaps post-#11 fix
- recorder UIA path capture works (PID attach + FromPoint)
- player resolver only matches last AutomationId → wrong elements
- recorder type-step target empty → player skips text input
- Box command not reproduced; coordinate replay partially works
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Static HmEG catalog via MetadataLoadContext, 13 assemblies, 11k+ candidates
- IEngineSnapshot API draft + probe design doc (plugin masquerade recommended)
- All DoD pass on first iteration
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- 5-module E2E integration runner, 6 tests, all DoD pass
- PROGRESS.md Done row, PLAN.md pivoted to live smoke test
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- recorder v1 (fail) → v2 (pass): drag state machine, focus events, ts/raw_coord
- player pass with caveats: reliability untestable in sandbox
- PROGRESS.md Done rows + follow-ups for live SUT smoke test
- PLAN.md P1 pivoted to test-runner + live smoke test
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- CLAUDE.md with collaboration rules and Planner/Generator/Evaluator cycle
- .claude/ agents, commands, skills, hooks per Claude Code conventions
- Sprint Contracts for sut-prober, normalizer, recorder, player, diff-reporter
- SUT catalog (EG-BIM Modeler, 187 plugins) and .gitignore excluding SUT tree
- PROGRESS.md / PLAN.md as shared agent handoff state
- Solution scaffold targeting sut-prober PoC
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
