4cee3c2d86
- Static HmEG catalog via MetadataLoadContext, 13 assemblies, 11k+ candidates - IEngineSnapshot API draft + probe design doc (plugin masquerade recommended) - All DoD pass on first iteration Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
61 lines
5.4 KiB
Markdown
61 lines
5.4 KiB
Markdown
# engine-bridge PoC v1 — Evaluation
|
|
|
|
- Contract: `docs/contracts/engine-bridge.md`
|
|
- Generator commit: `2a4f1d3`
|
|
- Evaluator date: 2026-04-07
|
|
- Issue: #9
|
|
|
|
## Verdict: **PASS**
|
|
|
|
All DoD items satisfied. Build green, 6/6 tests pass, probe exits 0, catalog output is deterministic across two runs, all 4 candidate categories present with non-trivial counts, skeleton API throws NotImplementedException, probe design doc contains 5 options + render-signal table + explicit recommendation. No runtime invocation or SUT writes found in `src/Recordingtest.EngineBridge*`.
|
|
|
|
## DoD table
|
|
|
|
| # | DoD item | Result | Evidence |
|
|
|---|---|---|---|
|
|
| 1 | `Recordingtest.EngineBridge` lib + `Recordingtest.EngineBridge.Probe` exe | PASS | `src/Recordingtest.EngineBridge/Recordingtest.EngineBridge.csproj`, `src/Recordingtest.EngineBridge.Probe/Recordingtest.EngineBridge.Probe.csproj` |
|
|
| 2 | Static analysis via `MetadataLoadContext` + `PathAssemblyResolver(sutRoot, runtimeDir)` | PASS | `MetadataLoader.cs` uses `MetadataLoadContext` + `PathAssemblyResolver` with both SUT root and `RuntimeEnvironment.GetRuntimeDirectory()`; only `LoadFromAssemblyPath` is called |
|
|
| 3 | Candidate identification (select/camera/scene/render keywords) | PASS | `CandidateFinder.cs` Categories array: Selection/Selected/Picked/Pick, Camera/Viewport/EyePoint/LookAt/View, Scene/Document/World/Root, Render/Draw/Frame/Dirty |
|
|
| 4 | Outputs `hmeg-types.json` + `hmeg-candidates.json`, sorted, deterministic | PASS | `CatalogWriter.cs` sorts by Ordinal, `WriteIndented=true`, forward-slash paths, `\r\n`→`\n` normalization; two probe runs produce byte-identical output (`diff -q` empty) |
|
|
| 5 | `IEngineSnapshot` interface + DTOs | PASS | `IEngineSnapshot.cs` matches contract shape verbatim: `SelectedObjectIds`, `Camera`, `Scene`, `IsRenderComplete`; `CameraState(double[] EyePoint, double[] Target, double[] Up, double Fov)`, `SceneSummary(int ObjectCount, string? DocumentPath)` |
|
|
| 6 | `HmEgSnapshot` skeleton with reflection hint constants | PASS | `HmEgSnapshot.cs` all members throw `NotImplementedException`; constants `HmEgAssemblyHint="HmEG"`, `EditorManagerTypeHint="HmEGAppManager"` (+ Selection/Camera/Scene/Render hints); catalog check: `HmEGAppManager` appears 8 times in `hmeg-candidates.json` TypeNames and is its own assembly `Editor02.HmEGAppManager` |
|
|
| 7 | Probe design doc with 3 injection options + render signal + recommendation | PASS | `docs/engine-bridge-probe-design.md` contains 5 options (ALC side-load, CLR profiler, managed injector, MEF plugin masquerade, AutomationPeer baseline), render-complete latency table (poll / event / `CompositionTarget.Rendering`), explicit recommendation: **plugin masquerade (option 4)** with CLR profiler as fallback |
|
|
| 8 | xUnit tests ≥ 5 (all 5 named cases) | PASS | `EngineBridgeTests.cs` contains all 5 required tests plus bonus `HmEgSnapshot_Constants_MatchCatalog`; 6/6 pass |
|
|
| 9 | `dotnet build` green + `dotnet test` all pass | PASS | `dotnet build recordingtest.sln` → 0 warnings, 0 errors; `dotnet test tests/Recordingtest.EngineBridge.Tests` → 6 passed, 0 failed |
|
|
| 10 | No SUT write access | PASS | Grep of `src/Recordingtest.EngineBridge*/**/*.cs` for `File.Write`/`File.Create`/`StreamWriter` returns 0 hits; `CatalogWriter.WriteJson` writes only to caller-provided `--out` path; `"EG-BIM Modeler"` literal does not appear inside any EngineBridge source file |
|
|
| 11 | Execution path `dotnet run --project src/Recordingtest.EngineBridge.Probe -- --sut "EG-BIM Modeler" --out ...` | PASS | `Program.cs` parses `--sut`/`--out`, returns 2 on missing SUT path, 0 on success; live run produced 13 assemblies loaded (HmEG 2285 / HmGeometry 532 / HmGeometry.V2 1669 / HmTriangle 113 / EditorCore 416 / Editor01..07 various / Editor.AI01.HttpConnector 15), candidates select=726 camera=4226 scene=3081 render=3602 |
|
|
|
|
## Static-only guarantee
|
|
|
|
Grep across `src/Recordingtest.EngineBridge*/**/*.cs` for runtime-invocation patterns returned zero matches:
|
|
|
|
- `Activator.CreateInstance` — 0
|
|
- `Assembly.Load(` (non-path) — 0
|
|
- `RunClassConstructor` — 0
|
|
- `DllImport` / `LibraryImport` (P/Invoke) — 0
|
|
|
|
`MetadataLoadContext` by design never runs type initializers or user code; the test `MetadataLoader_LoadsHmegAssembly_WithoutExecution` is the observable check.
|
|
|
|
## Determinism evidence
|
|
|
|
Two back-to-back probe runs against `EG-BIM Modeler` with different `--out` directories:
|
|
|
|
```
|
|
diff -q hmeg-types.json -> (empty)
|
|
diff -q hmeg-candidates.json -> (empty)
|
|
```
|
|
|
|
File sizes: `hmeg-types.json` ≈ 895 KB, `hmeg-candidates.json` ≈ 3.24 MB; both parse as valid JSON.
|
|
|
|
## Catalog content spot-check
|
|
|
|
- Categories: `['camera','render','scene','select']` all present; total candidates 11,635.
|
|
- `EditorManagerTypeHint = "HmEGAppManager"` appears in 8 candidate TypeNames (also the dedicated assembly `Editor02.HmEGAppManager`), satisfying the "constant cross-checked against catalog" DoD.
|
|
- `HmEgAssemblyHint = "HmEG"` is a prefix of `HmEG`, `HmGeometry`, `HmGeometry.V2`, `HmTriangle` — matches.
|
|
|
|
## Notes / non-blocking observations
|
|
|
|
- `CandidateFinder` enumerates with `BindingFlags.DeclaredOnly` (avoids double-counting inherited members) and dedupes before sort — good hygiene for byte-identical runs.
|
|
- The probe's default assembly patterns include `Editor*.dll` wildcard which picked up all 7 Editor-prefixed assemblies automatically; future sprints can rely on this.
|
|
- Probe design doc already answers the "AutomationPeer comparison" DoD item by explicitly scoping it out to sut-prober/recorder.
|