kimminsung/recordingtest
1
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity

Orchestrate P1 evaluations and update progress (#3, #4, #5)

Browse Source 
- sut-prober evaluation (pass)
- diff-reporter evaluation (pass with 1 partial follow-up)
- normalizer evaluations v1 (fail) + v2 (pass)
- PROGRESS.md Done rows for #3, #4, #5 + Follow-ups
- PLAN.md P0 reduced to hook verification

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
minsung
2026-04-07 14:20:55 +09:00
parent 05c7a3f388
commit e3d2ff6c77
11 changed files with 267 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
docs/contracts/normalizer.evaluation.v1.md Normal file
View File

@@ -0,0 +1,23 @@
# Evaluation — normalizer (2026-04-07 14:30)
Verdict: **fail**
| # | DoD item | Score | Evidence |
|---|----------|-------|----------|
| 1 | `Normalize(input, profile)` API in `Recordingtest.Normalizer` | pass | `src/Recordingtest.Normalizer/Normalizer.cs` exposes `Normalizer.Normalize(string, string)` returning `NormalizeResult`. Build green via `dotnet build recordingtest.sln`. |
| 2 | Default profile with >=5 rules (timestamps, GUIDs, paths, floats epsilon 1e-6, key sort) | pass | `src/Recordingtest.Normalizer/profiles/default.yaml` lists 5 rules; `Rules.cs` implements all five (TimestampRegex, GuidRegex, NormalizePaths with `<REPO>`/`<USER>`, RoundFloatsInNode at 6 decimals, SortJsonKeys recursive). |
| 3 | Profiles declared as `profiles/*.yaml`, code-free addition | pass | `Profile.Load` reads YAML; adding a YAML file in `profiles/` registers a new profile without code change. |
| 4 | Per-rule before/after sample test | pass | `tests/Recordingtest.Normalizer.Tests/RuleTests.cs` has one test per rule (StripTimestamps, MaskGuids, NormalizePaths, RoundFloats, SortJsonKeys) plus `Normalize_AppliesAllDefaultRules`. |
| 5 | Idempotent (same bytes on second pass) | pass | `RuleTests.Normalize_IsIdempotent` asserts `first.Output == second.Output`. |
| 6 | Sidecar log `normalization.log` | **fail** | `Normalizer.cs` only returns `NormalizeResult(Output, Log)` in-memory. No file is written; no `normalization.log` artifact exists anywhere in the repo. Generator self-flagged this. |
| 7 | `json-configs.json` suspected fields fully covered by default profile (per-field mapping) | **partial** | `CoverageTests.cs` builds the field set then short-circuits with `|| true` claiming `sort_json_keys` covers any scalar. There is no per-field mapping table; the assertion is vacuous beyond the `IsPathField` heuristic. Per the contract, this is `partial` (catch-all via generic rule). |
| 8 | All Normalizer tests pass | pass | `dotnet test tests/Recordingtest.Normalizer.Tests`: **8 passed, 0 failed, 0 skipped** (129 ms). |
## Notes
- Build: 0 warnings / 0 errors.
- Test count: 8 (7 in `RuleTests.cs`, 1 in `CoverageTests.cs`).
- Verdict is **fail** because DoD #6 (sidecar log) is unimplemented and DoD #7 (suspected-field coverage) is only catch-all. Both must be addressed before PROGRESS.md flips to done.
- Suggested remediation:
1. Add `Normalizer.Normalize(input, profile, sidecarPath)` overload (or always emit a `.normalization.log` next to output) recording `(ruleId, count)` lines.
2. Replace the `|| true` short-circuit with an explicit field->rule mapping table built from `json-configs.json`, asserting each suspected field maps to a non-trivial rule (not just sort).
- Strengths: rule implementations are clean, idempotency is genuinely tested, default profile YAML loader is straightforward.