e3d2ff6c77
- sut-prober evaluation (pass) - diff-reporter evaluation (pass with 1 partial follow-up) - normalizer evaluations v1 (fail) + v2 (pass) - PROGRESS.md Done rows for #3, #4, #5 + Follow-ups - PLAN.md P0 reduced to hook verification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2.8 KiB
2.8 KiB
Evaluation — normalizer (2026-04-07 14:30)
Verdict: fail
| # | DoD item | Score | Evidence |
|---|---|---|---|
| 1 | Normalize(input, profile) API in Recordingtest.Normalizer |
pass | src/Recordingtest.Normalizer/Normalizer.cs exposes Normalizer.Normalize(string, string) returning NormalizeResult. Build green via dotnet build recordingtest.sln. |
| 2 | Default profile with >=5 rules (timestamps, GUIDs, paths, floats epsilon 1e-6, key sort) | pass | src/Recordingtest.Normalizer/profiles/default.yaml lists 5 rules; Rules.cs implements all five (TimestampRegex, GuidRegex, NormalizePaths with <REPO>/<USER>, RoundFloatsInNode at 6 decimals, SortJsonKeys recursive). |
| 3 | Profiles declared as profiles/*.yaml, code-free addition |
pass | Profile.Load reads YAML; adding a YAML file in profiles/ registers a new profile without code change. |
| 4 | Per-rule before/after sample test | pass | tests/Recordingtest.Normalizer.Tests/RuleTests.cs has one test per rule (StripTimestamps, MaskGuids, NormalizePaths, RoundFloats, SortJsonKeys) plus Normalize_AppliesAllDefaultRules. |
| 5 | Idempotent (same bytes on second pass) | pass | RuleTests.Normalize_IsIdempotent asserts first.Output == second.Output. |
| 6 | Sidecar log normalization.log |
fail | Normalizer.cs only returns NormalizeResult(Output, Log) in-memory. No file is written; no normalization.log artifact exists anywhere in the repo. Generator self-flagged this. |
| 7 | json-configs.json suspected fields fully covered by default profile (per-field mapping) |
partial | CoverageTests.cs builds the field set then short-circuits with ` |
| 8 | All Normalizer tests pass | pass | dotnet test tests/Recordingtest.Normalizer.Tests: 8 passed, 0 failed, 0 skipped (129 ms). |
Notes
- Build: 0 warnings / 0 errors.
- Test count: 8 (7 in
RuleTests.cs, 1 inCoverageTests.cs). - Verdict is fail because DoD #6 (sidecar log) is unimplemented and DoD #7 (suspected-field coverage) is only catch-all. Both must be addressed before PROGRESS.md flips to done.
- Suggested remediation:
- Add
Normalizer.Normalize(input, profile, sidecarPath)overload (or always emit a.normalization.lognext to output) recording(ruleId, count)lines. - Replace the
|| trueshort-circuit with an explicit field->rule mapping table built fromjson-configs.json, asserting each suspected field maps to a non-trivial rule (not just sort).
- Add
- Strengths: rule implementations are clean, idempotency is genuinely tested, default profile YAML loader is straightforward.