Files
recordingtest/docs/contracts/normalizer.evaluation.md
minsung e3d2ff6c77 Orchestrate P1 evaluations and update progress (#3, #4, #5)
- sut-prober evaluation (pass)
- diff-reporter evaluation (pass with 1 partial follow-up)
- normalizer evaluations v1 (fail) + v2 (pass)
- PROGRESS.md Done rows for #3, #4, #5 + Follow-ups
- PLAN.md P0 reduced to hook verification

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 14:20:55 +09:00

3.6 KiB

Evaluation — normalizer (v2, 2026-04-07)

Verdict: pass

Generator iteration: commit 05c7a3f.

# DoD item Score Evidence
1 Normalize(input, profile) API pass src/Recordingtest.Normalizer/Normalizer.cs exposes Normalize(string, string) and overload Normalize(string, string, string?). Build green.
2 Default profile with >=5 rules pass src/Recordingtest.Normalizer/profiles/default.yaml lists 6 rules: strip_timestamps, mask_guids, normalize_paths, round_floats, mask_volatile_settings, sort_json_keys. All implemented in Rules.cs.
3 Profiles as profiles/*.yaml, code-free addition pass Profile.Load reads YAML by name.
4 Per-rule before/after sample test pass RuleTests.cs covers each rule plus Normalize_AppliesAllDefaultRules (asserts 6 entries in log including mask_volatile_settings).
5 Idempotent pass RuleTests.Normalize_IsIdempotent.
6 Sidecar log normalization.log pass Normalizer.cs lines 150-176: when sidecarPath supplied, writes file containing {RuleId}\tcount={Count} lines sorted by RuleId (Ordinal) and final total= line. Accepts either a file path or directory (in which case it writes normalization.log inside). Two real-temp-file tests: Normalize_WritesSidecarLogFile and Normalize_SidecarPath_AcceptsDirectory — both assert file existence and content (sorted order, total line, per-rule lines).
7 json-configs.json suspected fields fully covered pass CoverageTests.cs now declares an explicit Dictionary<string,string> FieldRuleMap (18 entries, StringComparer.Ordinal) with no `
8 All Normalizer tests pass pass dotnet test tests/Recordingtest.Normalizer.Tests: 10 passed, 0 failed, 0 skipped (167 ms).

Notes

  • dotnet build recordingtest.sln: 0 warnings, 0 errors.
  • Test count grew from 8 → 10 (added two sidecar tests). Coverage test rewritten in place.
  • New rule mask_volatile_settings (Rules.cs lines 172-224) is fully implemented (not a stub): allowlist HashSet of 16 known volatile field names, walks JsonNode recursively, replaces matching values with "<VOLATILE>" and counts replacements. Idempotent because the placeholder string itself is not in the allowlist's value space.
  • Risk (non-blocking): the volatile-field allowlist is keyed on local field name only (no JSON path scoping). A real bug that incidentally toggles a field named e.g. GridSnap in a structurally unrelated subtree would be masked and silently hidden by golden-file diffs. Allowlist is currently 16 names — narrow enough to be acceptable, but should be revisited if the catalog grows. Recommend documenting this allowlist scope in normalizer.md in a follow-up (does not block this iteration).
  • Coverage test no longer accepts catch-all to sort_json_keys; mapping is strict and explicit per the contract's field→rule requirement.
  • Sidecar format matches the spec exactly: tab-separated ruleId\tcount=N, ordinal-sorted, terminated by total=N.

Artifacts

  • src/Recordingtest.Normalizer/Normalizer.cs
  • src/Recordingtest.Normalizer/Rules.cs
  • src/Recordingtest.Normalizer/profiles/default.yaml
  • tests/Recordingtest.Normalizer.Tests/RuleTests.cs
  • tests/Recordingtest.Normalizer.Tests/CoverageTests.cs
  • Previous report: docs/contracts/normalizer.evaluation.v1.md