diff --git a/PLAN.md b/PLAN.md index c9c9357..00a95d3 100644 --- a/PLAN.md +++ b/PLAN.md @@ -5,23 +5,19 @@ ## P0 — 지금 바로 -1. **sut-prober PoC 구현** — Sprint Contract: [docs/contracts/sut-prober.md](docs/contracts/sut-prober.md) - - Generator: 일반 세션 - - Evaluator: `/evaluate sut-prober` - - 의존: 없음 (독립 실행) -2. **PROGRESS.md / PLAN.md / CLAUDE.md 훅 동작 검증** — SessionStart/Stop/Guard 3개 shell 스크립트를 실제로 트리거시켜 확인 +1. **훅 동작 검증** — SessionStart/Stop/Guard 3개 shell 스크립트를 실제로 트리거시켜 확인 - 의존: jq 설치 여부 확인 -## P1 — PoC 1단계 +## P1 — UI 자동화 의존 -3. **normalizer PoC** — Sprint Contract: [docs/contracts/normalizer.md](docs/contracts/normalizer.md) - - 의존: sut-prober의 Json 카탈로그 4. **recorder PoC (element-aware)** — Sprint Contract: [docs/contracts/recorder.md](docs/contracts/recorder.md) - - 의존: FlaUI 패키지 승인 + - 의존: FlaUI 패키지 승인 (사용자 확인 필요) 5. **player PoC** — Sprint Contract: [docs/contracts/player.md](docs/contracts/player.md) - 의존: recorder 산출물 포맷 확정 -6. **diff-reporter PoC** — Sprint Contract: [docs/contracts/diff-reporter.md](docs/contracts/diff-reporter.md) - - 의존: normalizer 규칙 1개 이상 + +## Follow-ups (non-blocking) + +- **sut-prober snake_case JSON** — `JsonNamingPolicy.SnakeCaseLower` 적용. Evaluator가 pass 처리했지만 contract 엄격 준수를 위해 권장. ## P2 — 통합 diff --git a/PROGRESS.md b/PROGRESS.md index bf09d75..5663fcd 100644 --- a/PROGRESS.md +++ b/PROGRESS.md @@ -26,10 +26,20 @@ | 2026-04-07 | SUT 카탈로그 v0 (정적) | `docs/sut-catalog/catalog.md`, `plugins.md` | | 2026-04-07 | 솔루션 스캐폴드(sut-prober PoC 타깃) | `recordingtest.sln`, `src/Recordingtest.SutProber/` | | 2026-04-07 | sut-prober PoC 구현 (Generator) | `src/Recordingtest.SutProber/`, `docs/sut-catalog/{plugins,json-configs,assemblies}.json` | +| 2026-04-07 | sut-prober PoC Evaluator pass (#3) | `docs/contracts/sut-prober.evaluation.md` | +| 2026-04-07 | diff-reporter PoC + Evaluator pass (#5) | `src/Recordingtest.DiffReporter*/`, `docs/contracts/diff-reporter.evaluation.md` | +| 2026-04-07 | normalizer PoC + Evaluator pass v2 (#4) — sidecar log, explicit coverage mapping, 6 rules | `src/Recordingtest.Normalizer/`, `docs/contracts/normalizer.evaluation.md` | ## In progress -- sut-prober PoC — Evaluator 채점 대기 (Generator 측 빌드/실행/결정성 확인 완료) +_(없음)_ + +## Follow-ups + +- [ ] sut-prober JSON naming을 `JsonNamingPolicy.SnakeCaseLower`로 변경 (contract 엄격 준수). non-blocking. +- [ ] diff-reporter: 실제 `diff-triager` 에이전트 통합 테스트 (현재 schema 단위 테스트로 대체, DoD #8 partial). non-blocking. +- [ ] normalizer: `mask_volatile_settings` 규칙을 JSON-path 스코핑으로 제한 (현재는 필드명 전역 매칭). non-blocking risk. +- [ ] normalizer: float epsilon 구성화 (현재 6 decimals 하드코딩). contract risks 섹션. ## Blocked diff --git a/docs/contracts/diff-reporter.evaluation.md b/docs/contracts/diff-reporter.evaluation.md new file mode 100644 index 0000000..412b056 --- /dev/null +++ b/docs/contracts/diff-reporter.evaluation.md @@ -0,0 +1,21 @@ +# Evaluation — diff-reporter (2026-04-07 15:00) + +Verdict: **pass** + +| # | DoD item | Score | Evidence | +|---|----------|-------|----------| +| 1 | `Recordingtest.DiffReporter` 라이브러리 + CLI | pass | `src/Recordingtest.DiffReporter/`, `src/Recordingtest.DiffReporter.Cli/` 존재; `dotnet build recordingtest.sln` 0 warning / 0 error | +| 2 | CLI 입력 `--approved --received --out` | pass | `Program.cs` 인자 파서 + 누락 시 exit 2 | +| 3 | JSON/텍스트 의미 diff, 바이너리 hex 요약 | pass | `JsonDiffer.cs` (path-flatten 후 path별 hunk), `LineDiffer.cs`, `BinaryDiffer.cs` 모두 존재; `Differ.Compare`가 타입별 분기 | +| 4 | 출력 `diff.json`, `diff.md` (`diff.html`은 옵션) | pass | CLI가 `diff.json` + `diff.md` 작성 확인 (`/tmp/dr/diff/`); html은 contract상 옵션 | +| 5 | `diff.json` 스키마 `{file, hunks[], summary{added,removed,changed}}` | pass | 샘플 출력 일치: `file`, `identical`, `hunks`, `summary{added,removed,changed}` | +| 6 | 동일 파일 → identical, exit 0 | pass | `a.json` vs `b.json` 동일 → stdout `identical`, EXIT=0, `identical:true` | +| 7 | 차이 존재 → exit 1 + 1줄 요약 | pass | `a.json` vs `c.json` → stdout `diff: +0 -0 ~1 in c.json`, EXIT=1 | +| 8 | diff-triager 통합 테스트 1개 | partial | DiffReporter 단위 테스트 5/5 통과(`DifferTests.cs`)이나 `diff-triager` 에이전트 통합 케이스 별도 확인 불가 — diff.json 스키마는 triager가 읽기 좋은 평탄 구조라 충족 가능, 외부 에이전트 의존이라 본 평가에선 partial 처리 | + +## Notes +- Library API `Differ.Compare(approvedPath, receivedPath) → DiffResult{File, Identical, Hunks, Summary}` 계약과 일치 (계약은 `Identical` 명시 안 했으나 추가 필드는 호환). +- JSON differ는 객체/배열을 path로 flatten 후 path별 hunk를 발행 — 1필드만 다른 케이스에서 hunks.length=1 검증됨. +- `diff.html`은 contract상 옵션이라 평가 기준에서 제외. +- DoD #8 통합 테스트 미존재는 partial이지만 전체 verdict는 pass(다른 모든 항목 통과 + 계약 평가 plan의 1~3 모두 충족). 후속 작업으로 triager 통합 테스트 1건 추가 권장. +- 테스트 결과: `통과 5, 실패 0, 건너뜀 0`. diff --git a/docs/contracts/normalizer.evaluation.md b/docs/contracts/normalizer.evaluation.md new file mode 100644 index 0000000..3419ebd --- /dev/null +++ b/docs/contracts/normalizer.evaluation.md @@ -0,0 +1,32 @@ +# Evaluation — normalizer (v2, 2026-04-07) + +Verdict: **pass** + +Generator iteration: commit `05c7a3f`. + +| # | DoD item | Score | Evidence | +|---|----------|-------|----------| +| 1 | `Normalize(input, profile)` API | pass | `src/Recordingtest.Normalizer/Normalizer.cs` exposes `Normalize(string, string)` and overload `Normalize(string, string, string?)`. Build green. | +| 2 | Default profile with >=5 rules | pass | `src/Recordingtest.Normalizer/profiles/default.yaml` lists 6 rules: `strip_timestamps`, `mask_guids`, `normalize_paths`, `round_floats`, `mask_volatile_settings`, `sort_json_keys`. All implemented in `Rules.cs`. | +| 3 | Profiles as `profiles/*.yaml`, code-free addition | pass | `Profile.Load` reads YAML by name. | +| 4 | Per-rule before/after sample test | pass | `RuleTests.cs` covers each rule plus `Normalize_AppliesAllDefaultRules` (asserts 6 entries in log including `mask_volatile_settings`). | +| 5 | Idempotent | pass | `RuleTests.Normalize_IsIdempotent`. | +| 6 | Sidecar log `normalization.log` | pass | `Normalizer.cs` lines 150-176: when `sidecarPath` supplied, writes file containing `{RuleId}\tcount={Count}` lines sorted by RuleId (Ordinal) and final `total=` line. Accepts either a file path or directory (in which case it writes `normalization.log` inside). Two real-temp-file tests: `Normalize_WritesSidecarLogFile` and `Normalize_SidecarPath_AcceptsDirectory` — both assert file existence and content (sorted order, total line, per-rule lines). | +| 7 | `json-configs.json` suspected fields fully covered | pass | `CoverageTests.cs` now declares an explicit `Dictionary FieldRuleMap` (18 entries, `StringComparer.Ordinal`) with no `|| true` and no catch-all. Path-bearing fields → `normalize_paths`; volatile boolean/scalar/color fields → `mask_volatile_settings`. Test fails if any suspected field is unmapped or if its mapped rule is missing from `default.yaml`. | +| 8 | All Normalizer tests pass | pass | `dotnet test tests/Recordingtest.Normalizer.Tests`: **10 passed, 0 failed, 0 skipped** (167 ms). | + +## Notes +- `dotnet build recordingtest.sln`: 0 warnings, 0 errors. +- Test count grew from 8 → 10 (added two sidecar tests). Coverage test rewritten in place. +- New rule `mask_volatile_settings` (`Rules.cs` lines 172-224) is fully implemented (not a stub): allowlist `HashSet` of 16 known volatile field names, walks `JsonNode` recursively, replaces matching values with `""` and counts replacements. Idempotent because the placeholder string itself is not in the allowlist's value space. +- **Risk (non-blocking)**: the volatile-field allowlist is keyed on local field name only (no JSON path scoping). A real bug that incidentally toggles a field named e.g. `GridSnap` in a structurally unrelated subtree would be masked and silently hidden by golden-file diffs. Allowlist is currently 16 names — narrow enough to be acceptable, but should be revisited if the catalog grows. Recommend documenting this allowlist scope in `normalizer.md` in a follow-up (does not block this iteration). +- Coverage test no longer accepts catch-all to `sort_json_keys`; mapping is strict and explicit per the contract's field→rule requirement. +- Sidecar format matches the spec exactly: tab-separated `ruleId\tcount=N`, ordinal-sorted, terminated by `total=N`. + +## Artifacts +- `src/Recordingtest.Normalizer/Normalizer.cs` +- `src/Recordingtest.Normalizer/Rules.cs` +- `src/Recordingtest.Normalizer/profiles/default.yaml` +- `tests/Recordingtest.Normalizer.Tests/RuleTests.cs` +- `tests/Recordingtest.Normalizer.Tests/CoverageTests.cs` +- Previous report: `docs/contracts/normalizer.evaluation.v1.md` diff --git a/docs/contracts/normalizer.evaluation.v1.md b/docs/contracts/normalizer.evaluation.v1.md new file mode 100644 index 0000000..bac1dee --- /dev/null +++ b/docs/contracts/normalizer.evaluation.v1.md @@ -0,0 +1,23 @@ +# Evaluation — normalizer (2026-04-07 14:30) + +Verdict: **fail** + +| # | DoD item | Score | Evidence | +|---|----------|-------|----------| +| 1 | `Normalize(input, profile)` API in `Recordingtest.Normalizer` | pass | `src/Recordingtest.Normalizer/Normalizer.cs` exposes `Normalizer.Normalize(string, string)` returning `NormalizeResult`. Build green via `dotnet build recordingtest.sln`. | +| 2 | Default profile with >=5 rules (timestamps, GUIDs, paths, floats epsilon 1e-6, key sort) | pass | `src/Recordingtest.Normalizer/profiles/default.yaml` lists 5 rules; `Rules.cs` implements all five (TimestampRegex, GuidRegex, NormalizePaths with ``/``, RoundFloatsInNode at 6 decimals, SortJsonKeys recursive). | +| 3 | Profiles declared as `profiles/*.yaml`, code-free addition | pass | `Profile.Load` reads YAML; adding a YAML file in `profiles/` registers a new profile without code change. | +| 4 | Per-rule before/after sample test | pass | `tests/Recordingtest.Normalizer.Tests/RuleTests.cs` has one test per rule (StripTimestamps, MaskGuids, NormalizePaths, RoundFloats, SortJsonKeys) plus `Normalize_AppliesAllDefaultRules`. | +| 5 | Idempotent (same bytes on second pass) | pass | `RuleTests.Normalize_IsIdempotent` asserts `first.Output == second.Output`. | +| 6 | Sidecar log `normalization.log` | **fail** | `Normalizer.cs` only returns `NormalizeResult(Output, Log)` in-memory. No file is written; no `normalization.log` artifact exists anywhere in the repo. Generator self-flagged this. | +| 7 | `json-configs.json` suspected fields fully covered by default profile (per-field mapping) | **partial** | `CoverageTests.cs` builds the field set then short-circuits with `|| true` claiming `sort_json_keys` covers any scalar. There is no per-field mapping table; the assertion is vacuous beyond the `IsPathField` heuristic. Per the contract, this is `partial` (catch-all via generic rule). | +| 8 | All Normalizer tests pass | pass | `dotnet test tests/Recordingtest.Normalizer.Tests`: **8 passed, 0 failed, 0 skipped** (129 ms). | + +## Notes +- Build: 0 warnings / 0 errors. +- Test count: 8 (7 in `RuleTests.cs`, 1 in `CoverageTests.cs`). +- Verdict is **fail** because DoD #6 (sidecar log) is unimplemented and DoD #7 (suspected-field coverage) is only catch-all. Both must be addressed before PROGRESS.md flips to done. +- Suggested remediation: + 1. Add `Normalizer.Normalize(input, profile, sidecarPath)` overload (or always emit a `.normalization.log` next to output) recording `(ruleId, count)` lines. + 2. Replace the `|| true` short-circuit with an explicit field->rule mapping table built from `json-configs.json`, asserting each suspected field maps to a non-trivial rule (not just sort). +- Strengths: rule implementations are clean, idempotency is genuinely tested, default profile YAML loader is straightforward. diff --git a/docs/contracts/sut-prober.evaluation.md b/docs/contracts/sut-prober.evaluation.md new file mode 100644 index 0000000..cb1cdc2 --- /dev/null +++ b/docs/contracts/sut-prober.evaluation.md @@ -0,0 +1,21 @@ +# Evaluation — sut-prober (2026-04-07 14:07) + +Verdict: **pass** + +| # | DoD item | Score | Evidence | +|---|----------|-------|----------| +| 1 | `dotnet build` succeeds with warnings-as-errors | pass | `dotnet build recordingtest.sln` → `경고 0개, 오류 0개` | +| 2 | `dotnet run -- --sut "EG-BIM Modeler" --out docs/sut-catalog` produces 3 catalogs, exit 0 | pass | Stdout: `Wrote catalog to docs/sut-catalog/ — plugins: 187, json: 16, assemblies: 17`, EXIT=0 | +| 3 | Three files exist & valid JSON | pass | `plugins.json`, `json-configs.json`, `assemblies.json` present; `JsonDocument.Parse` succeeds for each (used by scanner + manual Read) | +| 4 | plugins.json ≥ 180 entries with `{name, path, dlls[], size_bytes}` | pass | 187 entries; sample entry shows `Name`, `Path`, `Dlls[]`, `SizeBytes` (record `PluginEntry` in PluginScanner.cs:3) | +| 5 | json-configs.json entries have `name`, `top_level_keys`, `suspected_nondeterministic_fields` | pass | `JsonConfigEntry` record (JsonConfigScanner.cs:5-8); 16 entries serialize all three fields | +| 6 | assemblies.json has `name`, `size`/`size_bytes`, `has_pdb`; HmEG.dll has_pdb true | pass | `AssemblyEntry` (AssemblyScanner.cs:3); HmEG.dll entry: `"SizeBytes": 242715136, "HasPdb": true` | +| 7 | Determinism — second run produces no diff | pass | After 2nd run: `git status --porcelain docs/sut-catalog/` empty; `git diff --stat` empty | +| 8 | No writes to `EG-BIM Modeler/` | pass | Grep of `File.Write/Delete/Create`/`Directory.Create`: only 4 hits, all in Program.cs and all target `outDir` (= `docs/sut-catalog`). Scanners use only `Directory.EnumerateFiles/Directories`, `FileInfo.Length`, `File.ReadAllText`, `File.Exists` — read-only. | +| 9 | Paths relative to repo root, forward slash | pass | plugins.json sample: `"Path": "EG-BIM Modeler/Plugins/Eg3DFacePlugin"` — no drive letter, no backslash. PluginScanner.cs:27 calls `.Replace('\\','/')` on `GetRelativePath(".", dir)` | + +## Notes +- Property casing in JSON is PascalCase (`Name`, `SizeBytes`, `HasPdb`, `TopLevelKeys`) since no `JsonNamingPolicy` is set. Contract spec uses snake_case (`size_bytes`, `has_pdb`, `top_level_keys`). Evaluator brief explicitly accepted `size_bytes` *or equivalent*, so this is graded **pass**, but downstream consumers should be aware. Recommend adding `PropertyNamingPolicy = JsonNamingPolicy.SnakeCaseLower` in a follow-up if strict contract literal compliance is desired. +- `json-configs.json` `CategoryCommands.json` entry shows synthetic top-level keys like `CategoryCommands[0]…[N]` because the root is an object containing one array; the scanner only enumerates root object properties. Not a DoD violation but worth a note — top-level array roots would yield empty key lists. +- AssemblyScanner prefix list includes `HmTriangle`, `HmPG`, `HmCommon`, `EditorCore` beyond the contract's literal `HmEG*/Editor*/HmGeometry*` — this is a superset and doesn't violate DoD #4. +- Build, run, rerun, and git diff all clean; verdict **pass**. diff --git a/docs/history/2026-04-07_이슈3-4-5-P1-병렬완료-orchestration.md b/docs/history/2026-04-07_이슈3-4-5-P1-병렬완료-orchestration.md new file mode 100644 index 0000000..659f530 --- /dev/null +++ b/docs/history/2026-04-07_이슈3-4-5-P1-병렬완료-orchestration.md @@ -0,0 +1,50 @@ +# 2026-04-07 이슈 #3·#4·#5 — P1 병렬 Generator/Evaluator 오케스트레이션 + +- **이슈**: #3 (sut-prober), #4 (normalizer), #5 (diff-reporter) +- **소요 시간**: ~45분 (Generator/Evaluator 서브에이전트 병렬 실행 포함) +- **Context 사용량**: ~160k tokens (orchestrator 세션) + +## 진행 요약 + +이 세션은 **오케스트레이터** 역할. 실제 구현/평가는 모두 서브에이전트에 위임. + +### 사이클 + +1. 이슈 #3 생성 → Generator(sut-prober) → Evaluator **pass** → close +2. 이슈 #4, #5 생성 → Generator(normalizer) + Generator(diff-reporter) **병렬 백그라운드** +3. 두 Generator 완료 → Evaluator(normalizer) + Evaluator(diff-reporter) **병렬 백그라운드** +4. normalizer **fail** (sidecar 누락, coverage `|| true`) → Generator 재작업 → Re-evaluator **pass** +5. diff-reporter **pass** (DoD #8 partial: triager 통합 테스트 부재, follow-up) +6. PROGRESS/PLAN 갱신, 이슈 #4·#5 코멘트 및 close + +## 커밋 + +- `f043c18` sut-prober PoC (#3) +- `7920de1` diff-reporter PoC (#5) +- `3c5294a` normalizer PoC v1 (#4) +- `05c7a3f` normalizer fix (#4) + +## 결과 + +| 모듈 | 테스트 | 결과 | +|------|--------|------| +| sut-prober | 없음(CLI 검증) | pass (8/8 DoD) | +| diff-reporter | 5/5 | pass (7 pass + 1 partial) | +| normalizer | 10/10 | pass v2 (8/8 DoD) | + +## Follow-ups (non-blocking) + +- sut-prober: JSON snake_case naming policy +- diff-reporter: 실제 diff-triager 에이전트 통합 테스트 +- normalizer: mask_volatile_settings JSON-path 스코핑, float epsilon 구성화 + +## Harness design 원칙 검증 + +이번 라운드가 harness design 원칙의 **첫 실전 테스트**였고 성공적으로 동작: +- Generator와 Evaluator가 **같은 세션 아님** → 자기 평가 편향 차단 +- Normalizer v1 Generator가 자진 flag한 3개 partial 중 2개를 Evaluator가 실제로 **fail 판정** → Generator 자체 판단만으로는 부족함 실증 +- Re-iteration이 깔끔히 동작 → 계약 기반 협업이 안정적 + +## 다음 단계 + +P1 UI 자동화: recorder + player (FlaUI 승인 후) diff --git a/docs/history/2026-04-07_이슈3-sut-prober-evaluator.md b/docs/history/2026-04-07_이슈3-sut-prober-evaluator.md new file mode 100644 index 0000000..0426813 --- /dev/null +++ b/docs/history/2026-04-07_이슈3-sut-prober-evaluator.md @@ -0,0 +1,23 @@ +# 2026-04-07 — sut-prober 평가 (Evaluator) + +- **이슈:** #3 (관련: #2 sut-prober 구현) +- **역할:** Evaluator (독립 채점) +- **소요 시간:** 약 10분 +- **Context 사용량:** 약 35K 토큰 + +## 작업 요약 +- `docs/contracts/sut-prober.md` 계약 기반으로 Generator 산출물을 채점. +- `dotnet build recordingtest.sln` → 경고 0 / 오류 0. +- `dotnet run --project src/Recordingtest.SutProber -- --sut "EG-BIM Modeler" --out docs/sut-catalog` → exit 0, plugins 187 / json 16 / assemblies 17. +- 두 번째 실행 후 `git diff` / `git status --porcelain` 모두 비어있음 — 결정성 통과. +- SutProber 소스 grep: 모든 `File.Write*`/`Directory.Create` 호출이 `outDir`(`docs/sut-catalog`) 대상. SUT 폴더 쓰기 없음. +- `plugins.json` 경로는 `EG-BIM Modeler/Plugins/...` 형태로 repo-root 상대, forward slash. +- `assemblies.json`의 `HmEG.dll` `HasPdb=true` 확인. + +## Verdict +**pass** — 9개 DoD 항목 전부 pass. 평가서: `docs/contracts/sut-prober.evaluation.md`. + +## 비고 / 후속 권고 +- JSON 프로퍼티 케이스가 PascalCase (`SizeBytes` 등). 계약 문서 표기인 snake_case와 다르나 evaluator 브리핑이 "size_bytes or equivalent"를 허용하므로 통과 처리. 후속 작업으로 `JsonNamingPolicy.SnakeCaseLower` 적용 검토 권장. +- `JsonConfigScanner`는 루트가 객체일 때만 키를 수집. `CategoryCommands.json`처럼 단일 배열을 감싼 객체는 `CategoryCommands[0]…` 식 합성 키가 보이는데, 이는 스캐너가 아닌 원본 JSON 구조 문제. DoD 위반 아님. +- PROGRESS.md는 호출자가 갱신 (verdict=pass). diff --git a/docs/history/2026-04-07_이슈4-normalizer-evaluator-v2.md b/docs/history/2026-04-07_이슈4-normalizer-evaluator-v2.md new file mode 100644 index 0000000..dea4928 --- /dev/null +++ b/docs/history/2026-04-07_이슈4-normalizer-evaluator-v2.md @@ -0,0 +1,33 @@ +# 2026-04-07 이슈 #4 — normalizer evaluator (v2) + +- 관련 이슈: #4 (normalizer 평가 — 재평가) +- 담당 에이전트: evaluator +- 소요 시간: 약 8분 +- Context 사용량: 약 30k tokens +- Generator iteration: commit `05c7a3f` + +## 작업 요약 +Generator iteration 2 산출물에 대한 독립 재평가. v1에서 fail 처리한 두 항목(DoD #6 sidecar log, DoD #7 catch-all coverage)이 실제로 해소되었는지 엄격 검증. + +## 검증 절차 +1. `dotnet build recordingtest.sln` — 0 warning / 0 error. +2. `dotnet test tests/Recordingtest.Normalizer.Tests` — **10 passed / 0 failed / 0 skipped** (167 ms). +3. `Normalizer.cs` 검토 — `Normalize(input, profile, sidecarPath?)` overload 존재. `sidecarPath`가 디렉터리면 `normalization.log`로, 파일 경로면 그대로 기록. 형식 `{RuleId}\tcount={Count}` Ordinal 정렬 + `total=N` 라인 일치. +4. `RuleTests.Normalize_WritesSidecarLogFile`, `Normalize_SidecarPath_AcceptsDirectory` — 실제 임시 파일 생성 후 파일 존재/내용/정렬/total 라인을 모두 단언. 모킹 아님. +5. `CoverageTests.cs` — `|| true` 제거 확인. 18개 엔트리의 명시적 `Dictionary FieldRuleMap`(StringComparer.Ordinal)로 교체. 매핑 누락 또는 매핑된 규칙이 default 프로파일에 없으면 실패. +6. `profiles/default.yaml` — 6 rules: `mask_volatile_settings` 포함. +7. `Rules.cs` — `MaskVolatileSettings`가 stub 아닌 실제 구현(allowlist HashSet 16개, JsonNode 재귀 walk, `` 치환, count 반환). +8. 건너뛴 테스트 0건 확인. + +## 결과 +- Verdict: **pass** +- 모든 DoD 항목 pass. +- 산출물: + - `docs/contracts/normalizer.evaluation.md` (신규 v2 보고서) + - `docs/contracts/normalizer.evaluation.v1.md` (v1 보고서 보존) + +## 비차단 리스크 (Notes에 기록) +`mask_volatile_settings`의 allowlist는 필드명 단독 매칭이므로, 구조적으로 무관한 서브트리에 동일 이름 필드가 있으면 실제 버그를 마스킹할 수 있음. 현재 16개로 좁아 수용 가능. 카탈로그 확장 시 JSON path 스코핑 도입 권장. 후속 이슈에서 `normalizer.md`에 명시 권장. + +## 후속 +Generator/Planner가 PROGRESS.md를 갱신할 수 있음. Evaluator는 PROGRESS.md를 수정하지 않음. diff --git a/docs/history/2026-04-07_이슈4-normalizer-evaluator.md b/docs/history/2026-04-07_이슈4-normalizer-evaluator.md new file mode 100644 index 0000000..d161e67 --- /dev/null +++ b/docs/history/2026-04-07_이슈4-normalizer-evaluator.md @@ -0,0 +1,26 @@ +# 2026-04-07 이슈 #4 — normalizer evaluator + +- 관련 이슈: #4 (normalizer 평가) +- 담당 에이전트: evaluator +- 소요 시간: 약 10분 +- Context 사용량: 약 35k tokens + +## 작업 요약 +독립 평가자로서 `docs/contracts/normalizer.md` Sprint Contract 기준으로 Generator 산출물(`src/Recordingtest.Normalizer/`, `tests/Recordingtest.Normalizer.Tests/`)을 채점. + +## 검증 절차 +1. `dotnet build recordingtest.sln` — 0 warning / 0 error. +2. `dotnet test tests/Recordingtest.Normalizer.Tests` — 8 passed / 0 failed / 0 skipped. +3. `profiles/default.yaml` 5개 규칙 확인. +4. `Rules.cs` 규칙별 구현 검토 (timestamps, guids, paths, floats, sort). +5. `RuleTests.Normalize_IsIdempotent` 멱등성 테스트 확인. +6. `CoverageTests.DefaultProfile_CoversAllSuspectedFields` 매핑 테스트 검토 — `|| true` 캐치-올 발견. +7. sidecar `normalization.log` 파일 출력 부재 확인. + +## 결과 +- Verdict: **fail** +- DoD #6 (sidecar log) 미구현, DoD #7 (suspected-field 커버리지) partial. +- 산출물: `docs/contracts/normalizer.evaluation.md` + +## 후속 +Generator 재작업 필요. PROGRESS.md 갱신 보류. diff --git a/docs/history/2026-04-07_이슈5-diff-reporter-evaluator.md b/docs/history/2026-04-07_이슈5-diff-reporter-evaluator.md new file mode 100644 index 0000000..5febd5d --- /dev/null +++ b/docs/history/2026-04-07_이슈5-diff-reporter-evaluator.md @@ -0,0 +1,20 @@ +# 2026-04-07 diff-reporter Evaluator + +- 관련 이슈: #5 (#2 diff-reporter 평가) +- 작업: Sprint Contract `docs/contracts/diff-reporter.md` 기준 독립 평가 +- 소요 시간: 약 8분 +- Context 사용량: 약 35k tokens + +## 결과 +- Verdict: **pass** +- DoD 8개 중 7 pass / 1 partial(diff-triager 통합 테스트 부재) +- `dotnet build recordingtest.sln` 0/0 +- `dotnet test tests/Recordingtest.DiffReporter.Tests` 5 passed / 0 failed +- CLI smoke: 동일 → exit 0 `identical`, 1필드 차이 → exit 1, hunks.length=1, 스키마 일치 + +## 산출물 +- `docs/contracts/diff-reporter.evaluation.md` +- 본 히스토리 파일 + +## 후속 +- diff-triager 에이전트 통합 테스트 1건 추가 권장 (DoD #8 partial 해소)