wip: phase_z2 evidence 파이프라인 + matching 실험(phase2~26) + 프론트 trace 패널 진행분 스냅샷

- src: phase_z2 composition/mapper/pipeline/placement_planner/retry, ai_fallback(prompts/schema/validate), mdx_text_atoms 신규 - Front: PipelineTracePanel 신규, FramePanel/SlideCanvas/Home/designAgentApi 등 갱신 + 테스트 4종 추가 - templates/phase_z2: catalog(component_expansion_registry, node_slot_mapping 신규), frames, families, slide_base 갱신 - tests/matching: phase2~26 매칭 실험 스크립트·리포트·온톨로지 전체 (미커밋 진행분) - tests: b4_v4 evidence, task5~28.5 시리즈, regression(imp95 baseline) 등 신규 테스트 대량 추가 - docs/reference: MDX 구조 인벤토리, MDX→Frame 구조 계약 문서 - scripts: mdx 계약/parity/coverage/viewport 체크, gitea comment, run sync 유틸 - .gitignore: tmp*.json, chromedriver, .orchestrator, *.pkl, Front_test* 등 임시/스냅샷 제외 미완성 작업의 보존용 스냅샷 커밋 (2026-07-02) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 17:03:42 +09:00
parent 97b7833a1b
commit b836e79ee1
527 changed files with 673036 additions and 717 deletions
@@ -0,0 +1,133 @@
+"""Pipeline Step 15 — IDF 기반 vs 현재 0.30/0.50/0.20 가중치 랭킹 비교.
+
+질문: 현재 0.30/0.50/0.20 고정 가중치 대신 TF-IDF 원리로 계산하면
+     프레임 랭킹이 같게 나오는가?
+
+방법:
+  1. 각 프레임의 개별 키워드 (핵심+연관) + 복합 키워드(세트) 모으기
+  2. 개별 토큰 IDF = log(32 / frame_df(token))
+  3. 복합 토큰 IDF = log(32 / "그 모든 키워드를 다 가진 프레임 수")
+  4. 매칭 점수 = Σ IDF × match / Σ IDF
+     - 개별 토큰: match = 1 if 등장 else 0
+     - 복합 토큰: match = coverage (부분 일치 허용)
+  5. 기존 0.30/0.50/0.20 랭킹과 비교
+"""
+from collections import defaultdict
+from math import log
+from pathlib import Path
+import yaml
+
+HERE = Path(__file__).parent
+N_FRAMES = 32
+
+
+def main():
+    auto = yaml.safe_load((HERE / 'auto_anchor_candidates.yaml').read_text(encoding='utf-8'))
+    v1 = yaml.safe_load((HERE / 'mdx_matching_result.yaml').read_text(encoding='utf-8'))
+
+    # ─── 1. 프레임별 개별 토큰 집합 ───
+    frame_tokens = defaultdict(set)
+    token_frame_df = {}
+    for set_id, s in auto['source_text_sets'].items():
+        fid = s['frame_id']
+        for t in s.get('terms', []):
+            tok = t['token']
+            frame_tokens[fid].add(tok)
+            if tok not in token_frame_df:
+                token_frame_df[tok] = t.get('frame_df', 1)
+
+    # ─── 2. 프레임별 복합 토큰 (세트) ───
+    frame_compounds = defaultdict(list)
+    for set_id, s in auto['source_text_sets'].items():
+        fid = s['frame_id']
+        comp = tuple(sorted(set(s['term_values'])))
+        if comp:
+            frame_compounds[fid].append(comp)
+
+    # ─── 3. 복합 토큰 df (모든 키워드를 포함하는 프레임 수) ───
+    all_compounds = set()
+    for comps in frame_compounds.values():
+        all_compounds.update(comps)
+    compound_df = {}
+    for comp in all_compounds:
+        df = sum(1 for fid, tokens in frame_tokens.items() if all(k in tokens for k in comp))
+        compound_df[comp] = max(df, 1)
+
+    # ─── 4. IDF 계산 ───
+    idf_individual = {t: log(N_FRAMES / df) for t, df in token_frame_df.items() if df}
+    idf_compound = {c: log(N_FRAMES / df) for c, df in compound_df.items()}
+
+    # ─── 5. 각 MDX 섹션 × 프레임 IDF-based 점수 ───
+    print('=' * 80)
+    print(f'{"섹션":<8} | {"순위":<4} | {"현재 (0.30/0.50/0.20)":<30} | {"IDF-based":<30}')
+    print('=' * 80)
+
+    agreement_top1 = 0
+    agreement_top3 = 0
+    total_sections = 0
+
+    for sid, sec in v1['mdx_sections'].items():
+        idf_scores = {}
+        for frame_id, detail in sec['per_frame_detail'].items():
+            # 개별 토큰 (핵심 + 연관)
+            individual_tokens = frame_tokens.get(frame_id, set())
+            standalone_hit = set(detail['standalone']['hit'])
+            related_hit = set(detail['related']['hit'])
+            all_hits = standalone_hit | related_hit
+
+            total_idf = 0.0
+            matched_idf = 0.0
+
+            for tok in individual_tokens:
+                idf = idf_individual.get(tok, 0)
+                total_idf += idf
+                if tok in all_hits:
+                    matched_idf += idf
+
+            # 복합 토큰 (세트)
+            for comp in frame_compounds.get(frame_id, []):
+                idf = idf_compound.get(comp, 0)
+                total_idf += idf
+                # coverage 를 per_frame_detail.keyword_group.groups 에서 찾기
+                coverage = 0
+                for g in detail.get('keyword_group', {}).get('groups', []):
+                    g_comp = tuple(sorted(set(g['keywords'])))
+                    if g_comp == comp:
+                        coverage = g.get('coverage', 0)
+                        break
+                matched_idf += idf * coverage
+
+            score = matched_idf / total_idf if total_idf > 0 else 0
+            idf_scores[frame_id] = (score, detail['frame_number'])
+
+        idf_ranking = sorted(idf_scores.items(), key=lambda x: -x[1][0])
+        current_ranking = sec['rank_by_matching_score']
+
+        total_sections += 1
+        if current_ranking[0]['frame_id'] == idf_ranking[0][0]:
+            agreement_top1 += 1
+        current_top3 = {r['frame_id'] for r in current_ranking[:3]}
+        idf_top3 = {x[0] for x in idf_ranking[:3]}
+        if current_top3 == idf_top3:
+            agreement_top3 += 1
+
+        for i in range(5):
+            cur = current_ranking[i]
+            idf_r = idf_ranking[i]
+            cur_s = f'Frame {cur["frame_number"]:>2}  {cur["matching_score"]:.3f}'
+            idf_s = f'Frame {idf_r[1][1]:>2}  {idf_r[1][0]:.3f}'
+            if i == 0:
+                print(f'{sid:<8} | {i+1:<4} | {cur_s:<30} | {idf_s:<30}')
+            else:
+                print(f'{"":<8} | {i+1:<4} | {cur_s:<30} | {idf_s:<30}')
+        print('-' * 80)
+
+    print()
+    print('=' * 80)
+    print(f'Top-1 일치: {agreement_top1}/{total_sections}')
+    print(f'Top-3 집합 일치: {agreement_top3}/{total_sections}')
+    print('=' * 80)
+
+
+if __name__ == '__main__':
+    main()