From f39b093106a12acc06b436c2cf86fa864018bbff Mon Sep 17 00:00:00 2001 From: nbright Date: Mon, 24 Aug 2026 09:07:31 +0900 Subject: [PATCH] Record results so far and fix the memory blowup in the palette decode Adds STATUS.md as the handoff document: benchmark numbers, the bare-earth metrics that actually matter for this project, the Korean-data domain gap that retraining will not fix, and what to do on the 24 GB machine. The memory problem was in how a prediction's colours were turned back into class indices. Every consumer built an (N, 13, 3) float64 temporary: d = ((rgb[:, None, :] - COLOR_MAP[None, :, :]) ** 2).sum(axis=2) That is ~250 MB of intermediates per 800k-point tile, several live at once, and a full 4.7M-point block pushes it into gigabytes. main.py writes exact palette entries, so an exact hash lookup resolves nearly every point with no large temporary; only leftovers fall back to a chunked distance search. Peak RSS on a 470k-point tile drops to 61 MB. Extracted to sumparts_palette.py and shared by coarse_eval.py and split_by_class.py. Also from this round: - patch_cm_mutation.sh: ConfusionMatrix.update() rewrote the caller's pred tensor in place, folding every ignore_index point into class num_classes-1. test() saves its visualization from that same tensor afterwards, so an unlabelled tile came out 100% wall and the model looked degenerate when it was not. - patch_class_mask.sh: SUMPARTS_MASK_CLASSES drops known-absent classes from the argmax. Measured on Seosan and it does not help - the runner-up for "water" is "wall", not "terrain" - but the experiment is worth keeping. - split_by_class.py now writes .ply alongside .obj. A vertex-only OBJ has zero faces and most viewers render nothing, which is why the first export looked broken. - verify_outputs.sh reads exported files back with a parser, so "here are your files" can be checked rather than asserted. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 5 +- STATUS.md | 246 ++++++++++++++++++++++++++++++++++ scripts/after_train.sh | 51 +++++++ scripts/coarse_eval.py | 93 +++++++++---- scripts/dump_pred.py | 85 ++++++++++++ scripts/dump_seosan_pred.sh | 21 +++ scripts/eval_coarse.sh | 9 +- scripts/export_pred_viewer.sh | 48 +++++++ scripts/final_eval.sh | 10 +- scripts/mask_experiment.sh | 67 +++++++++ scripts/patch_class_mask.sh | 81 +++++++++++ scripts/patch_cm_mutation.sh | 80 +++++++++++ scripts/poc_check_pred.sh | 5 +- scripts/poc_infer.sh | 10 +- scripts/rescore.sh | 27 ++++ scripts/resolve_ckpt.sh | 52 +++++++ scripts/split_by_class.py | 124 +++++++++++++++++ scripts/sumparts_palette.py | 131 ++++++++++++++++++ scripts/verify_outputs.sh | 45 +++++++ 19 files changed, 1144 insertions(+), 46 deletions(-) create mode 100644 STATUS.md create mode 100644 scripts/after_train.sh create mode 100644 scripts/dump_pred.py create mode 100644 scripts/dump_seosan_pred.sh create mode 100644 scripts/export_pred_viewer.sh create mode 100644 scripts/mask_experiment.sh create mode 100644 scripts/patch_class_mask.sh create mode 100644 scripts/patch_cm_mutation.sh create mode 100644 scripts/rescore.sh create mode 100644 scripts/resolve_ckpt.sh create mode 100644 scripts/split_by_class.py create mode 100644 scripts/sumparts_palette.py create mode 100644 scripts/verify_outputs.sh diff --git a/README.md b/README.md index ef4dc5f..a04de05 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# sum-parts-test +# sum-parts-test 드론 사진측량 메시를 **건물 / 수목 / 차량 / 지면**으로 분할하기 위한 [SUM Parts](https://github.com/tudelft3d/SUM-Parts-Benchmarks) (CVPR 2025) 재현 및 적용 작업. @@ -9,7 +9,8 @@ | 경로 | 내용 | |---|---| -| [SETUP.md](SETUP.md) | **1단계 — 환경 구축 (GPU 불필요). 여기부터** | +| [STATUS.md](STATUS.md) | **현재 상태·결과·미해결 문제 — 이어받을 때 여기부터** | +| [SETUP.md](SETUP.md) | 1단계 — 환경 구축 (GPU 불필요) | | [TRAIN.md](TRAIN.md) | 2단계 — 학습 (GPU 필요) | | [docs/pipeline.html](docs/pipeline.html) | 전체 6단계 공정 정의 (브라우저로 열 것) | | [docs/SUM-Parts-검토노트.md](docs/SUM-Parts-검토노트.md) | 트러블슈팅 16건, 데이터 스키마 실측, 라이선스 | diff --git a/STATUS.md b/STATUS.md new file mode 100644 index 0000000..9b2b361 --- /dev/null +++ b/STATUS.md @@ -0,0 +1,246 @@ +# 현재 상태 — 2026-08-21 + +3060 머신에서 여기까지 왔다. 이 문서는 **3090으로 옮겨서 이어가기 위한 인수인계**다. + +목표: 드론 사진측량 메시에서 **구조물을 제거한 지형(bare earth)** 추출. +부수적으로 건물·수목·차량 분리. + +--- + +## 한 줄 요약 + +SUM Parts 벤치마크는 재현했다. **지면 precision 95.24%로 목표 용도에 충분하다.** +문제는 벤치마크가 아니라 **한국 데이터에서의 도메인 갭**이다. + +--- + +## 1. 완료된 것 + +| 항목 | 결과 | +|---|---| +| WSL2 환경 (CUDA 11.8 + torch 2.0.1 + 확장 5종) | ✅ | +| SUM Parts 데이터 (train 24 / val 8 / test 8) | ✅ | +| PointNet 100 epoch | mIoU 17.19 (논문 15.1 — **재현 확인**) | +| **PointVector 100 epoch** | **mIoU 51.83 / OA 80.41** | +| 서산 OBJ → PLY 변환 | ✅ 타일 1장 | +| 서산 추론 | ✅ 13클래스 예측 | +| 클래스별 분리 출력 | ✅ `D:\AI_Test\sum-part\` | + +--- + +## 2. SUM 벤치마크 성적 (PointVector, voxel_max 24000) + +### 4클래스 통합 + +| 클래스 | IoU | precision | recall | +|---|---|---|---| +| ground | 60.69% | **95.24%** | 62.59% | +| vegetation | 90.70% | 95.11% | 95.14% | +| building | 86.10% | 88.01% | 97.54% | +| vehicle | 51.05% | 78.81% | 59.17% | +| **mIoU 72.14% / OA 89.72%** | | | | + +### bare earth (지면 vs 나머지) ← 실제 목표 지표 + +``` +지면 정확히 유지 523,983 +비지면 섞여 들어옴 26,217 ← 지형을 오염시킴 +지면 놓침 268,755 ← 구멍, 보간 가능 + +precision 95.24% +recall 66.10% +IoU 63.98% +F1 78.04% +``` + +**오염원 분해** + +| 출처 | 비중 | +|---|---| +| facade_surface | **49.90%** ← 절반이 건물 외벽 | +| high_vegetation | 17.42% | +| water | 12.04% | +| car | 7.83% | +| wall (옹벽) | 7.54% | +| roof_surface | 5.05% | + +건물 외벽 하단 — 지면과 만나는 경계에서 새는 것으로 보인다. +**후처리로 상당 부분 잡을 수 있다**: 지면 예측 중 수직으로 튀는 포인트 제거. + +> precision 95.24%는 목표 용도(구조물 제거 지형도)에 **이미 쓸 만하다.** +> recall 66.10%로 지면 34%를 놓치지만, 구멍은 보간으로 메운다. + +--- + +## 3. ★ 미해결 — 한국 데이터 도메인 갭 + +서산 타일 1장 추론 결과: + +| 클래스 | 비율 | 판정 | +|---|---|---| +| terrain | 24.29% | 타당 | +| **unclassified** | **21.70%** | ❌ 모델이 판단 못 함 | +| **water** | **21.11%** | ❌ **물 없다고 확인됨** | +| high_vegetation | 7.85% | 타당 | +| wall | 6.48% | ? | +| roof_surface | 6.11% | ? | +| **boat** | **5.07%** | ❌ **배 있을 리 없다** | +| facade_surface | 0.10% | ❌ 지붕 6%인데 벽 0.1%는 모순 | + +**신뢰 불가 47.9%.** + +### 원인 + +닫힌 집합 분류기다. 13개 중 반드시 하나를 고르고 **"모르겠음"이 없다.** +학습 어휘에 없는 것을 만나면 특징 공간에서 가장 가까운 것으로 간다. + +헬싱키에서 배운 `water` = 어둡고 평평하고 균질하고 수평. 서산 포장면이 그 설명에 맞는다. +`boat`는 물 위에 있는 것이라 따라온다. **일관성 있는 착각이다.** + +비율 매칭은 아니다 — SUM val의 water는 11.17%인데 서산 예측은 21.11%로 2배다. + +### 시도했고 실패한 것: 클래스 마스킹 + +`water`·`boat` 로짓을 `-inf`로 죽여 2순위로 넘기는 실험. + +``` +wall +63,783 (6.48% → 20.05%) ← 절반 이상이 여기로 +unclassified +33,867 +terrain +13,656 ← 11%만 회복 +``` + +**실패.** 2순위가 `terrain`이 아니라 `wall`이었다. +모델이 그 영역을 수직 구조물로 읽고 있다는 뜻이고, 도메인 갭이 얕지 않다는 증거다. +마스킹은 되돌렸다 (`SUMPARTS_MASK_CLASSES` 환경변수로 언제든 다시 켤 수 있다). + +--- + +## 4. 발견한 업스트림 버그 4건 (전부 패치됨, 멱등, 원본 보존) + +| # | 증상 | 원인 | 패치 | +|---|---|---|---| +| A | `bincount ... non-negative` | 블라인드 test셋 라벨이 전부 `-1` | `patch_unlabeled_test.sh` | +| B | `UnboundLocalError: 'epoch'` | `mode=val`이 학습 루프 변수 참조 | `patch_val_mode.sh` | +| C | `np.long` / `collections.Iterable` | numpy 1.24 / python 3.10에서 제거·이동 | `patch_numpy_aliases.sh` | +| D | **예측이 전부 한 클래스로 나옴** | `ConfusionMatrix.update()`가 **입력을 제자리 변조** | `patch_cm_mutation.sh` | + +**D가 가장 위험했다.** `ignore_index`에 해당하는 포인트의 예측을 +`num_classes-1`(=wall)로 덮어쓰는데, `main.py`가 그 다음에 시각화를 저장한다. +라벨이 없는 우리 타일은 전 포인트가 `ignore_index`라 **100% wall로 저장됐다.** +모델이 퇴화한 것처럼 보였지만 멀쩡했다. + +--- + +## 5. 메모리 문제 (3090에서 겪은 것) + +팔레트 색상 → 클래스 복원에서 `(N, 13, 3)` 배열을 통째로 만들고 있었다. + +```python +d = ((rgb[:, None, :] - COLOR_MAP[None, :, :]) ** 2).sum(axis=2) +``` + +80만 포인트면 임시 배열 하나가 250 MB이고 여러 개가 동시에 존재한다. +전체 블록(470만 포인트)이면 GB 단위로 뛴다. + +**수정 완료** — `sumparts_palette.py`로 분리하고 해시 조회 + 청크 폴백으로 바꿨다. + +``` +개선 후 peak RSS: 61 MB (470k 포인트 처리) +``` + +`coarse_eval.py`, `split_by_class.py`가 이 모듈을 쓴다. + +--- + +## 6. 실측 성능표 (RTX 3060 12GB) + +| 모델 | voxel_max | peak VRAM | s/iter | 100 epoch | 논문 mIoU | +|---|---|---|---|---|---| +| pointnet | 64000 | 6.01 G | 0.291 | 2.9h | 15.1% | +| pointnet++msg | 64000 | 4.16 G | 0.675 | 6.8h | 33.1% | +| pointvector-xl | 24000 | 6.46 G | 0.402 | **4.1h** | 70.0% | +| pointvector-xl | 64000 | **16.49 G** | 13.113 | ❌ 12GB 불가 | | +| pointnext-xl | 32000 | 8.03 G | 0.635 | 6.4h | 65.3% | +| pointnext-xl | 64000 | **15.47 G** | 46.980 | ❌ 12GB 불가 | | + +> **WSL2에서 VRAM 초과는 OOM을 내지 않는다.** 호스트 RAM으로 흘려서 +> 조용히 완주한다 — 25~100배 느리게. peak VRAM과 **전력(W)**으로 판정할 것. +> 사용률 100%인데 전력이 낮으면 연산이 아니라 PCIe 전송 대기다. + +--- + +## 7. 3090에서 할 일 + +### 우선순위 1 — 논문 설정 재학습 + +`voxel_max 64000`은 16.5 GB가 필요해 12 GB 카드에서 불가능했다. 24 GB면 된다. + +기대 효과: + +| | 현재 (24000) | 재학습 (64000) 기대 | +|---|---|---| +| SUM terrain IoU | 60.69% | ~85% | +| 지면 precision | 95.24% | ~96% | +| 지면 recall | 66.10% | ~85% | +| **서산 물 오분류 21%** | — | **그대로** | + +**주의**: `voxel_max`는 중립적 손잡이가 아니다. 64000으로 학습하면 +**추론도 64000 청크로 해야** 성능이 나온다 → 서산 추론도 3090에서 돌려야 한다. + +### 우선순위 2 — 서산 예측 육안 확인 + +`D:\AI_Test\sum-part\unseen\*.ply`에 42.82%가 들어 있다. +**이게 뭔지 모르면 나머지가 전부 추측이다.** + +### 우선순위 3 — 도메인 갭 대응 + +재학습으로 안 고쳐진다. 선택지: + +1. **SAM 3.1 자동 라벨링 + 파인튜닝** — 드론 원본 8,120장 + 카메라 포즈(PPK) 보유. + SAM 3의 텍스트 프롬프트로 명명된 마스크 → 포즈로 3D 투영 → 면 투표. + 사람 몫은 전수 라벨링이 아니라 **1~2타일 검증**이다. +2. **후처리** — 지면 예측 중 수직 이상치 제거로 건물 외벽 오염(49.9%) 감소. + +--- + +## 8. 산출물 위치 + +``` +D:\AI_Test\sum-part\ +├── ground\ 114,159 pts (24.29%) +├── building\ 93,076 pts (19.80%) +├── tree\ 36,876 pts ( 7.85%) +├── vehicle\ 24,631 pts ( 5.24%) +└── unseen\ 201,258 pts (42.82%) ← unclassified + water +``` + +각 폴더에 `.ply`와 `.obj`가 있다. **`.ply`를 열어라** — +OBJ는 면이 없어서 대부분의 뷰어가 아무것도 안 보여준다. +색상은 클래스 색이 아니라 **드론 사진 텍스처**라, 그 클래스로 분류된 게 실제로 뭔지 보인다. + +``` +D:\MYCLAUDE_PROJECT\sum-parts-test\output\ +├── seosan_pred_viewer.ply 클래스 색 +└── seosan_rgb_viewer.ply 사진 색 +``` + +--- + +## 9. 제약 + +- **test 세트는 블라인드다.** 라벨이 전부 `-1`이라 로컬 채점 불가. + 논문 수치와 직접 대조하려면 예측을 저자(gaoweixiaocuhk@gmail.com)에게 보내야 한다. +- **어노테이션 도구는 비공개다.** 저자가 유료 서비스로 판매 중. +- **학습 가중치도 비공개다.** 직접 학습이 유일한 경로. +- **라이선스**: 데이터 CC BY-NC 4.0, 코드 GPL-3.0. 상업 이용은 저자 허락 필요. + +--- + +## 10. 문서 + +| 문서 | 용도 | +|---|---| +| [SETUP.md](SETUP.md) | 1단계 — 환경 구축 (GPU 불필요, ~80분) | +| [TRAIN.md](TRAIN.md) | 2단계 — 학습 (GPU 필요) | +| [docs/pipeline.html](docs/pipeline.html) | 전체 6단계 공정 정의 | +| [docs/SUM-Parts-검토노트.md](docs/SUM-Parts-검토노트.md) | 트러블슈팅 상세, 데이터 스키마 실측 | diff --git a/scripts/after_train.sh b/scripts/after_train.sh new file mode 100644 index 0000000..f442d80 --- /dev/null +++ b/scripts/after_train.sh @@ -0,0 +1,51 @@ +#!/usr/bin/env bash +# SUM Parts - wait for training to finish, then run every evaluation we care about +# +# Runs unattended so nobody has to sit watching for the last epoch: +# 1. wait for the trainer to exit +# 2. coarse evaluation on SUM val -> building / vegetation / vehicle / ground +# 3. inference on the Seosan tile -> does our own data work with this model +set -uo pipefail + +SCRIPTS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +OUT="$HOME/sum-parts/runs/after_train" +LOG="$OUT/after_train.log" + +mkdir -p "$OUT" +exec > >(tee -a "$LOG") 2>&1 + +say() { echo "[$(date '+%F %T')] $*"; } + +say "waiting for training to finish" +waited=0 +while pgrep -f 'main.py --cfg' > /dev/null; do + sleep 30 + waited=$((waited + 30)) + [ $((waited % 300)) -eq 0 ] && say " still running (${waited}s)" + [ "$waited" -gt 7200 ] && { say " timed out after 2h"; break; } +done +say "trainer no longer running" + +# also wait out the watchdog, so it does not relaunch under us +pkill -f train_watchdog.sh 2>/dev/null && say "stopped the watchdog" +sleep 5 + +echo +say "=== best checkpoint ===" +CKPT=$(find "$HOME/sum-parts/semantic_segmentation/PointNeXt_bundle/examples/segmentation/log/sumv2_triangle" \ + -name '*pointvector*_ckpt_best.pth' -printf '%T@ %p\n' 2>/dev/null \ + | sort -rn | head -1 | cut -d' ' -f2-) +say "$CKPT" +grep -ahE 'Best ckpt @E' "$HOME/sum-parts/runs/pointvector-xl_"*/train.log 2>/dev/null | tail -2 + +echo +say "=== 1/2 coarse evaluation on SUM val (4 classes) ===" +TEST_VOXEL_MAX=24000 bash "$SCRIPTS/eval_coarse.sh" || say "eval_coarse returned non-zero" + +echo +say "=== 2/2 inference on the Seosan tile ===" +bash "$SCRIPTS/poc_infer.sh" || say "poc_infer returned non-zero" +bash "$SCRIPTS/poc_check_pred.sh" || say "poc_check_pred returned non-zero" + +echo +say "AFTER TRAIN DONE -- log: $LOG" diff --git a/scripts/coarse_eval.py b/scripts/coarse_eval.py index a8ee0d0..9eec39e 100644 --- a/scripts/coarse_eval.py +++ b/scripts/coarse_eval.py @@ -37,7 +37,9 @@ import sys from pathlib import Path import numpy as np -from plyfile import PlyData + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from sumparts_palette import read_point_classes # noqa: E402 FINE = ['unclassified', 'terrain', 'high_vegetation', 'facade_surface', 'water', 'car', 'boat', 'roof_surface', 'chimney', 'dormer', @@ -73,28 +75,14 @@ COLOR_MAP = np.array([ def load_labels(path: Path) -> np.ndarray: - """Fine class per point: from a `label` field, else decoded from colour.""" - v = PlyData.read(str(path))["vertex"] - props = [p.name for p in v.properties] + """Fine class per point. - if "label" in props: - lab = np.asarray(v["label"]).astype(np.int64).ravel() - # a prediction ply may carry a placeholder label; fall through if so - if lab.max() >= 0 and not (lab == lab[0]).all(): - return lab - if lab.max() >= 0 and lab[0] >= 0: - return lab - - rgb_set = next((s for s in (("red", "green", "blue"), ("r", "g", "b")) - if all(c in props for c in s)), None) - if rgb_set is None: - raise SystemExit(f"{path}: no label field and no colour to decode") - - rgb = np.stack([np.asarray(v[c], dtype=np.float64) for c in rgb_set], axis=1) - if rgb.max() <= 1.0: - rgb *= 255.0 - d = ((rgb[:, None, :] - COLOR_MAP[None, :, :]) ** 2).sum(axis=2) - return d.argmin(axis=1).astype(np.int64) + Delegates to sumparts_palette, whose colour decode avoids building an + (N, 13, 3) temporary -- that pattern needs ~250 MB of intermediates per + 800k-point tile and takes the machine down on a full block. + """ + _, cls = read_point_classes(path) + return cls.astype(np.int64) def confusion(pred: np.ndarray, true: np.ndarray, n: int) -> np.ndarray: @@ -104,19 +92,24 @@ def confusion(pred: np.ndarray, true: np.ndarray, n: int) -> np.ndarray: def report(cm: np.ndarray, names: list[str], skip: set[int]) -> None: tp = np.diag(cm).astype(np.float64) - actual = cm.sum(axis=1).astype(np.float64) - predicted = cm.sum(axis=0).astype(np.float64) + actual = cm.sum(axis=1).astype(np.float64) # ground truth per class + predicted = cm.sum(axis=0).astype(np.float64) # predictions per class union = actual + predicted - tp - print(f" {'class':<12} {'IoU':>7} {'recall':>8} {'points':>12}") + # precision matters as much as recall here, and for some jobs more. + # Stripping structures off a terrain model is precision-first: a hole where + # ground was missed can be interpolated, but a wall left sitting in the + # surface is a fake landform. + print(f" {'class':<12} {'IoU':>7} {'prec':>8} {'recall':>8} {'points':>12}") ious = [] for i, name in enumerate(names): if i in skip: continue - iou = 100.0 * tp[i] / union[i] if union[i] > 0 else 0.0 - rec = 100.0 * tp[i] / actual[i] if actual[i] > 0 else 0.0 + iou = 100.0 * tp[i] / union[i] if union[i] > 0 else 0.0 + prec = 100.0 * tp[i] / predicted[i] if predicted[i] > 0 else 0.0 + rec = 100.0 * tp[i] / actual[i] if actual[i] > 0 else 0.0 ious.append(iou) - print(f" {name:<12} {iou:>6.2f}% {rec:>7.2f}% {int(actual[i]):>12,}") + print(f" {name:<12} {iou:>6.2f}% {prec:>7.2f}% {rec:>7.2f}% {int(actual[i]):>12,}") scored = [i for i in range(len(names)) if i not in skip] oa_tp = tp[scored].sum() @@ -126,6 +119,44 @@ def report(cm: np.ndarray, names: list[str], skip: set[int]) -> None: print(f" OA : {100.0 * oa_tp / oa_n if oa_n else 0:.2f}%") +def report_bare_earth(cm_fine: np.ndarray) -> None: + """Ground vs everything else, the way a terrain model is actually judged. + + Reported separately from the 4-class view because the failure modes are not + symmetric. Ground missed -> a hole, which interpolation fills. Non-ground + kept -> a retaining wall or a roof baked into the terrain, which nothing + downstream can tell from a real landform. + """ + ground = {1} # terrain + n = cm_fine.shape[0] + rest = [i for i in range(n) if i not in ground and i != 0] + + tp = cm_fine[1, 1] + fp = cm_fine[np.ix_(rest, [1])].sum() # non-ground predicted as ground + fn = cm_fine[np.ix_([1], rest)].sum() # ground predicted as something else + + prec = 100.0 * tp / (tp + fp) if tp + fp else 0.0 + rec = 100.0 * tp / (tp + fn) if tp + fn else 0.0 + iou = 100.0 * tp / (tp + fp + fn) if tp + fp + fn else 0.0 + f1 = 2 * prec * rec / (prec + rec) if prec + rec else 0.0 + + print(f" ground kept correctly {int(tp):>12,}") + print(f" non-ground leaked in {int(fp):>12,} <- contaminates the surface") + print(f" ground missed {int(fn):>12,} <- holes, interpolable") + print() + print(f" precision : {prec:6.2f}% (of what we call ground, how much is)") + print(f" recall : {rec:6.2f}% (of real ground, how much we caught)") + print(f" IoU : {iou:6.2f}%") + print(f" F1 : {f1:6.2f}%") + print() + + if fp: + print(" where the contamination comes from:") + leaks = [(int(cm_fine[i, 1]), FINE[i]) for i in rest if cm_fine[i, 1] > 0] + for cnt, name in sorted(leaks, reverse=True)[:8]: + print(f" {name:<20} {cnt:>10,} {100.0*cnt/fp:>6.2f}% of leak") + + def main() -> None: ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) @@ -171,9 +202,13 @@ def main() -> None: report(cm_fine, FINE, skip={0}) print() - print("=== coarse (what this project needs) ===") + print("=== coarse (building / vegetation / vehicle / ground) ===") report(cm_coarse, COARSE, skip={0}) + print() + print("=== bare earth (ground vs everything else) ===") + report_bare_earth(cm_fine) + if __name__ == "__main__": main() diff --git a/scripts/dump_pred.py b/scripts/dump_pred.py new file mode 100644 index 0000000..9a61d87 --- /dev/null +++ b/scripts/dump_pred.py @@ -0,0 +1,85 @@ +#!/usr/bin/env python3 +"""Dump what a prediction PLY actually contains. + +check_pred.py recovers the class by matching RGB against the palette. If that +reports one class for every point, the question is whether the model really +collapsed or whether the colour decode is wrong — so read the raw fields +instead of interpreting them. + +Usage: + python dump_pred.py pred.ply +""" +from __future__ import annotations + +import sys +from pathlib import Path + +import numpy as np +from plyfile import PlyData + +CLASSES = ['unclassified', 'terrain', 'high_vegetation', 'facade_surface', + 'water', 'car', 'boat', 'roof_surface', 'chimney', 'dormer', + 'balcony', 'roof_installation', 'wall'] + +COLOR_MAP = np.array([ + (0., 0., 0.), (170., 85., 0.), (0., 255., 0.), (255., 255., 0.), + (0., 255., 255.), (255., 0., 255.), (0., 0., 153.), (85., 85., 127.), + (255., 50., 50.), (85., 0., 127.), (50., 125., 150.), (50., 0., 50.), + (215., 160., 140.), +]) + + +def main() -> None: + if len(sys.argv) != 2: + print(__doc__) + raise SystemExit(2) + + path = Path(sys.argv[1]) + v = PlyData.read(str(path))["vertex"] + props = [p.name for p in v.properties] + + print(f"file : {path.name}") + print(f"points: {len(v):,}") + print(f"props : {props}") + print() + + if "label" in props: + lab = np.asarray(v["label"]) + u, c = np.unique(lab, return_counts=True) + print(f"label field: dtype={lab.dtype}") + for k, n in sorted(zip(u.tolist(), c.tolist()), key=lambda t: -t[1]): + name = CLASSES[k] if 0 <= k < len(CLASSES) else f"?{k}" + print(f" {k:>3} {name:<20} {n:>10,} {100*n/len(lab):>6.2f}%") + print() + + rgb_set = next((s for s in (("red", "green", "blue"), ("r", "g", "b")) + if all(c in props for c in s)), None) + if rgb_set: + rgb = np.stack([np.asarray(v[c]) for c in rgb_set], axis=1) + uniq, cnt = np.unique(rgb.reshape(-1, 3), axis=0, return_counts=True) + print(f"colour field {rgb_set}: dtype={rgb.dtype}, {len(uniq)} distinct") + order = np.argsort(-cnt) + for i in order[:15]: + col = uniq[i] + d = ((COLOR_MAP - col.astype(np.float64)) ** 2).sum(axis=1) + k = int(d.argmin()) + exact = "exact" if d[k] < 1 else f"nearest (dist {np.sqrt(d[k]):.0f})" + name = CLASSES[k] if k < len(CLASSES) else f"?{k}" + print(f" {tuple(int(x) for x in col)!s:<20} {cnt[i]:>10,} " + f"{100*cnt[i]/len(rgb):>6.2f}% -> {name} ({exact})") + + print() + if "label" in props and rgb_set: + lab = np.asarray(v["label"]) + rgb = np.stack([np.asarray(v[c]) for c in rgb_set], axis=1) + d = ((rgb[:, None, :].astype(np.float64) - COLOR_MAP[None, :, :]) ** 2).sum(axis=2) + from_colour = d.argmin(axis=1) + agree = int((from_colour == lab).sum()) + print(f"label vs colour agreement: {agree:,}/{len(lab):,} " + f"({100*agree/len(lab):.2f}%)") + if agree < len(lab): + print(" -> the two disagree; the label field is authoritative") + + +if __name__ == "__main__": + main() diff --git a/scripts/dump_seosan_pred.sh b/scripts/dump_seosan_pred.sh new file mode 100644 index 0000000..5e4f66b --- /dev/null +++ b/scripts/dump_seosan_pred.sh @@ -0,0 +1,21 @@ +#!/usr/bin/env bash +# SUM Parts - dump the raw contents of the newest Seosan prediction +set -uo pipefail + +SCRIPTS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +LOGROOT="$HOME/sum-parts/semantic_segmentation/PointNeXt_bundle/examples/segmentation/log" + +source "$HOME/miniconda3/etc/profile.d/conda.sh" +conda activate sumparts + +PRED=$(find "$LOGROOT" -name 'seosan*_pred.ply' -printf '%T@ %p\n' 2>/dev/null \ + | sort -rn | head -1 | cut -d' ' -f2-) + +if [ -z "$PRED" ]; then + echo "no seosan prediction found under $LOGROOT" + exit 1 +fi + +echo "path: $PRED" +echo +python "$SCRIPTS/dump_pred.py" "$PRED" diff --git a/scripts/eval_coarse.sh b/scripts/eval_coarse.sh index 7e3f84d..0458ecd 100644 --- a/scripts/eval_coarse.sh +++ b/scripts/eval_coarse.sh @@ -20,9 +20,10 @@ export PYTORCH_CUDA_ALLOC_CONF="garbage_collection_threshold:0.7,max_split_size_ mkdir -p "$OUT" cd "$SEG" -CKPT=$(find "$SEG/log/sumv2_triangle" -name '*_ckpt_best.pth' -printf '%T@ %p\n' \ - | sort -rn | head -1 | cut -d' ' -f2-) -echo "checkpoint: $(basename "$CKPT")" +# The cfg has to match the architecture that wrote the checkpoint. Hardcoding +# pointnet.yaml here meant a PointVector checkpoint loaded into the wrong model: +# RuntimeError: Error(s) in loading state_dict for BaseSeg +source "$SCRIPTS/resolve_ckpt.sh" # test() writes prediction plys and slides over whole tiles; point it at val so # there is ground truth to score against. @@ -40,7 +41,7 @@ VM_ARG=() echo "=== inference over val split (sliding window) ===" echo " test voxel_max: ${TEST_VOXEL_MAX:-null (cfg default)}" python -u main.py \ - --cfg ../../cfgs/sumv2_triangle/pointnet.yaml \ + --cfg "../../cfgs/sumv2_triangle/${CKPT_CFG}.yaml" \ mode=test \ --pretrained_path "$CKPT" \ dataset.common.data_root="$DATA" \ diff --git a/scripts/export_pred_viewer.sh b/scripts/export_pred_viewer.sh new file mode 100644 index 0000000..feea6e3 --- /dev/null +++ b/scripts/export_pred_viewer.sh @@ -0,0 +1,48 @@ +#!/usr/bin/env bash +# SUM Parts - put the Seosan prediction somewhere it can be opened and looked at +# +# Two files, because they answer different questions: +# *_pred_viewer.ply class colours -> where did each class land +# *_rgb_viewer.ply photo texture -> what is actually there +# +# Open both, flip between them. That is how you find out whether the 21% the +# model calls water is asphalt, shadow, or something else entirely. +set -uo pipefail + +SCRIPTS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +LOGROOT="$HOME/sum-parts/semantic_segmentation/PointNeXt_bundle/examples/segmentation/log/sumv2_triangle" +DEST="/mnt/d/MYCLAUDE_PROJECT/sum-parts-test/output" + +source "$HOME/miniconda3/etc/profile.d/conda.sh" +conda activate sumparts + +mkdir -p "$DEST" + +PRED=$(find "$LOGROOT" -name 'seosan*_pred.ply' -printf '%T@ %p\n' 2>/dev/null \ + | sort -rn | head -1 | cut -d' ' -f2-) +[ -n "$PRED" ] || { echo "no seosan prediction found"; exit 1; } + +echo "prediction : $PRED" +cp "$PRED" "$DEST/seosan_pred_viewer.ply" +echo " -> $DEST/seosan_pred_viewer.ply" + +SRC="$HOME/sum-parts/data/korea_poc/seosan_BlockYBA_tile0.ply" +if [ -f "$SRC" ]; then + python "$SCRIPTS/ply_for_viewer.py" "$SRC" "$DEST/seosan_rgb_viewer.ply" +fi + +echo +echo "=== class colour legend ===" +python - <<'PY' +CLASSES = ['unclassified', 'terrain', 'high_vegetation', 'facade_surface', + 'water', 'car', 'boat', 'roof_surface', 'chimney', 'dormer', + 'balcony', 'roof_installation', 'wall'] +COLORS = [(0,0,0), (170,85,0), (0,255,0), (255,255,0), (0,255,255), + (255,0,255), (0,0,153), (85,85,127), (255,50,50), (85,0,127), + (50,125,150), (50,0,50), (215,160,140)] +for i, (n, c) in enumerate(zip(CLASSES, COLORS)): + print(f" {i:>2} {n:<20} RGB {c}") +PY + +echo +ls -lh "$DEST"/seosan_*viewer.ply diff --git a/scripts/final_eval.sh b/scripts/final_eval.sh index 012f451..fc76d40 100644 --- a/scripts/final_eval.sh +++ b/scripts/final_eval.sh @@ -31,11 +31,11 @@ export PYTORCH_CUDA_ALLOC_CONF="garbage_collection_threshold:0.7,max_split_size_ mkdir -p "$OUT" -CKPT=$(find "$SEG/log/sumv2_triangle" -name '*_ckpt_best.pth' -printf '%T@ %p\n' \ - | sort -rn | head -1 | cut -d' ' -f2-) -[ -n "$CKPT" ] || { echo "error: no checkpoint found" >&2; exit 1; } +# The cfg has to match the architecture that wrote the checkpoint; hardcoding +# one here loads the weights into the wrong model and torch raises on the +# state_dict. +source "$SCRIPTS/resolve_ckpt.sh" -echo "checkpoint: $CKPT" echo "data : $DATA" echo @@ -46,7 +46,7 @@ run_mode() { echo "=== mode=$mode ===" set +e python -u main.py \ - --cfg ../../cfgs/sumv2_triangle/pointnet.yaml \ + --cfg "../../cfgs/sumv2_triangle/${CKPT_CFG}.yaml" \ mode="$mode" \ --pretrained_path "$CKPT" \ dataset.common.data_root="$DATA" \ diff --git a/scripts/mask_experiment.sh b/scripts/mask_experiment.sh new file mode 100644 index 0000000..26b07e1 --- /dev/null +++ b/scripts/mask_experiment.sh @@ -0,0 +1,67 @@ +#!/usr/bin/env bash +# SUM Parts - measure what masking absent classes does to the Seosan prediction +# +# Runs inference twice on the same tile and the same checkpoint: once as-is, +# once with water and boat masked out of the argmax. The interesting number is +# where those 26% of points land when they can no longer be called water. +set -uo pipefail + +SCRIPTS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +OUT="$HOME/sum-parts/runs/mask_experiment" +mkdir -p "$OUT" + +source "$HOME/miniconda3/etc/profile.d/conda.sh" +conda activate sumparts + +# 4 = water, 6 = boat (openpoints/dataset/sumv2_triangle/sumv2_triangle.py) +MASK="${MASK:-4,6}" + +run_and_dump() { + local tag="$1" maskval="$2" + echo + echo "════ $tag ════" + SUMPARTS_MASK_CLASSES="$maskval" bash "$SCRIPTS/poc_infer.sh" \ + > "$OUT/infer_${tag}.log" 2>&1 + local rc=$? + if [ $rc -ne 0 ]; then + echo " inference FAILED rc=$rc" + tail -12 "$OUT/infer_${tag}.log" + return 1 + fi + grep -a 'masking classes' "$OUT/infer_${tag}.log" | head -1 + bash "$SCRIPTS/dump_seosan_pred.sh" 2>&1 | tee "$OUT/dump_${tag}.txt" \ + | sed -n '/label field/,/^$/p' +} + +run_and_dump baseline "" +run_and_dump masked "$MASK" + +echo +echo "════ comparison ════" +python - "$OUT/dump_baseline.txt" "$OUT/dump_masked.txt" <<'PY' +import re +import sys +from pathlib import Path + +def parse(path): + out = {} + for line in Path(path).read_text(encoding="utf-8", errors="replace").splitlines(): + m = re.match(r"\s+(\d+)\s+(\S+)\s+([\d,]+)\s+([\d.]+)%", line) + if m: + out[m.group(2)] = (int(m.group(3).replace(",", "")), float(m.group(4))) + return out + +a, b = parse(sys.argv[1]), parse(sys.argv[2]) +names = sorted(set(a) | set(b), key=lambda n: -max(a.get(n, (0,))[0], b.get(n, (0,))[0])) + +print(f" {'class':<20} {'baseline':>12} {'masked':>12} {'change':>12}") +for n in names: + an, ap = a.get(n, (0, 0.0)) + bn, bp = b.get(n, (0, 0.0)) + d = bn - an + arrow = "" if d == 0 else (" <- absorbed" if d > 0 else "") + print(f" {n:<20} {ap:>10.2f}% {bp:>10.2f}% {d:>+11,}{arrow}") +PY + +echo +echo "logs in $OUT" diff --git a/scripts/patch_class_mask.sh b/scripts/patch_class_mask.sh new file mode 100644 index 0000000..3e8c810 --- /dev/null +++ b/scripts/patch_class_mask.sh @@ -0,0 +1,81 @@ +#!/usr/bin/env bash +# SUM Parts - let inference rule out classes we know are absent +# +# The network is a closed-set classifier: 13 logits, softmax, argmax. There is +# no "none of these". A point whose true class was never in the training +# vocabulary still gets a label - whichever learned concept sits closest in +# feature space. +# +# On the Seosan road tiles that produces 21% water and 5% boat. Neither exists +# there. What the model learned as "water" in Helsinki - dark, flat, smooth, +# horizontal - describes asphalt exactly, and boats are what sits on water, so +# the hallucination is internally consistent. +# +# We know those classes are absent. The model does not. Masking their logits +# before the argmax hands each of those points to its runner-up class instead. +# +# This is not a fix for the domain gap; it is telling the model something we +# already know. Whether it helps depends entirely on what the runner-up is, +# which is why it is worth measuring rather than assuming. +# +# Usage: +# bash scripts/patch_class_mask.sh +# SUMPARTS_MASK_CLASSES=4,6 # 4=water, 6=boat +# +# Idempotent. +set -euo pipefail + +source "$HOME/miniconda3/etc/profile.d/conda.sh" +conda activate sumparts + +REPO="${1:-$HOME/sum-parts/semantic_segmentation/PointNeXt_bundle}" +MAIN="$REPO/examples/segmentation/main.py" + +[ -f "$MAIN" ] || { echo "error: $MAIN not found" >&2; exit 1; } + +if grep -q 'SUMPARTS-CLASS-MASK' "$MAIN"; then + echo "already patched" + exit 0 +fi + +cp -n "$MAIN" "$MAIN.orig" 2>/dev/null || true + +python - "$MAIN" <<'PY' +import sys +from pathlib import Path + +p = Path(sys.argv[1]) +src = p.read_text(encoding="utf-8") + +old = """ pred = all_logits.argmax(dim=1) + if label is not None: + cm.update(pred, label)""" + +new = """ # SUMPARTS-CLASS-MASK: drop classes we know cannot occur in this scene + # before taking the argmax, so their points fall through to the + # runner-up instead. Set SUMPARTS_MASK_CLASSES to a comma-separated + # list of class indices, e.g. "4,6" for water and boat. + _mask = os.environ.get('SUMPARTS_MASK_CLASSES', '').strip() + if _mask: + _idx = [int(x) for x in _mask.split(',') if x.strip() != ''] + if cloud_idx == 0: + logging.info(f' masking classes {_idx} out of the argmax') + all_logits[:, _idx] = float('-inf') + + pred = all_logits.argmax(dim=1) + if label is not None: + cm.update(pred, label)""" + +if old not in src: + print("PATTERN NOT FOUND -- main.py differs from what this patch expects", + file=sys.stderr) + raise SystemExit(1) + +p.write_text(src.replace(old, new), encoding="utf-8") +print("patched:", p) +PY + +python -c "import ast,sys; ast.parse(open(sys.argv[1], encoding='utf-8').read())" "$MAIN" \ + && echo "syntax OK" + +echo "PATCH DONE (original kept at $MAIN.orig)" diff --git a/scripts/patch_cm_mutation.sh b/scripts/patch_cm_mutation.sh new file mode 100644 index 0000000..cb5c55c --- /dev/null +++ b/scripts/patch_cm_mutation.sh @@ -0,0 +1,80 @@ +#!/usr/bin/env bash +# SUM Parts - stop ConfusionMatrix.update from overwriting its caller's predictions +# +# openpoints/utils/metrics.py: +# +# if (true == self.ignore_index).sum() > 0: +# pred[true == self.ignore_index] = self.virtual_num_classes - 1 +# true[true == self.ignore_index] = self.virtual_num_classes - 1 +# +# Folding ignored points into the last bucket is fine for scoring, but it is +# done in place on the tensors the caller passed in. main.py's test() calls +# cm.update(pred, label) and *then* writes the visualization from that same +# pred, so the file on disk is the mutated copy, not what the model predicted. +# +# On a tile labelled entirely with the ignore class - which is exactly what an +# unlabelled tile of our own data looks like, label=0 with ignore_index=0 - +# every single point gets rewritten to class num_classes-1 (wall). The +# prediction file then reads 100% wall no matter what the network actually +# said, and the model looks degenerate when it is not. +# +# Fix: clone before masking. Scoring is unchanged; the caller's tensors survive. +# +# Idempotent. +set -euo pipefail + +source "$HOME/miniconda3/etc/profile.d/conda.sh" +conda activate sumparts + +REPO="${1:-$HOME/sum-parts/semantic_segmentation/PointNeXt_bundle}" +METRICS="$REPO/openpoints/utils/metrics.py" + +[ -f "$METRICS" ] || { echo "error: $METRICS not found" >&2; exit 1; } + +if grep -q 'SUMPARTS-NO-MUTATE' "$METRICS"; then + echo "already patched" + exit 0 +fi + +cp -n "$METRICS" "$METRICS.orig" 2>/dev/null || true + +python - "$METRICS" <<'PY' +import sys +from pathlib import Path + +p = Path(sys.argv[1]) +src = p.read_text(encoding="utf-8") + +old = """ true = true.flatten() + pred = pred.flatten() + if self.ignore_index is not None: + if (true == self.ignore_index).sum() > 0: + pred[true == self.ignore_index] = self.virtual_num_classes -1 + true[true == self.ignore_index] = self.virtual_num_classes -1""" + +new = """ # SUMPARTS-NO-MUTATE: clone before masking. These used to be written + # in place, which silently rewrote the caller's prediction tensor -- + # main.py's test() saves its visualization from the same `pred` right + # after calling this, so the file on disk showed the folded values + # rather than the model's output. On a tile labelled entirely with the + # ignore class every point came out as num_classes-1. + true = true.flatten().clone() + pred = pred.flatten().clone() + if self.ignore_index is not None: + if (true == self.ignore_index).sum() > 0: + pred[true == self.ignore_index] = self.virtual_num_classes -1 + true[true == self.ignore_index] = self.virtual_num_classes -1""" + +if old not in src: + print("PATTERN NOT FOUND -- metrics.py differs from what this patch expects", + file=sys.stderr) + raise SystemExit(1) + +p.write_text(src.replace(old, new), encoding="utf-8") +print("patched:", p) +PY + +python -c "import ast,sys; ast.parse(open(sys.argv[1], encoding='utf-8').read())" "$METRICS" \ + && echo "syntax OK" + +echo "PATCH DONE (original kept at $METRICS.orig)" diff --git a/scripts/poc_check_pred.sh b/scripts/poc_check_pred.sh index e2cb216..b94c57e 100644 --- a/scripts/poc_check_pred.sh +++ b/scripts/poc_check_pred.sh @@ -10,7 +10,10 @@ SCRIPTS="/mnt/d/MYCLAUDE_PROJECT/sum-parts-test/scripts" source "$CONDA_ROOT/etc/profile.d/conda.sh" conda activate "$ENV_NAME" -PRED=$(find "$LOGROOT" -name '*_pred.ply' -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2-) +# Match the Seosan tile by name. Taking simply the newest *_pred.ply picks up +# whatever the last SUM evaluation wrote instead. +PATTERN="${PRED_PATTERN:-*seosan*_pred.ply}" +PRED=$(find "$LOGROOT" -name "$PATTERN" -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2-) if [ -z "$PRED" ]; then echo "error: no *_pred.ply found under $LOGROOT" >&2 exit 1 diff --git a/scripts/poc_infer.sh b/scripts/poc_infer.sh index d7a5d61..eb2a96a 100644 --- a/scripts/poc_infer.sh +++ b/scripts/poc_infer.sh @@ -32,16 +32,16 @@ for split in train val test; do done rm -rf "$TRACK/processed" -CKPT=$(find "$SEG/log/sumv2_triangle" -name '*_ckpt_best.pth' -printf '%T@ %p\n' \ - | sort -rn | head -1 | cut -d' ' -f2-) -[ -n "$CKPT" ] || { echo "error: no checkpoint found -- run smoke_train.sh first" >&2; exit 1; } -echo "checkpoint: $CKPT" +# The cfg has to match the architecture that wrote the checkpoint; hardcoding +# one here loads the weights into the wrong model and torch raises on the +# state_dict. +source "/mnt/d/MYCLAUDE_PROJECT/sum-parts-test/scripts/resolve_ckpt.sh" cd "$SEG" set +e python -u main.py \ - --cfg ../../cfgs/sumv2_triangle/pointnet.yaml \ + --cfg "../../cfgs/sumv2_triangle/${CKPT_CFG}.yaml" \ mode=test \ --pretrained_path "$CKPT" \ dataset.common.data_root="$TRACK" \ diff --git a/scripts/rescore.sh b/scripts/rescore.sh new file mode 100644 index 0000000..7c73f94 --- /dev/null +++ b/scripts/rescore.sh @@ -0,0 +1,27 @@ +#!/usr/bin/env bash +# SUM Parts - re-score existing predictions without re-running inference +# +# The prediction plys are already on disk; only the scoring changed. No GPU, +# takes seconds. +set -uo pipefail + +SCRIPTS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +DATA="$HOME/sum-parts/data/face_labeling/texsp_pcl" +LOGROOT="$HOME/sum-parts/semantic_segmentation/PointNeXt_bundle/examples/segmentation/log/sumv2_triangle" +OUT="$HOME/sum-parts/runs/coarse_eval" + +source "$HOME/miniconda3/etc/profile.d/conda.sh" +conda activate sumparts + +mkdir -p "$OUT" + +VIS=$(find "$LOGROOT" -type d -name visualization -printf '%T@ %p\n' \ + | sort -rn | head -1 | cut -d' ' -f2-) +[ -n "$VIS" ] || { echo "no visualization directory found"; exit 1; } + +echo "predictions: $VIS" +echo "ground truth: $DATA/val" +echo + +python "$SCRIPTS/coarse_eval.py" --pred-dir "$VIS" --gt-dir "$DATA/val" \ + | tee "$OUT/coarse.txt" diff --git a/scripts/resolve_ckpt.sh b/scripts/resolve_ckpt.sh new file mode 100644 index 0000000..889809f --- /dev/null +++ b/scripts/resolve_ckpt.sh @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# SUM Parts - find the newest checkpoint and the cfg that matches it +# +# The eval scripts used to hardcode pointnet.yaml. Point them at a PointVector +# checkpoint and the weights load into the wrong architecture: +# RuntimeError: Error(s) in loading state_dict for BaseSeg +# +# main.py bakes the model name into the run directory, so the cfg can be read +# back out of the checkpoint path instead of guessed. +# +# Source it, don't run it: +# source scripts/resolve_ckpt.sh # newest checkpoint of any model +# CKPT_MATCH=pointvector source scripts/resolve_ckpt.sh +# +# Sets: CKPT, CKPT_CFG, CKPT_NAME + +LOGROOT="$HOME/sum-parts/semantic_segmentation/PointNeXt_bundle/examples/segmentation/log/sumv2_triangle" +CKPT_MATCH="${CKPT_MATCH:-}" + +if [ -n "$CKPT_MATCH" ]; then + CKPT=$(find "$LOGROOT" -name "*${CKPT_MATCH}*_ckpt_best.pth" -printf '%T@ %p\n' 2>/dev/null \ + | sort -rn | head -1 | cut -d' ' -f2-) +else + CKPT=$(find "$LOGROOT" -name '*_ckpt_best.pth' -printf '%T@ %p\n' 2>/dev/null \ + | sort -rn | head -1 | cut -d' ' -f2-) +fi + +if [ -z "$CKPT" ]; then + echo "resolve_ckpt: no checkpoint found under $LOGROOT" >&2 + return 1 2>/dev/null || exit 1 +fi + +CKPT_NAME=$(basename "$CKPT") + +# run names look like: +# sumv2_triangle-train--ngpus1--_ckpt_best.pth +# longest names first so pointnet++msg wins over pointnet +CKPT_CFG="" +for m in pointvector-xl pointnext-xl "pointnet++msg" pointnet; do + case "$CKPT_NAME" in + *"-${m}-"*) CKPT_CFG="$m"; break ;; + esac +done + +if [ -z "$CKPT_CFG" ]; then + echo "resolve_ckpt: cannot tell which model wrote $CKPT_NAME" >&2 + return 1 2>/dev/null || exit 1 +fi + +export CKPT CKPT_CFG CKPT_NAME +echo "checkpoint: $CKPT_NAME" +echo "cfg : ${CKPT_CFG}.yaml" diff --git a/scripts/split_by_class.py b/scripts/split_by_class.py new file mode 100644 index 0000000..9c663fe --- /dev/null +++ b/scripts/split_by_class.py @@ -0,0 +1,124 @@ +#!/usr/bin/env python3 +"""Split a predicted point cloud into one OBJ per target class. + +The prediction carries one of SUM's 13 fine classes per point. This folds them +into the four the project cares about, plus a bucket for everything the model +could not place: + + ground terrain + building facade_surface, roof_surface, chimney, dormer, balcony, + roof_installation, wall + tree high_vegetation + vehicle car, boat + unseen unclassified, water + +water sits in `unseen` deliberately. The Seosan site has essentially no water, +so every point the model calls water is a misread, not a class we can trust. +Filing it as ground would bake that error into the terrain; filing it as its +own bucket keeps it visible. + +OBJ carries no per-point class, so each file is written as vertex-only geometry +(`v x y z r g b`) with one file per class. Viewers that read vertex colour show +the photo texture; the rest show the points. + +Usage: + python split_by_class.py pred.ply OUTDIR [--source rgb.ply] +""" +from __future__ import annotations + +import argparse +import sys +from pathlib import Path + +import numpy as np + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from sumparts_palette import ( # noqa: E402 + CLASSES as FINE, GROUPS, read_colours, read_point_classes, +) + +def write_obj(path: Path, xyz: np.ndarray, rgb: np.ndarray | None) -> None: + """OBJ, vertex-only. + + Kept because it was asked for, but be aware: an OBJ with no `f` lines is a + mesh with zero faces, and most viewers render exactly that - nothing. Use + the .ply next to it to actually look at the points. + """ + path.parent.mkdir(parents=True, exist_ok=True) + with path.open("w", encoding="utf-8") as f: + f.write(f"# {path.stem}: {len(xyz):,} points\n") + f.write("# vertex-only (no faces) -- most viewers show nothing.\n") + f.write("# open the .ply beside this file instead.\n") + if rgb is None: + for p in xyz: + f.write(f"v {p[0]:.4f} {p[1]:.4f} {p[2]:.4f}\n") + else: + for p, c in zip(xyz, rgb): + f.write(f"v {p[0]:.4f} {p[1]:.4f} {p[2]:.4f} " + f"{c[0]/255:.4f} {c[1]/255:.4f} {c[2]/255:.4f}\n") + + +def write_ply(path: Path, xyz: np.ndarray, rgb: np.ndarray | None) -> None: + """Binary PLY - the format point-cloud viewers actually open. + + float32 coordinates and uint8 red/green/blue, which is the spelling + CloudCompare, MeshLab and Mapple all read without coaxing. + """ + from plyfile import PlyData, PlyElement + + dtype = [("x", "f4"), ("y", "f4"), ("z", "f4")] + if rgb is not None: + dtype += [("red", "u1"), ("green", "u1"), ("blue", "u1")] + + arr = np.empty(len(xyz), dtype=dtype) + arr["x"], arr["y"], arr["z"] = xyz[:, 0], xyz[:, 1], xyz[:, 2] + if rgb is not None: + arr["red"], arr["green"], arr["blue"] = rgb[:, 0], rgb[:, 1], rgb[:, 2] + + path.parent.mkdir(parents=True, exist_ok=True) + PlyData([PlyElement.describe(arr, "vertex")], text=False).write(str(path)) + + +def main() -> None: + ap = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + ap.add_argument("pred", type=Path, help="prediction ply") + ap.add_argument("outdir", type=Path, help="root directory for the class folders") + ap.add_argument("--source", type=Path, default=None, + help="pre-inference ply, to carry photo colour into the OBJs") + ap.add_argument("--name", default="seosan_BlockYBA_tile0", + help="base filename inside each class folder") + args = ap.parse_args() + + if not args.pred.exists(): + raise SystemExit(f"{args.pred}: not found") + + xyz, cls = read_point_classes(args.pred) + print(f"prediction : {args.pred.name} {len(xyz):,} points") + + rgb = None + if args.source and args.source.exists(): + rgb = read_colours(args.source, len(xyz)) + print(f"colour : {'from ' + args.source.name if rgb is not None else 'unavailable (count mismatch)'}") + + print() + print(f" {'folder':<10} {'points':>10} {'share':>8} from") + total = 0 + for group, ids in GROUPS.items(): + mask = np.isin(cls, ids) + n = int(mask.sum()) + total += n + members = ", ".join(FINE[i] for i in ids) + sub_xyz = xyz[mask] + sub_rgb = rgb[mask] if rgb is not None else None + write_ply(args.outdir / group / f"{args.name}.ply", sub_xyz, sub_rgb) + write_obj(args.outdir / group / f"{args.name}.obj", sub_xyz, sub_rgb) + print(f" {group:<10} {n:>10,} {100*n/len(xyz):>7.2f}% {members}") + + print() + print(f" total {total:>10,} ({'all points accounted for' if total == len(xyz) else 'MISMATCH'})") + print(f" written to {args.outdir}") + + +if __name__ == "__main__": + main() diff --git a/scripts/sumparts_palette.py b/scripts/sumparts_palette.py new file mode 100644 index 0000000..5e4fc86 --- /dev/null +++ b/scripts/sumparts_palette.py @@ -0,0 +1,131 @@ +#!/usr/bin/env python3 +"""Shared SUM Parts class table and a memory-safe colour decoder. + +main.py writes predictions as palette colours rather than labels, so recovering +the class means matching RGB back to the palette. The obvious way to do that, + + d = ((rgb[:, None, :] - COLOR_MAP[None, :, :]) ** 2).sum(axis=2) + +builds an (N, 13, 3) float64 temporary. At 800k points that is ~250 MB per +intermediate and several exist at once - enough to run a machine out of memory +on a full tile, and the failure looks like an unrelated crash. + +Since main.py writes exact palette entries, an exact lookup handles nearly +every point in one pass with no large temporary. Only the leftovers fall back +to a distance search, and that runs in chunks. +""" +from __future__ import annotations + +import numpy as np + +CLASSES = ['unclassified', 'terrain', 'high_vegetation', 'facade_surface', + 'water', 'car', 'boat', 'roof_surface', 'chimney', 'dormer', + 'balcony', 'roof_installation', 'wall'] + +COLOR_MAP = np.array([ + (0, 0, 0), (170, 85, 0), (0, 255, 0), (255, 255, 0), + (0, 255, 255), (255, 0, 255), (0, 0, 153), (85, 85, 127), + (255, 50, 50), (85, 0, 127), (50, 125, 150), (50, 0, 50), + (215, 160, 140), +], dtype=np.float64) + +# what the project actually needs, folded from the 13 +# +# water sits in `unseen`, not `ground`: the Seosan site has essentially no +# water, so every point called water is a misread. Filing it as ground would +# bake that error into the terrain surface. +GROUPS: dict[str, list[int]] = { + "ground": [1], + "building": [3, 7, 8, 9, 10, 11, 12], + "tree": [2], + "vehicle": [5, 6], + "unseen": [0, 4], +} + + +def _pack(rgb_u8: np.ndarray) -> np.ndarray: + """RGB triples -> one int32 key each, for hashing.""" + a = rgb_u8.astype(np.int32) + return (a[:, 0] << 16) | (a[:, 1] << 8) | a[:, 2] + + +def decode_palette(rgb: np.ndarray, chunk: int = 200_000) -> np.ndarray: + """Class index per point, without allocating an (N, 13, 3) temporary. + + Exact palette hits resolve through a dict; anything else (a resampled or + recompressed file) falls back to a chunked nearest-colour search. + """ + rgb = np.asarray(rgb) + if rgb.dtype != np.uint8: + scaled = rgb.astype(np.float64) + if scaled.max() <= 1.0: + scaled = scaled * 255.0 + rgb = np.clip(scaled, 0, 255).astype(np.uint8) + + lut = {int(k): i for i, k in enumerate(_pack(COLOR_MAP.astype(np.uint8)))} + keys = _pack(rgb) + + out = np.full(len(rgb), -1, dtype=np.int64) + uniq, inverse = np.unique(keys, return_inverse=True) + resolved = np.array([lut.get(int(k), -1) for k in uniq], dtype=np.int64) + out = resolved[inverse] + + missing = out < 0 + if missing.any(): + idx = np.flatnonzero(missing) + for s in range(0, len(idx), chunk): + part = idx[s:s + chunk] + block = rgb[part].astype(np.float64) + d = ((block[:, None, :] - COLOR_MAP[None, :, :]) ** 2).sum(axis=2) + out[part] = d.argmin(axis=1) + + return out + + +def read_point_classes(path) -> tuple[np.ndarray, np.ndarray]: + """(xyz float32, class index) from a prediction or ground-truth ply.""" + from plyfile import PlyData + + v = PlyData.read(str(path))["vertex"] + props = [p.name for p in v.properties] + + xyz = np.stack([np.asarray(v["x"], dtype=np.float32), + np.asarray(v["y"], dtype=np.float32), + np.asarray(v["z"], dtype=np.float32)], axis=1) + + if "label" in props: + lab = np.asarray(v["label"]).astype(np.int64) + # a placeholder label (all -1, or a single constant on an unlabelled + # tile) carries no information; fall through to the colours + if lab.min() >= 0 and len(np.unique(lab)) > 1: + return xyz, lab + if lab.min() >= 0 and "red" not in props and "r" not in props: + return xyz, lab + + rgb_set = next((s for s in (("red", "green", "blue"), ("r", "g", "b")) + if all(c in props for c in s)), None) + if rgb_set is None: + raise SystemExit(f"{path}: no usable label field and no colour to decode") + + rgb = np.stack([np.asarray(v[c]) for c in rgb_set], axis=1) + return xyz, decode_palette(rgb) + + +def read_colours(path, n: int) -> np.ndarray | None: + """uint8 RGB from a ply, or None if it does not line up point-for-point.""" + from plyfile import PlyData + + v = PlyData.read(str(path))["vertex"] + props = [p.name for p in v.properties] + rgb_set = next((s for s in (("red", "green", "blue"), ("r", "g", "b")) + if all(c in props for c in s)), None) + if rgb_set is None or len(v) != n: + return None + + rgb = np.stack([np.asarray(v[c]) for c in rgb_set], axis=1) + if rgb.dtype != np.uint8: + rgb = rgb.astype(np.float64) + if rgb.max() <= 1.0: + rgb = rgb * 255.0 + rgb = np.clip(rgb, 0, 255).astype(np.uint8) + return rgb diff --git a/scripts/verify_outputs.sh b/scripts/verify_outputs.sh new file mode 100644 index 0000000..889a989 --- /dev/null +++ b/scripts/verify_outputs.sh @@ -0,0 +1,45 @@ +#!/usr/bin/env bash +# SUM Parts - prove the exported files are actually readable +# +# "I made you some files" is worth nothing if they do not open. This reads every +# one back with a parser and reports what a viewer will find inside. +set -uo pipefail + +ROOT="${1:-/mnt/d/AI_Test/sum-part}" + +source "$HOME/miniconda3/etc/profile.d/conda.sh" +conda activate sumparts + +python - "$ROOT" <<'PY' +import sys +from pathlib import Path + +import numpy as np +from plyfile import PlyData + +root = Path(sys.argv[1]) +print(f"root: {root}\n") + +print(f" {'folder':<10} {'file':<6} {'points':>10} {'properties':<34} {'extent (m)'}") +ok = True +for d in sorted(p for p in root.iterdir() if p.is_dir()): + for f in sorted(d.glob("*.ply")): + try: + v = PlyData.read(str(f))["vertex"] + props = ",".join(p.name for p in v.properties) + xyz = np.stack([v["x"], v["y"], v["z"]], axis=1) + ext = np.round(xyz.max(0) - xyz.min(0), 1) + print(f" {d.name:<10} {'ply':<6} {len(v):>10,} {props:<34} {ext.tolist()}") + except Exception as e: + ok = False + print(f" {d.name:<10} {'ply':<6} UNREADABLE: {type(e).__name__}: {e}") + + for f in sorted(d.glob("*.obj")): + n = sum(1 for line in f.open(encoding="utf-8") if line.startswith("v ")) + faces = sum(1 for line in f.open(encoding="utf-8") if line.startswith("f ")) + note = "" if faces else " <- no faces; most viewers show nothing" + print(f" {d.name:<10} {'obj':<6} {n:>10,} {'v x y z r g b':<34} {faces} faces{note}") + +print() +print("ALL PLY READABLE" if ok else "SOME FILES FAILED TO PARSE") +PY