kimminsung/sum-parts-test
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Record results so far and fix the memory blowup in the palette decode

Browse Source 
Adds STATUS.md as the handoff document: benchmark numbers, the bare-earth
metrics that actually matter for this project, the Korean-data domain gap that
retraining will not fix, and what to do on the 24 GB machine.

The memory problem was in how a prediction's colours were turned back into
class indices. Every consumer built an (N, 13, 3) float64 temporary:

    d = ((rgb[:, None, :] - COLOR_MAP[None, :, :]) ** 2).sum(axis=2)

That is ~250 MB of intermediates per 800k-point tile, several live at once, and
a full 4.7M-point block pushes it into gigabytes. main.py writes exact palette
entries, so an exact hash lookup resolves nearly every point with no large
temporary; only leftovers fall back to a chunked distance search. Peak RSS on a
470k-point tile drops to 61 MB. Extracted to sumparts_palette.py and shared by
coarse_eval.py and split_by_class.py.

Also from this round:

- patch_cm_mutation.sh: ConfusionMatrix.update() rewrote the caller's pred
  tensor in place, folding every ignore_index point into class num_classes-1.
  test() saves its visualization from that same tensor afterwards, so an
  unlabelled tile came out 100% wall and the model looked degenerate when it
  was not.
- patch_class_mask.sh: SUMPARTS_MASK_CLASSES drops known-absent classes from
  the argmax. Measured on Seosan and it does not help - the runner-up for
  "water" is "wall", not "terrain" - but the experiment is worth keeping.
- split_by_class.py now writes .ply alongside .obj. A vertex-only OBJ has zero
  faces and most viewers render nothing, which is why the first export looked
  broken.
- verify_outputs.sh reads exported files back with a parser, so "here are your
  files" can be checked rather than asserted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
nbright
2026-08-24 09:07:31 +09:00
co-authored by Claude Opus 5
parent 3fdd7ab3f0
commit f39b093106
19 changed files with 1144 additions and 46 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+3 -2
View File
@@ -1,4 +1,4 @@
# sum-parts-test # sum-parts-test
드론 사진측량 메시를 **건물 / 수목 / 차량 / 지면**으로 분할하기 위한 드론 사진측량 메시를 **건물 / 수목 / 차량 / 지면**으로 분할하기 위한
[SUM Parts](https://github.com/tudelft3d/SUM-Parts-Benchmarks) (CVPR 2025) 재현 및 적용 작업. [SUM Parts](https://github.com/tudelft3d/SUM-Parts-Benchmarks) (CVPR 2025) 재현 및 적용 작업.
@@ -9,7 +9,8 @@
| 경로 | 내용 | | 경로 | 내용 |
|---|---| |---|---|
| [SETUP.md](SETUP.md) | **1단계 — 환경 구축 (GPU 불필요). 여기부터** | | [STATUS.md](STATUS.md) | **현재 상태·결과·미해결 문제 — 이어받을 때 여기부터** |
| [SETUP.md](SETUP.md) | 1단계 — 환경 구축 (GPU 불필요) |
| [TRAIN.md](TRAIN.md) | 2단계 — 학습 (GPU 필요) | | [TRAIN.md](TRAIN.md) | 2단계 — 학습 (GPU 필요) |
| [docs/pipeline.html](docs/pipeline.html) | 전체 6단계 공정 정의 (브라우저로 열 것) | | [docs/pipeline.html](docs/pipeline.html) | 전체 6단계 공정 정의 (브라우저로 열 것) |
| [docs/SUM-Parts-검토노트.md](docs/SUM-Parts-검토노트.md) | 트러블슈팅 16건, 데이터 스키마 실측, 라이선스 | | [docs/SUM-Parts-검토노트.md](docs/SUM-Parts-검토노트.md) | 트러블슈팅 16건, 데이터 스키마 실측, 라이선스 |
STATUS.md
+246
View File
@@ -0,0 +1,246 @@
# 현재 상태 — 2026-08-21
3060 머신에서 여기까지 왔다. 이 문서는 **3090으로 옮겨서 이어가기 위한 인수인계**다.
목표: 드론 사진측량 메시에서 **구조물을 제거한 지형(bare earth)** 추출.
부수적으로 건물·수목·차량 분리.
---
## 한 줄 요약
SUM Parts 벤치마크는 재현했다. **지면 precision 95.24%로 목표 용도에 충분하다.**
문제는 벤치마크가 아니라 **한국 데이터에서의 도메인 갭**이다.
---
## 1. 완료된 것
| 항목 | 결과 |
|---|---|
| WSL2 환경 (CUDA 11.8 + torch 2.0.1 + 확장 5종) | ✅ |
| SUM Parts 데이터 (train 24 / val 8 / test 8) | ✅ |
| PointNet 100 epoch | mIoU 17.19 (논문 15.1 — **재현 확인**) |
| **PointVector 100 epoch** | **mIoU 51.83 / OA 80.41** |
| 서산 OBJ → PLY 변환 | ✅ 타일 1장 |
| 서산 추론 | ✅ 13클래스 예측 |
| 클래스별 분리 출력 | ✅ `D:\AI_Test\sum-part\` |
---
## 2. SUM 벤치마크 성적 (PointVector, voxel_max 24000)
### 4클래스 통합
| 클래스 | IoU | precision | recall |
|---|---|---|---|
| ground | 60.69% | **95.24%** | 62.59% |
| vegetation | 90.70% | 95.11% | 95.14% |
| building | 86.10% | 88.01% | 97.54% |
| vehicle | 51.05% | 78.81% | 59.17% |
| **mIoU 72.14% / OA 89.72%** | | | |
### bare earth (지면 vs 나머지) ← 실제 목표 지표
```
지면 정확히 유지 523,983
비지면 섞여 들어옴 26,217 ← 지형을 오염시킴
지면 놓침 268,755 ← 구멍, 보간 가능
precision 95.24%
recall 66.10%
IoU 63.98%
F1 78.04%
```
**오염원 분해**
| 출처 | 비중 |
|---|---|
| facade_surface | **49.90%** ← 절반이 건물 외벽 |
| high_vegetation | 17.42% |
| water | 12.04% |
| car | 7.83% |
| wall (옹벽) | 7.54% |
| roof_surface | 5.05% |
건물 외벽 하단 — 지면과 만나는 경계에서 새는 것으로 보인다.
**후처리로 상당 부분 잡을 수 있다**: 지면 예측 중 수직으로 튀는 포인트 제거.
> precision 95.24%는 목표 용도(구조물 제거 지형도)에 **이미 쓸 만하다.**
> recall 66.10%로 지면 34%를 놓치지만, 구멍은 보간으로 메운다.
---
## 3. ★ 미해결 — 한국 데이터 도메인 갭
서산 타일 1장 추론 결과:
| 클래스 | 비율 | 판정 |
|---|---|---|
| terrain | 24.29% | 타당 |
| **unclassified** | **21.70%** | ❌ 모델이 판단 못 함 |
| **water** | **21.11%** | ❌ **물 없다고 확인됨** |
| high_vegetation | 7.85% | 타당 |
| wall | 6.48% | ? |
| roof_surface | 6.11% | ? |
| **boat** | **5.07%** | ❌ **배 있을 리 없다** |
| facade_surface | 0.10% | ❌ 지붕 6%인데 벽 0.1%는 모순 |
**신뢰 불가 47.9%.**
### 원인
닫힌 집합 분류기다. 13개 중 반드시 하나를 고르고 **"모르겠음"이 없다.**
학습 어휘에 없는 것을 만나면 특징 공간에서 가장 가까운 것으로 간다.
헬싱키에서 배운 `water` = 어둡고 평평하고 균질하고 수평. 서산 포장면이 그 설명에 맞는다.
`boat`는 물 위에 있는 것이라 따라온다. **일관성 있는 착각이다.**
비율 매칭은 아니다 — SUM val의 water는 11.17%인데 서산 예측은 21.11%로 2배다.
### 시도했고 실패한 것: 클래스 마스킹
`water`·`boat` 로짓을 `-inf`로 죽여 2순위로 넘기는 실험.
```
wall +63,783 (6.48% → 20.05%) ← 절반 이상이 여기로
unclassified +33,867
terrain +13,656 ← 11%만 회복
```
**실패.** 2순위가 `terrain`이 아니라 `wall`이었다.
모델이 그 영역을 수직 구조물로 읽고 있다는 뜻이고, 도메인 갭이 얕지 않다는 증거다.
마스킹은 되돌렸다 (`SUMPARTS_MASK_CLASSES` 환경변수로 언제든 다시 켤 수 있다).
---
## 4. 발견한 업스트림 버그 4건 (전부 패치됨, 멱등, 원본 보존)
| # | 증상 | 원인 | 패치 |
|---|---|---|---|
| A | `bincount ... non-negative` | 블라인드 test셋 라벨이 전부 `-1` | `patch_unlabeled_test.sh` |
| B | `UnboundLocalError: 'epoch'` | `mode=val`이 학습 루프 변수 참조 | `patch_val_mode.sh` |
| C | `np.long` / `collections.Iterable` | numpy 1.24 / python 3.10에서 제거·이동 | `patch_numpy_aliases.sh` |
| D | **예측이 전부 한 클래스로 나옴** | `ConfusionMatrix.update()`가 **입력을 제자리 변조** | `patch_cm_mutation.sh` |
**D가 가장 위험했다.** `ignore_index`에 해당하는 포인트의 예측을
`num_classes-1`(=wall)로 덮어쓰는데, `main.py`가 그 다음에 시각화를 저장한다.
라벨이 없는 우리 타일은 전 포인트가 `ignore_index`라 **100% wall로 저장됐다.**
모델이 퇴화한 것처럼 보였지만 멀쩡했다.
---
## 5. 메모리 문제 (3090에서 겪은 것)
팔레트 색상 → 클래스 복원에서 `(N, 13, 3)` 배열을 통째로 만들고 있었다.
```python
d = ((rgb[:, None, :] - COLOR_MAP[None, :, :]) ** 2).sum(axis=2)
```
80만 포인트면 임시 배열 하나가 250 MB이고 여러 개가 동시에 존재한다.
전체 블록(470만 포인트)이면 GB 단위로 뛴다.
**수정 완료** — `sumparts_palette.py`로 분리하고 해시 조회 + 청크 폴백으로 바꿨다.
```
개선 후 peak RSS: 61 MB (470k 포인트 처리)
```
`coarse_eval.py`, `split_by_class.py`가 이 모듈을 쓴다.
---
## 6. 실측 성능표 (RTX 3060 12GB)
| 모델 | voxel_max | peak VRAM | s/iter | 100 epoch | 논문 mIoU |
|---|---|---|---|---|---|
| pointnet | 64000 | 6.01 G | 0.291 | 2.9h | 15.1% |
| pointnet++msg | 64000 | 4.16 G | 0.675 | 6.8h | 33.1% |
| pointvector-xl | 24000 | 6.46 G | 0.402 | **4.1h** | 70.0% |
| pointvector-xl | 64000 | **16.49 G** | 13.113 | ❌ 12GB 불가 | |
| pointnext-xl | 32000 | 8.03 G | 0.635 | 6.4h | 65.3% |
| pointnext-xl | 64000 | **15.47 G** | 46.980 | ❌ 12GB 불가 | |
> **WSL2에서 VRAM 초과는 OOM을 내지 않는다.** 호스트 RAM으로 흘려서
> 조용히 완주한다 — 25~100배 느리게. peak VRAM과 **전력(W)**으로 판정할 것.
> 사용률 100%인데 전력이 낮으면 연산이 아니라 PCIe 전송 대기다.
---
## 7. 3090에서 할 일
### 우선순위 1 — 논문 설정 재학습
`voxel_max 64000`은 16.5 GB가 필요해 12 GB 카드에서 불가능했다. 24 GB면 된다.
기대 효과:
| | 현재 (24000) | 재학습 (64000) 기대 |
|---|---|---|
| SUM terrain IoU | 60.69% | ~85% |
| 지면 precision | 95.24% | ~96% |
| 지면 recall | 66.10% | ~85% |
| **서산 물 오분류 21%** | — | **그대로** |
**주의**: `voxel_max`는 중립적 손잡이가 아니다. 64000으로 학습하면
**추론도 64000 청크로 해야** 성능이 나온다 → 서산 추론도 3090에서 돌려야 한다.
### 우선순위 2 — 서산 예측 육안 확인
`D:\AI_Test\sum-part\unseen\*.ply`에 42.82%가 들어 있다.
**이게 뭔지 모르면 나머지가 전부 추측이다.**
### 우선순위 3 — 도메인 갭 대응
재학습으로 안 고쳐진다. 선택지:
1. **SAM 3.1 자동 라벨링 + 파인튜닝** — 드론 원본 8,120장 + 카메라 포즈(PPK) 보유.
SAM 3의 텍스트 프롬프트로 명명된 마스크 → 포즈로 3D 투영 → 면 투표.
사람 몫은 전수 라벨링이 아니라 **1~2타일 검증**이다.
2. **후처리** — 지면 예측 중 수직 이상치 제거로 건물 외벽 오염(49.9%) 감소.
---
## 8. 산출물 위치
```
D:\AI_Test\sum-part\
├── ground\ 114,159 pts (24.29%)
├── building\ 93,076 pts (19.80%)
├── tree\ 36,876 pts ( 7.85%)
├── vehicle\ 24,631 pts ( 5.24%)
└── unseen\ 201,258 pts (42.82%) ← unclassified + water
```
각 폴더에 `.ply`와 `.obj`가 있다. **`.ply`를 열어라** —
OBJ는 면이 없어서 대부분의 뷰어가 아무것도 안 보여준다.
색상은 클래스 색이 아니라 **드론 사진 텍스처**라, 그 클래스로 분류된 게 실제로 뭔지 보인다.
```
D:\MYCLAUDE_PROJECT\sum-parts-test\output\
├── seosan_pred_viewer.ply 클래스 색
└── seosan_rgb_viewer.ply 사진 색
```
---
## 9. 제약
- **test 세트는 블라인드다.** 라벨이 전부 `-1`이라 로컬 채점 불가.
논문 수치와 직접 대조하려면 예측을 저자(gaoweixiaocuhk@gmail.com)에게 보내야 한다.
- **어노테이션 도구는 비공개다.** 저자가 유료 서비스로 판매 중.
- **학습 가중치도 비공개다.** 직접 학습이 유일한 경로.
- **라이선스**: 데이터 CC BY-NC 4.0, 코드 GPL-3.0. 상업 이용은 저자 허락 필요.
---
## 10. 문서
| 문서 | 용도 |
|---|---|
| [SETUP.md](SETUP.md) | 1단계 — 환경 구축 (GPU 불필요, ~80분) |
| [TRAIN.md](TRAIN.md) | 2단계 — 학습 (GPU 필요) |
| [docs/pipeline.html](docs/pipeline.html) | 전체 6단계 공정 정의 |
| [docs/SUM-Parts-검토노트.md](docs/SUM-Parts-검토노트.md) | 트러블슈팅 상세, 데이터 스키마 실측 |
scripts/after_train.sh
+51
View File
@@ -0,0 +1,51 @@
#!/usr/bin/env bash
# SUM Parts - wait for training to finish, then run every evaluation we care about
#
# Runs unattended so nobody has to sit watching for the last epoch:
# 1. wait for the trainer to exit
# 2. coarse evaluation on SUM val -> building / vegetation / vehicle / ground
# 3. inference on the Seosan tile -> does our own data work with this model
set -uo pipefail
SCRIPTS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
OUT="$HOME/sum-parts/runs/after_train"
LOG="$OUT/after_train.log"
mkdir -p "$OUT"
exec > >(tee -a "$LOG") 2>&1
say() { echo "[$(date '+%F %T')] $*"; }
say "waiting for training to finish"
waited=0
while pgrep -f 'main.py --cfg' > /dev/null; do
sleep 30
waited=$((waited + 30))
[ $((waited % 300)) -eq 0 ] && say " still running (${waited}s)"
[ "$waited" -gt 7200 ] && { say " timed out after 2h"; break; }
done
say "trainer no longer running"
# also wait out the watchdog, so it does not relaunch under us
pkill -f train_watchdog.sh 2>/dev/null && say "stopped the watchdog"
sleep 5
echo
say "=== best checkpoint ==="
CKPT=$(find "$HOME/sum-parts/semantic_segmentation/PointNeXt_bundle/examples/segmentation/log/sumv2_triangle" \
-name '*pointvector*_ckpt_best.pth' -printf '%T@ %p\n' 2>/dev/null \
| sort -rn | head -1 | cut -d' ' -f2-)
say "$CKPT"
grep -ahE 'Best ckpt @E' "$HOME/sum-parts/runs/pointvector-xl_"*/train.log 2>/dev/null | tail -2
echo
say "=== 1/2 coarse evaluation on SUM val (4 classes) ==="
TEST_VOXEL_MAX=24000 bash "$SCRIPTS/eval_coarse.sh" || say "eval_coarse returned non-zero"
echo
say "=== 2/2 inference on the Seosan tile ==="
bash "$SCRIPTS/poc_infer.sh" || say "poc_infer returned non-zero"
bash "$SCRIPTS/poc_check_pred.sh" || say "poc_check_pred returned non-zero"
echo
say "AFTER TRAIN DONE -- log: $LOG"
scripts/coarse_eval.py
+64 -29
View File
@@ -37,7 +37,9 @@ import sys
from pathlib import Path from pathlib import Path
import numpy as np import numpy as np
from plyfile import PlyData
sys.path.insert(0, str(Path(__file__).resolve().parent))
from sumparts_palette import read_point_classes # noqa: E402
FINE = ['unclassified', 'terrain', 'high_vegetation', 'facade_surface', FINE = ['unclassified', 'terrain', 'high_vegetation', 'facade_surface',
'water', 'car', 'boat', 'roof_surface', 'chimney', 'dormer', 'water', 'car', 'boat', 'roof_surface', 'chimney', 'dormer',
@@ -73,28 +75,14 @@ COLOR_MAP = np.array([
def load_labels(path: Path) -> np.ndarray: def load_labels(path: Path) -> np.ndarray:
"""Fine class per point: from a `label` field, else decoded from colour.""" """Fine class per point.
v = PlyData.read(str(path))["vertex"]
props = [p.name for p in v.properties]
if "label" in props: Delegates to sumparts_palette, whose colour decode avoids building an
lab = np.asarray(v["label"]).astype(np.int64).ravel() (N, 13, 3) temporary -- that pattern needs ~250 MB of intermediates per
# a prediction ply may carry a placeholder label; fall through if so 800k-point tile and takes the machine down on a full block.
if lab.max() >= 0 and not (lab == lab[0]).all(): """
return lab _, cls = read_point_classes(path)
if lab.max() >= 0 and lab[0] >= 0: return cls.astype(np.int64)
return lab
rgb_set = next((s for s in (("red", "green", "blue"), ("r", "g", "b"))
if all(c in props for c in s)), None)
if rgb_set is None:
raise SystemExit(f"{path}: no label field and no colour to decode")
rgb = np.stack([np.asarray(v[c], dtype=np.float64) for c in rgb_set], axis=1)
if rgb.max() <= 1.0:
rgb *= 255.0
d = ((rgb[:, None, :] - COLOR_MAP[None, :, :]) ** 2).sum(axis=2)
return d.argmin(axis=1).astype(np.int64)
def confusion(pred: np.ndarray, true: np.ndarray, n: int) -> np.ndarray: def confusion(pred: np.ndarray, true: np.ndarray, n: int) -> np.ndarray:
@@ -104,19 +92,24 @@ def confusion(pred: np.ndarray, true: np.ndarray, n: int) -> np.ndarray:
def report(cm: np.ndarray, names: list[str], skip: set[int]) -> None: def report(cm: np.ndarray, names: list[str], skip: set[int]) -> None:
tp = np.diag(cm).astype(np.float64) tp = np.diag(cm).astype(np.float64)
actual = cm.sum(axis=1).astype(np.float64) actual = cm.sum(axis=1).astype(np.float64) # ground truth per class
predicted = cm.sum(axis=0).astype(np.float64) predicted = cm.sum(axis=0).astype(np.float64) # predictions per class
union = actual + predicted - tp union = actual + predicted - tp
print(f" {'class':<12} {'IoU':>7} {'recall':>8} {'points':>12}") # precision matters as much as recall here, and for some jobs more.
# Stripping structures off a terrain model is precision-first: a hole where
# ground was missed can be interpolated, but a wall left sitting in the
# surface is a fake landform.
print(f" {'class':<12} {'IoU':>7} {'prec':>8} {'recall':>8} {'points':>12}")
ious = [] ious = []
for i, name in enumerate(names): for i, name in enumerate(names):
if i in skip: if i in skip:
continue continue
iou = 100.0 * tp[i] / union[i] if union[i] > 0 else 0.0 iou = 100.0 * tp[i] / union[i] if union[i] > 0 else 0.0
rec = 100.0 * tp[i] / actual[i] if actual[i] > 0 else 0.0 prec = 100.0 * tp[i] / predicted[i] if predicted[i] > 0 else 0.0
rec = 100.0 * tp[i] / actual[i] if actual[i] > 0 else 0.0
ious.append(iou) ious.append(iou)
print(f" {name:<12} {iou:>6.2f}% {rec:>7.2f}% {int(actual[i]):>12,}") print(f" {name:<12} {iou:>6.2f}% {prec:>7.2f}% {rec:>7.2f}% {int(actual[i]):>12,}")
scored = [i for i in range(len(names)) if i not in skip] scored = [i for i in range(len(names)) if i not in skip]
oa_tp = tp[scored].sum() oa_tp = tp[scored].sum()
@@ -126,6 +119,44 @@ def report(cm: np.ndarray, names: list[str], skip: set[int]) -> None:
print(f" OA : {100.0 * oa_tp / oa_n if oa_n else 0:.2f}%") print(f" OA : {100.0 * oa_tp / oa_n if oa_n else 0:.2f}%")
def report_bare_earth(cm_fine: np.ndarray) -> None:
"""Ground vs everything else, the way a terrain model is actually judged.
Reported separately from the 4-class view because the failure modes are not
symmetric. Ground missed -> a hole, which interpolation fills. Non-ground
kept -> a retaining wall or a roof baked into the terrain, which nothing
downstream can tell from a real landform.
"""
ground = {1} # terrain
n = cm_fine.shape[0]
rest = [i for i in range(n) if i not in ground and i != 0]
tp = cm_fine[1, 1]
fp = cm_fine[np.ix_(rest, [1])].sum() # non-ground predicted as ground
fn = cm_fine[np.ix_([1], rest)].sum() # ground predicted as something else
prec = 100.0 * tp / (tp + fp) if tp + fp else 0.0
rec = 100.0 * tp / (tp + fn) if tp + fn else 0.0
iou = 100.0 * tp / (tp + fp + fn) if tp + fp + fn else 0.0
f1 = 2 * prec * rec / (prec + rec) if prec + rec else 0.0
print(f" ground kept correctly {int(tp):>12,}")
print(f" non-ground leaked in {int(fp):>12,} <- contaminates the surface")
print(f" ground missed {int(fn):>12,} <- holes, interpolable")
print()
print(f" precision : {prec:6.2f}% (of what we call ground, how much is)")
print(f" recall : {rec:6.2f}% (of real ground, how much we caught)")
print(f" IoU : {iou:6.2f}%")
print(f" F1 : {f1:6.2f}%")
print()
if fp:
print(" where the contamination comes from:")
leaks = [(int(cm_fine[i, 1]), FINE[i]) for i in rest if cm_fine[i, 1] > 0]
for cnt, name in sorted(leaks, reverse=True)[:8]:
print(f" {name:<20} {cnt:>10,} {100.0*cnt/fp:>6.2f}% of leak")
def main() -> None: def main() -> None:
ap = argparse.ArgumentParser(description=__doc__, ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter) formatter_class=argparse.RawDescriptionHelpFormatter)
@@ -171,9 +202,13 @@ def main() -> None:
report(cm_fine, FINE, skip={0}) report(cm_fine, FINE, skip={0})
print() print()
print("=== coarse (what this project needs) ===") print("=== coarse (building / vegetation / vehicle / ground) ===")
report(cm_coarse, COARSE, skip={0}) report(cm_coarse, COARSE, skip={0})
print()
print("=== bare earth (ground vs everything else) ===")
report_bare_earth(cm_fine)
if __name__ == "__main__": if __name__ == "__main__":
main() main()
scripts/dump_pred.py
+85
View File
@@ -0,0 +1,85 @@
#!/usr/bin/env python3
"""Dump what a prediction PLY actually contains.
check_pred.py recovers the class by matching RGB against the palette. If that
reports one class for every point, the question is whether the model really
collapsed or whether the colour decode is wrong — so read the raw fields
instead of interpreting them.
Usage:
python dump_pred.py pred.ply
"""
from __future__ import annotations
import sys
from pathlib import Path
import numpy as np
from plyfile import PlyData
CLASSES = ['unclassified', 'terrain', 'high_vegetation', 'facade_surface',
'water', 'car', 'boat', 'roof_surface', 'chimney', 'dormer',
'balcony', 'roof_installation', 'wall']
COLOR_MAP = np.array([
(0., 0., 0.), (170., 85., 0.), (0., 255., 0.), (255., 255., 0.),
(0., 255., 255.), (255., 0., 255.), (0., 0., 153.), (85., 85., 127.),
(255., 50., 50.), (85., 0., 127.), (50., 125., 150.), (50., 0., 50.),
(215., 160., 140.),
])
def main() -> None:
if len(sys.argv) != 2:
print(__doc__)
raise SystemExit(2)
path = Path(sys.argv[1])
v = PlyData.read(str(path))["vertex"]
props = [p.name for p in v.properties]
print(f"file : {path.name}")
print(f"points: {len(v):,}")
print(f"props : {props}")
print()
if "label" in props:
lab = np.asarray(v["label"])
u, c = np.unique(lab, return_counts=True)
print(f"label field: dtype={lab.dtype}")
for k, n in sorted(zip(u.tolist(), c.tolist()), key=lambda t: -t[1]):
name = CLASSES[k] if 0 <= k < len(CLASSES) else f"?{k}"
print(f" {k:>3} {name:<20} {n:>10,} {100*n/len(lab):>6.2f}%")
print()
rgb_set = next((s for s in (("red", "green", "blue"), ("r", "g", "b"))
if all(c in props for c in s)), None)
if rgb_set:
rgb = np.stack([np.asarray(v[c]) for c in rgb_set], axis=1)
uniq, cnt = np.unique(rgb.reshape(-1, 3), axis=0, return_counts=True)
print(f"colour field {rgb_set}: dtype={rgb.dtype}, {len(uniq)} distinct")
order = np.argsort(-cnt)
for i in order[:15]:
col = uniq[i]
d = ((COLOR_MAP - col.astype(np.float64)) ** 2).sum(axis=1)
k = int(d.argmin())
exact = "exact" if d[k] < 1 else f"nearest (dist {np.sqrt(d[k]):.0f})"
name = CLASSES[k] if k < len(CLASSES) else f"?{k}"
print(f" {tuple(int(x) for x in col)!s:<20} {cnt[i]:>10,} "
f"{100*cnt[i]/len(rgb):>6.2f}% -> {name} ({exact})")
print()
if "label" in props and rgb_set:
lab = np.asarray(v["label"])
rgb = np.stack([np.asarray(v[c]) for c in rgb_set], axis=1)
d = ((rgb[:, None, :].astype(np.float64) - COLOR_MAP[None, :, :]) ** 2).sum(axis=2)
from_colour = d.argmin(axis=1)
agree = int((from_colour == lab).sum())
print(f"label vs colour agreement: {agree:,}/{len(lab):,} "
f"({100*agree/len(lab):.2f}%)")
if agree < len(lab):
print(" -> the two disagree; the label field is authoritative")
if __name__ == "__main__":
main()
scripts/dump_seosan_pred.sh
+21
View File
@@ -0,0 +1,21 @@
#!/usr/bin/env bash
# SUM Parts - dump the raw contents of the newest Seosan prediction
set -uo pipefail
SCRIPTS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
LOGROOT="$HOME/sum-parts/semantic_segmentation/PointNeXt_bundle/examples/segmentation/log"
source "$HOME/miniconda3/etc/profile.d/conda.sh"
conda activate sumparts
PRED=$(find "$LOGROOT" -name 'seosan*_pred.ply' -printf '%T@ %p\n' 2>/dev/null \
| sort -rn | head -1 | cut -d' ' -f2-)
if [ -z "$PRED" ]; then
echo "no seosan prediction found under $LOGROOT"
exit 1
fi
echo "path: $PRED"
echo
python "$SCRIPTS/dump_pred.py" "$PRED"
scripts/eval_coarse.sh
+5 -4
View File
@@ -20,9 +20,10 @@ export PYTORCH_CUDA_ALLOC_CONF="garbage_collection_threshold:0.7,max_split_size_
mkdir -p "$OUT" mkdir -p "$OUT"
cd "$SEG" cd "$SEG"
CKPT=$(find "$SEG/log/sumv2_triangle" -name '*_ckpt_best.pth' -printf '%T@ %p\n' \ # The cfg has to match the architecture that wrote the checkpoint. Hardcoding
| sort -rn | head -1 | cut -d' ' -f2-) # pointnet.yaml here meant a PointVector checkpoint loaded into the wrong model:
echo "checkpoint: $(basename "$CKPT")" # RuntimeError: Error(s) in loading state_dict for BaseSeg
source "$SCRIPTS/resolve_ckpt.sh"
# test() writes prediction plys and slides over whole tiles; point it at val so # test() writes prediction plys and slides over whole tiles; point it at val so
# there is ground truth to score against. # there is ground truth to score against.
@@ -40,7 +41,7 @@ VM_ARG=()
echo "=== inference over val split (sliding window) ===" echo "=== inference over val split (sliding window) ==="
echo " test voxel_max: ${TEST_VOXEL_MAX:-null (cfg default)}" echo " test voxel_max: ${TEST_VOXEL_MAX:-null (cfg default)}"
python -u main.py \ python -u main.py \
--cfg ../../cfgs/sumv2_triangle/pointnet.yaml \ --cfg "../../cfgs/sumv2_triangle/${CKPT_CFG}.yaml" \
mode=test \ mode=test \
--pretrained_path "$CKPT" \ --pretrained_path "$CKPT" \
dataset.common.data_root="$DATA" \ dataset.common.data_root="$DATA" \
scripts/export_pred_viewer.sh
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# SUM Parts - put the Seosan prediction somewhere it can be opened and looked at
#
# Two files, because they answer different questions:
# *_pred_viewer.ply class colours -> where did each class land
# *_rgb_viewer.ply photo texture -> what is actually there
#
# Open both, flip between them. That is how you find out whether the 21% the
# model calls water is asphalt, shadow, or something else entirely.
set -uo pipefail
SCRIPTS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
LOGROOT="$HOME/sum-parts/semantic_segmentation/PointNeXt_bundle/examples/segmentation/log/sumv2_triangle"
DEST="/mnt/d/MYCLAUDE_PROJECT/sum-parts-test/output"
source "$HOME/miniconda3/etc/profile.d/conda.sh"
conda activate sumparts
mkdir -p "$DEST"
PRED=$(find "$LOGROOT" -name 'seosan*_pred.ply' -printf '%T@ %p\n' 2>/dev/null \
| sort -rn | head -1 | cut -d' ' -f2-)
[ -n "$PRED" ] || { echo "no seosan prediction found"; exit 1; }
echo "prediction : $PRED"
cp "$PRED" "$DEST/seosan_pred_viewer.ply"
echo " -> $DEST/seosan_pred_viewer.ply"
SRC="$HOME/sum-parts/data/korea_poc/seosan_BlockYBA_tile0.ply"
if [ -f "$SRC" ]; then
python "$SCRIPTS/ply_for_viewer.py" "$SRC" "$DEST/seosan_rgb_viewer.ply"
fi
echo
echo "=== class colour legend ==="
python - <<'PY'
CLASSES = ['unclassified', 'terrain', 'high_vegetation', 'facade_surface',
'water', 'car', 'boat', 'roof_surface', 'chimney', 'dormer',
'balcony', 'roof_installation', 'wall']
COLORS = [(0,0,0), (170,85,0), (0,255,0), (255,255,0), (0,255,255),
(255,0,255), (0,0,153), (85,85,127), (255,50,50), (85,0,127),
(50,125,150), (50,0,50), (215,160,140)]
for i, (n, c) in enumerate(zip(CLASSES, COLORS)):
print(f" {i:>2} {n:<20} RGB {c}")
PY
echo
ls -lh "$DEST"/seosan_*viewer.ply
scripts/final_eval.sh
+5 -5
View File
@@ -31,11 +31,11 @@ export PYTORCH_CUDA_ALLOC_CONF="garbage_collection_threshold:0.7,max_split_size_
mkdir -p "$OUT" mkdir -p "$OUT"
CKPT=$(find "$SEG/log/sumv2_triangle" -name '*_ckpt_best.pth' -printf '%T@ %p\n' \ # The cfg has to match the architecture that wrote the checkpoint; hardcoding
| sort -rn | head -1 | cut -d' ' -f2-) # one here loads the weights into the wrong model and torch raises on the
[ -n "$CKPT" ] || { echo "error: no checkpoint found" >&2; exit 1; } # state_dict.
source "$SCRIPTS/resolve_ckpt.sh"
echo "checkpoint: $CKPT"
echo "data : $DATA" echo "data : $DATA"
echo echo
@@ -46,7 +46,7 @@ run_mode() {
echo "=== mode=$mode ===" echo "=== mode=$mode ==="
set +e set +e
python -u main.py \ python -u main.py \
--cfg ../../cfgs/sumv2_triangle/pointnet.yaml \ --cfg "../../cfgs/sumv2_triangle/${CKPT_CFG}.yaml" \
mode="$mode" \ mode="$mode" \
--pretrained_path "$CKPT" \ --pretrained_path "$CKPT" \
dataset.common.data_root="$DATA" \ dataset.common.data_root="$DATA" \
scripts/mask_experiment.sh
+67
View File
@@ -0,0 +1,67 @@
#!/usr/bin/env bash
# SUM Parts - measure what masking absent classes does to the Seosan prediction
#
# Runs inference twice on the same tile and the same checkpoint: once as-is,
# once with water and boat masked out of the argmax. The interesting number is
# where those 26% of points land when they can no longer be called water.
set -uo pipefail
SCRIPTS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
OUT="$HOME/sum-parts/runs/mask_experiment"
mkdir -p "$OUT"
source "$HOME/miniconda3/etc/profile.d/conda.sh"
conda activate sumparts
# 4 = water, 6 = boat (openpoints/dataset/sumv2_triangle/sumv2_triangle.py)
MASK="${MASK:-4,6}"
run_and_dump() {
local tag="$1" maskval="$2"
echo
echo "════ $tag ════"
SUMPARTS_MASK_CLASSES="$maskval" bash "$SCRIPTS/poc_infer.sh" \
> "$OUT/infer_${tag}.log" 2>&1
local rc=$?
if [ $rc -ne 0 ]; then
echo " inference FAILED rc=$rc"
tail -12 "$OUT/infer_${tag}.log"
return 1
fi
grep -a 'masking classes' "$OUT/infer_${tag}.log" | head -1
bash "$SCRIPTS/dump_seosan_pred.sh" 2>&1 | tee "$OUT/dump_${tag}.txt" \
| sed -n '/label field/,/^$/p'
}
run_and_dump baseline ""
run_and_dump masked "$MASK"
echo
echo "════ comparison ════"
python - "$OUT/dump_baseline.txt" "$OUT/dump_masked.txt" <<'PY'
import re
import sys
from pathlib import Path
def parse(path):
out = {}
for line in Path(path).read_text(encoding="utf-8", errors="replace").splitlines():
m = re.match(r"\s+(\d+)\s+(\S+)\s+([\d,]+)\s+([\d.]+)%", line)
if m:
out[m.group(2)] = (int(m.group(3).replace(",", "")), float(m.group(4)))
return out
a, b = parse(sys.argv[1]), parse(sys.argv[2])
names = sorted(set(a) | set(b), key=lambda n: -max(a.get(n, (0,))[0], b.get(n, (0,))[0]))
print(f" {'class':<20} {'baseline':>12} {'masked':>12} {'change':>12}")
for n in names:
an, ap = a.get(n, (0, 0.0))
bn, bp = b.get(n, (0, 0.0))
d = bn - an
arrow = "" if d == 0 else (" <- absorbed" if d > 0 else "")
print(f" {n:<20} {ap:>10.2f}% {bp:>10.2f}% {d:>+11,}{arrow}")
PY
echo
echo "logs in $OUT"
scripts/patch_class_mask.sh
+81
View File
@@ -0,0 +1,81 @@
#!/usr/bin/env bash
# SUM Parts - let inference rule out classes we know are absent
#
# The network is a closed-set classifier: 13 logits, softmax, argmax. There is
# no "none of these". A point whose true class was never in the training
# vocabulary still gets a label - whichever learned concept sits closest in
# feature space.
#
# On the Seosan road tiles that produces 21% water and 5% boat. Neither exists
# there. What the model learned as "water" in Helsinki - dark, flat, smooth,
# horizontal - describes asphalt exactly, and boats are what sits on water, so
# the hallucination is internally consistent.
#
# We know those classes are absent. The model does not. Masking their logits
# before the argmax hands each of those points to its runner-up class instead.
#
# This is not a fix for the domain gap; it is telling the model something we
# already know. Whether it helps depends entirely on what the runner-up is,
# which is why it is worth measuring rather than assuming.
#
# Usage:
# bash scripts/patch_class_mask.sh
# SUMPARTS_MASK_CLASSES=4,6 <run inference> # 4=water, 6=boat
#
# Idempotent.
set -euo pipefail
source "$HOME/miniconda3/etc/profile.d/conda.sh"
conda activate sumparts
REPO="${1:-$HOME/sum-parts/semantic_segmentation/PointNeXt_bundle}"
MAIN="$REPO/examples/segmentation/main.py"
[ -f "$MAIN" ] || { echo "error: $MAIN not found" >&2; exit 1; }
if grep -q 'SUMPARTS-CLASS-MASK' "$MAIN"; then
echo "already patched"
exit 0
fi
cp -n "$MAIN" "$MAIN.orig" 2>/dev/null || true
python - "$MAIN" <<'PY'
import sys
from pathlib import Path
p = Path(sys.argv[1])
src = p.read_text(encoding="utf-8")
old = """ pred = all_logits.argmax(dim=1)
if label is not None:
cm.update(pred, label)"""
new = """ # SUMPARTS-CLASS-MASK: drop classes we know cannot occur in this scene
# before taking the argmax, so their points fall through to the
# runner-up instead. Set SUMPARTS_MASK_CLASSES to a comma-separated
# list of class indices, e.g. "4,6" for water and boat.
_mask = os.environ.get('SUMPARTS_MASK_CLASSES', '').strip()
if _mask:
_idx = [int(x) for x in _mask.split(',') if x.strip() != '']
if cloud_idx == 0:
logging.info(f' masking classes {_idx} out of the argmax')
all_logits[:, _idx] = float('-inf')
pred = all_logits.argmax(dim=1)
if label is not None:
cm.update(pred, label)"""
if old not in src:
print("PATTERN NOT FOUND -- main.py differs from what this patch expects",
file=sys.stderr)
raise SystemExit(1)
p.write_text(src.replace(old, new), encoding="utf-8")
print("patched:", p)
PY
python -c "import ast,sys; ast.parse(open(sys.argv[1], encoding='utf-8').read())" "$MAIN" \
&& echo "syntax OK"
echo "PATCH DONE (original kept at $MAIN.orig)"
scripts/patch_cm_mutation.sh
+80
View File
@@ -0,0 +1,80 @@
#!/usr/bin/env bash
# SUM Parts - stop ConfusionMatrix.update from overwriting its caller's predictions
#
# openpoints/utils/metrics.py:
#
# if (true == self.ignore_index).sum() > 0:
# pred[true == self.ignore_index] = self.virtual_num_classes - 1
# true[true == self.ignore_index] = self.virtual_num_classes - 1
#
# Folding ignored points into the last bucket is fine for scoring, but it is
# done in place on the tensors the caller passed in. main.py's test() calls
# cm.update(pred, label) and *then* writes the visualization from that same
# pred, so the file on disk is the mutated copy, not what the model predicted.
#
# On a tile labelled entirely with the ignore class - which is exactly what an
# unlabelled tile of our own data looks like, label=0 with ignore_index=0 -
# every single point gets rewritten to class num_classes-1 (wall). The
# prediction file then reads 100% wall no matter what the network actually
# said, and the model looks degenerate when it is not.
#
# Fix: clone before masking. Scoring is unchanged; the caller's tensors survive.
#
# Idempotent.
set -euo pipefail
source "$HOME/miniconda3/etc/profile.d/conda.sh"
conda activate sumparts
REPO="${1:-$HOME/sum-parts/semantic_segmentation/PointNeXt_bundle}"
METRICS="$REPO/openpoints/utils/metrics.py"
[ -f "$METRICS" ] || { echo "error: $METRICS not found" >&2; exit 1; }
if grep -q 'SUMPARTS-NO-MUTATE' "$METRICS"; then
echo "already patched"
exit 0
fi
cp -n "$METRICS" "$METRICS.orig" 2>/dev/null || true
python - "$METRICS" <<'PY'
import sys
from pathlib import Path
p = Path(sys.argv[1])
src = p.read_text(encoding="utf-8")
old = """ true = true.flatten()
pred = pred.flatten()
if self.ignore_index is not None:
if (true == self.ignore_index).sum() > 0:
pred[true == self.ignore_index] = self.virtual_num_classes -1
true[true == self.ignore_index] = self.virtual_num_classes -1"""
new = """ # SUMPARTS-NO-MUTATE: clone before masking. These used to be written
# in place, which silently rewrote the caller's prediction tensor --
# main.py's test() saves its visualization from the same `pred` right
# after calling this, so the file on disk showed the folded values
# rather than the model's output. On a tile labelled entirely with the
# ignore class every point came out as num_classes-1.
true = true.flatten().clone()
pred = pred.flatten().clone()
if self.ignore_index is not None:
if (true == self.ignore_index).sum() > 0:
pred[true == self.ignore_index] = self.virtual_num_classes -1
true[true == self.ignore_index] = self.virtual_num_classes -1"""
if old not in src:
print("PATTERN NOT FOUND -- metrics.py differs from what this patch expects",
file=sys.stderr)
raise SystemExit(1)
p.write_text(src.replace(old, new), encoding="utf-8")
print("patched:", p)
PY
python -c "import ast,sys; ast.parse(open(sys.argv[1], encoding='utf-8').read())" "$METRICS" \
&& echo "syntax OK"
echo "PATCH DONE (original kept at $METRICS.orig)"
scripts/poc_check_pred.sh
+4 -1
View File
@@ -10,7 +10,10 @@ SCRIPTS="/mnt/d/MYCLAUDE_PROJECT/sum-parts-test/scripts"
source "$CONDA_ROOT/etc/profile.d/conda.sh" source "$CONDA_ROOT/etc/profile.d/conda.sh"
conda activate "$ENV_NAME" conda activate "$ENV_NAME"
PRED=$(find "$LOGROOT" -name '*_pred.ply' -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2-) # Match the Seosan tile by name. Taking simply the newest *_pred.ply picks up
# whatever the last SUM evaluation wrote instead.
PATTERN="${PRED_PATTERN:-*seosan*_pred.ply}"
PRED=$(find "$LOGROOT" -name "$PATTERN" -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2-)
if [ -z "$PRED" ]; then if [ -z "$PRED" ]; then
echo "error: no *_pred.ply found under $LOGROOT" >&2 echo "error: no *_pred.ply found under $LOGROOT" >&2
exit 1 exit 1
scripts/poc_infer.sh
+5 -5
View File
@@ -32,16 +32,16 @@ for split in train val test; do
done done
rm -rf "$TRACK/processed" rm -rf "$TRACK/processed"
CKPT=$(find "$SEG/log/sumv2_triangle" -name '*_ckpt_best.pth' -printf '%T@ %p\n' \ # The cfg has to match the architecture that wrote the checkpoint; hardcoding
| sort -rn | head -1 | cut -d' ' -f2-) # one here loads the weights into the wrong model and torch raises on the
[ -n "$CKPT" ] || { echo "error: no checkpoint found -- run smoke_train.sh first" >&2; exit 1; } # state_dict.
echo "checkpoint: $CKPT" source "/mnt/d/MYCLAUDE_PROJECT/sum-parts-test/scripts/resolve_ckpt.sh"
cd "$SEG" cd "$SEG"
set +e set +e
python -u main.py \ python -u main.py \
--cfg ../../cfgs/sumv2_triangle/pointnet.yaml \ --cfg "../../cfgs/sumv2_triangle/${CKPT_CFG}.yaml" \
mode=test \ mode=test \
--pretrained_path "$CKPT" \ --pretrained_path "$CKPT" \
dataset.common.data_root="$TRACK" \ dataset.common.data_root="$TRACK" \
scripts/rescore.sh
+27
View File
@@ -0,0 +1,27 @@
#!/usr/bin/env bash
# SUM Parts - re-score existing predictions without re-running inference
#
# The prediction plys are already on disk; only the scoring changed. No GPU,
# takes seconds.
set -uo pipefail
SCRIPTS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
DATA="$HOME/sum-parts/data/face_labeling/texsp_pcl"
LOGROOT="$HOME/sum-parts/semantic_segmentation/PointNeXt_bundle/examples/segmentation/log/sumv2_triangle"
OUT="$HOME/sum-parts/runs/coarse_eval"
source "$HOME/miniconda3/etc/profile.d/conda.sh"
conda activate sumparts
mkdir -p "$OUT"
VIS=$(find "$LOGROOT" -type d -name visualization -printf '%T@ %p\n' \
| sort -rn | head -1 | cut -d' ' -f2-)
[ -n "$VIS" ] || { echo "no visualization directory found"; exit 1; }
echo "predictions: $VIS"
echo "ground truth: $DATA/val"
echo
python "$SCRIPTS/coarse_eval.py" --pred-dir "$VIS" --gt-dir "$DATA/val" \
| tee "$OUT/coarse.txt"
scripts/resolve_ckpt.sh
+52
View File
@@ -0,0 +1,52 @@
#!/usr/bin/env bash
# SUM Parts - find the newest checkpoint and the cfg that matches it
#
# The eval scripts used to hardcode pointnet.yaml. Point them at a PointVector
# checkpoint and the weights load into the wrong architecture:
# RuntimeError: Error(s) in loading state_dict for BaseSeg
#
# main.py bakes the model name into the run directory, so the cfg can be read
# back out of the checkpoint path instead of guessed.
#
# Source it, don't run it:
# source scripts/resolve_ckpt.sh # newest checkpoint of any model
# CKPT_MATCH=pointvector source scripts/resolve_ckpt.sh
#
# Sets: CKPT, CKPT_CFG, CKPT_NAME
LOGROOT="$HOME/sum-parts/semantic_segmentation/PointNeXt_bundle/examples/segmentation/log/sumv2_triangle"
CKPT_MATCH="${CKPT_MATCH:-}"
if [ -n "$CKPT_MATCH" ]; then
CKPT=$(find "$LOGROOT" -name "*${CKPT_MATCH}*_ckpt_best.pth" -printf '%T@ %p\n' 2>/dev/null \
| sort -rn | head -1 | cut -d' ' -f2-)
else
CKPT=$(find "$LOGROOT" -name '*_ckpt_best.pth' -printf '%T@ %p\n' 2>/dev/null \
| sort -rn | head -1 | cut -d' ' -f2-)
fi
if [ -z "$CKPT" ]; then
echo "resolve_ckpt: no checkpoint found under $LOGROOT" >&2
return 1 2>/dev/null || exit 1
fi
CKPT_NAME=$(basename "$CKPT")
# run names look like:
# sumv2_triangle-train-<model>-ngpus1-<stamp>-<uuid>_ckpt_best.pth
# longest names first so pointnet++msg wins over pointnet
CKPT_CFG=""
for m in pointvector-xl pointnext-xl "pointnet++msg" pointnet; do
case "$CKPT_NAME" in
*"-${m}-"*) CKPT_CFG="$m"; break ;;
esac
done
if [ -z "$CKPT_CFG" ]; then
echo "resolve_ckpt: cannot tell which model wrote $CKPT_NAME" >&2
return 1 2>/dev/null || exit 1
fi
export CKPT CKPT_CFG CKPT_NAME
echo "checkpoint: $CKPT_NAME"
echo "cfg : ${CKPT_CFG}.yaml"
scripts/split_by_class.py
+124
View File
@@ -0,0 +1,124 @@
#!/usr/bin/env python3
"""Split a predicted point cloud into one OBJ per target class.
The prediction carries one of SUM's 13 fine classes per point. This folds them
into the four the project cares about, plus a bucket for everything the model
could not place:
ground terrain
building facade_surface, roof_surface, chimney, dormer, balcony,
roof_installation, wall
tree high_vegetation
vehicle car, boat
unseen unclassified, water
water sits in `unseen` deliberately. The Seosan site has essentially no water,
so every point the model calls water is a misread, not a class we can trust.
Filing it as ground would bake that error into the terrain; filing it as its
own bucket keeps it visible.
OBJ carries no per-point class, so each file is written as vertex-only geometry
(`v x y z r g b`) with one file per class. Viewers that read vertex colour show
the photo texture; the rest show the points.
Usage:
python split_by_class.py pred.ply OUTDIR [--source rgb.ply]
"""
from __future__ import annotations
import argparse
import sys
from pathlib import Path
import numpy as np
sys.path.insert(0, str(Path(__file__).resolve().parent))
from sumparts_palette import ( # noqa: E402
CLASSES as FINE, GROUPS, read_colours, read_point_classes,
)
def write_obj(path: Path, xyz: np.ndarray, rgb: np.ndarray | None) -> None:
"""OBJ, vertex-only.
Kept because it was asked for, but be aware: an OBJ with no `f` lines is a
mesh with zero faces, and most viewers render exactly that - nothing. Use
the .ply next to it to actually look at the points.
"""
path.parent.mkdir(parents=True, exist_ok=True)
with path.open("w", encoding="utf-8") as f:
f.write(f"# {path.stem}: {len(xyz):,} points\n")
f.write("# vertex-only (no faces) -- most viewers show nothing.\n")
f.write("# open the .ply beside this file instead.\n")
if rgb is None:
for p in xyz:
f.write(f"v {p[0]:.4f} {p[1]:.4f} {p[2]:.4f}\n")
else:
for p, c in zip(xyz, rgb):
f.write(f"v {p[0]:.4f} {p[1]:.4f} {p[2]:.4f} "
f"{c[0]/255:.4f} {c[1]/255:.4f} {c[2]/255:.4f}\n")
def write_ply(path: Path, xyz: np.ndarray, rgb: np.ndarray | None) -> None:
"""Binary PLY - the format point-cloud viewers actually open.
float32 coordinates and uint8 red/green/blue, which is the spelling
CloudCompare, MeshLab and Mapple all read without coaxing.
"""
from plyfile import PlyData, PlyElement
dtype = [("x", "f4"), ("y", "f4"), ("z", "f4")]
if rgb is not None:
dtype += [("red", "u1"), ("green", "u1"), ("blue", "u1")]
arr = np.empty(len(xyz), dtype=dtype)
arr["x"], arr["y"], arr["z"] = xyz[:, 0], xyz[:, 1], xyz[:, 2]
if rgb is not None:
arr["red"], arr["green"], arr["blue"] = rgb[:, 0], rgb[:, 1], rgb[:, 2]
path.parent.mkdir(parents=True, exist_ok=True)
PlyData([PlyElement.describe(arr, "vertex")], text=False).write(str(path))
def main() -> None:
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("pred", type=Path, help="prediction ply")
ap.add_argument("outdir", type=Path, help="root directory for the class folders")
ap.add_argument("--source", type=Path, default=None,
help="pre-inference ply, to carry photo colour into the OBJs")
ap.add_argument("--name", default="seosan_BlockYBA_tile0",
help="base filename inside each class folder")
args = ap.parse_args()
if not args.pred.exists():
raise SystemExit(f"{args.pred}: not found")
xyz, cls = read_point_classes(args.pred)
print(f"prediction : {args.pred.name} {len(xyz):,} points")
rgb = None
if args.source and args.source.exists():
rgb = read_colours(args.source, len(xyz))
print(f"colour : {'from ' + args.source.name if rgb is not None else 'unavailable (count mismatch)'}")
print()
print(f" {'folder':<10} {'points':>10} {'share':>8} from")
total = 0
for group, ids in GROUPS.items():
mask = np.isin(cls, ids)
n = int(mask.sum())
total += n
members = ", ".join(FINE[i] for i in ids)
sub_xyz = xyz[mask]
sub_rgb = rgb[mask] if rgb is not None else None
write_ply(args.outdir / group / f"{args.name}.ply", sub_xyz, sub_rgb)
write_obj(args.outdir / group / f"{args.name}.obj", sub_xyz, sub_rgb)
print(f" {group:<10} {n:>10,} {100*n/len(xyz):>7.2f}% {members}")
print()
print(f" total {total:>10,} ({'all points accounted for' if total == len(xyz) else 'MISMATCH'})")
print(f" written to {args.outdir}")
if __name__ == "__main__":
main()
scripts/sumparts_palette.py
+131
View File
@@ -0,0 +1,131 @@
#!/usr/bin/env python3
"""Shared SUM Parts class table and a memory-safe colour decoder.
main.py writes predictions as palette colours rather than labels, so recovering
the class means matching RGB back to the palette. The obvious way to do that,
d = ((rgb[:, None, :] - COLOR_MAP[None, :, :]) ** 2).sum(axis=2)
builds an (N, 13, 3) float64 temporary. At 800k points that is ~250 MB per
intermediate and several exist at once - enough to run a machine out of memory
on a full tile, and the failure looks like an unrelated crash.
Since main.py writes exact palette entries, an exact lookup handles nearly
every point in one pass with no large temporary. Only the leftovers fall back
to a distance search, and that runs in chunks.
"""
from __future__ import annotations
import numpy as np
CLASSES = ['unclassified', 'terrain', 'high_vegetation', 'facade_surface',
'water', 'car', 'boat', 'roof_surface', 'chimney', 'dormer',
'balcony', 'roof_installation', 'wall']
COLOR_MAP = np.array([
(0, 0, 0), (170, 85, 0), (0, 255, 0), (255, 255, 0),
(0, 255, 255), (255, 0, 255), (0, 0, 153), (85, 85, 127),
(255, 50, 50), (85, 0, 127), (50, 125, 150), (50, 0, 50),
(215, 160, 140),
], dtype=np.float64)
# what the project actually needs, folded from the 13
#
# water sits in `unseen`, not `ground`: the Seosan site has essentially no
# water, so every point called water is a misread. Filing it as ground would
# bake that error into the terrain surface.
GROUPS: dict[str, list[int]] = {
"ground": [1],
"building": [3, 7, 8, 9, 10, 11, 12],
"tree": [2],
"vehicle": [5, 6],
"unseen": [0, 4],
}
def _pack(rgb_u8: np.ndarray) -> np.ndarray:
"""RGB triples -> one int32 key each, for hashing."""
a = rgb_u8.astype(np.int32)
return (a[:, 0] << 16) | (a[:, 1] << 8) | a[:, 2]
def decode_palette(rgb: np.ndarray, chunk: int = 200_000) -> np.ndarray:
"""Class index per point, without allocating an (N, 13, 3) temporary.
Exact palette hits resolve through a dict; anything else (a resampled or
recompressed file) falls back to a chunked nearest-colour search.
"""
rgb = np.asarray(rgb)
if rgb.dtype != np.uint8:
scaled = rgb.astype(np.float64)
if scaled.max() <= 1.0:
scaled = scaled * 255.0
rgb = np.clip(scaled, 0, 255).astype(np.uint8)
lut = {int(k): i for i, k in enumerate(_pack(COLOR_MAP.astype(np.uint8)))}
keys = _pack(rgb)
out = np.full(len(rgb), -1, dtype=np.int64)
uniq, inverse = np.unique(keys, return_inverse=True)
resolved = np.array([lut.get(int(k), -1) for k in uniq], dtype=np.int64)
out = resolved[inverse]
missing = out < 0
if missing.any():
idx = np.flatnonzero(missing)
for s in range(0, len(idx), chunk):
part = idx[s:s + chunk]
block = rgb[part].astype(np.float64)
d = ((block[:, None, :] - COLOR_MAP[None, :, :]) ** 2).sum(axis=2)
out[part] = d.argmin(axis=1)
return out
def read_point_classes(path) -> tuple[np.ndarray, np.ndarray]:
"""(xyz float32, class index) from a prediction or ground-truth ply."""
from plyfile import PlyData
v = PlyData.read(str(path))["vertex"]
props = [p.name for p in v.properties]
xyz = np.stack([np.asarray(v["x"], dtype=np.float32),
np.asarray(v["y"], dtype=np.float32),
np.asarray(v["z"], dtype=np.float32)], axis=1)
if "label" in props:
lab = np.asarray(v["label"]).astype(np.int64)
# a placeholder label (all -1, or a single constant on an unlabelled
# tile) carries no information; fall through to the colours
if lab.min() >= 0 and len(np.unique(lab)) > 1:
return xyz, lab
if lab.min() >= 0 and "red" not in props and "r" not in props:
return xyz, lab
rgb_set = next((s for s in (("red", "green", "blue"), ("r", "g", "b"))
if all(c in props for c in s)), None)
if rgb_set is None:
raise SystemExit(f"{path}: no usable label field and no colour to decode")
rgb = np.stack([np.asarray(v[c]) for c in rgb_set], axis=1)
return xyz, decode_palette(rgb)
def read_colours(path, n: int) -> np.ndarray | None:
"""uint8 RGB from a ply, or None if it does not line up point-for-point."""
from plyfile import PlyData
v = PlyData.read(str(path))["vertex"]
props = [p.name for p in v.properties]
rgb_set = next((s for s in (("red", "green", "blue"), ("r", "g", "b"))
if all(c in props for c in s)), None)
if rgb_set is None or len(v) != n:
return None
rgb = np.stack([np.asarray(v[c]) for c in rgb_set], axis=1)
if rgb.dtype != np.uint8:
rgb = rgb.astype(np.float64)
if rgb.max() <= 1.0:
rgb = rgb * 255.0
rgb = np.clip(rgb, 0, 255).astype(np.uint8)
return rgb
scripts/verify_outputs.sh
+45
View File
@@ -0,0 +1,45 @@
#!/usr/bin/env bash
# SUM Parts - prove the exported files are actually readable
#
# "I made you some files" is worth nothing if they do not open. This reads every
# one back with a parser and reports what a viewer will find inside.
set -uo pipefail
ROOT="${1:-/mnt/d/AI_Test/sum-part}"
source "$HOME/miniconda3/etc/profile.d/conda.sh"
conda activate sumparts
python - "$ROOT" <<'PY'
import sys
from pathlib import Path
import numpy as np
from plyfile import PlyData
root = Path(sys.argv[1])
print(f"root: {root}\n")
print(f" {'folder':<10} {'file':<6} {'points':>10} {'properties':<34} {'extent (m)'}")
ok = True
for d in sorted(p for p in root.iterdir() if p.is_dir()):
for f in sorted(d.glob("*.ply")):
try:
v = PlyData.read(str(f))["vertex"]
props = ",".join(p.name for p in v.properties)
xyz = np.stack([v["x"], v["y"], v["z"]], axis=1)
ext = np.round(xyz.max(0) - xyz.min(0), 1)
print(f" {d.name:<10} {'ply':<6} {len(v):>10,} {props:<34} {ext.tolist()}")
except Exception as e:
ok = False
print(f" {d.name:<10} {'ply':<6} UNREADABLE: {type(e).__name__}: {e}")
for f in sorted(d.glob("*.obj")):
n = sum(1 for line in f.open(encoding="utf-8") if line.startswith("v "))
faces = sum(1 for line in f.open(encoding="utf-8") if line.startswith("f "))
note = "" if faces else " <- no faces; most viewers show nothing"
print(f" {d.name:<10} {'obj':<6} {n:>10,} {'v x y z r g b':<34} {faces} faces{note}")
print()
print("ALL PLY READABLE" if ok else "SOME FILES FAILED TO PARSE")
PY