nbrightandClaude Opus 5
f39b093106
Adds STATUS.md as the handoff document: benchmark numbers, the bare-earth
metrics that actually matter for this project, the Korean-data domain gap that
retraining will not fix, and what to do on the 24 GB machine.
The memory problem was in how a prediction's colours were turned back into
class indices. Every consumer built an (N, 13, 3) float64 temporary:
d = ((rgb[:, None, :] - COLOR_MAP[None, :, :]) ** 2).sum(axis=2)
That is ~250 MB of intermediates per 800k-point tile, several live at once, and
a full 4.7M-point block pushes it into gigabytes. main.py writes exact palette
entries, so an exact hash lookup resolves nearly every point with no large
temporary; only leftovers fall back to a chunked distance search. Peak RSS on a
470k-point tile drops to 61 MB. Extracted to sumparts_palette.py and shared by
coarse_eval.py and split_by_class.py.
Also from this round:
- patch_cm_mutation.sh: ConfusionMatrix.update() rewrote the caller's pred
tensor in place, folding every ignore_index point into class num_classes-1.
test() saves its visualization from that same tensor afterwards, so an
unlabelled tile came out 100% wall and the model looked degenerate when it
was not.
- patch_class_mask.sh: SUMPARTS_MASK_CLASSES drops known-absent classes from
the argmax. Measured on Seosan and it does not help - the runner-up for
"water" is "wall", not "terrain" - but the experiment is worth keeping.
- split_by_class.py now writes .ply alongside .obj. A vertex-only OBJ has zero
faces and most viewers render nothing, which is why the first export looked
broken.
- verify_outputs.sh reads exported files back with a parser, so "here are your
files" can be checked rather than asserted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
49 lines
1.8 KiB
Bash
49 lines
1.8 KiB
Bash
#!/usr/bin/env bash
|
|
# SUM Parts - put the Seosan prediction somewhere it can be opened and looked at
|
|
#
|
|
# Two files, because they answer different questions:
|
|
# *_pred_viewer.ply class colours -> where did each class land
|
|
# *_rgb_viewer.ply photo texture -> what is actually there
|
|
#
|
|
# Open both, flip between them. That is how you find out whether the 21% the
|
|
# model calls water is asphalt, shadow, or something else entirely.
|
|
set -uo pipefail
|
|
|
|
SCRIPTS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
LOGROOT="$HOME/sum-parts/semantic_segmentation/PointNeXt_bundle/examples/segmentation/log/sumv2_triangle"
|
|
DEST="/mnt/d/MYCLAUDE_PROJECT/sum-parts-test/output"
|
|
|
|
source "$HOME/miniconda3/etc/profile.d/conda.sh"
|
|
conda activate sumparts
|
|
|
|
mkdir -p "$DEST"
|
|
|
|
PRED=$(find "$LOGROOT" -name 'seosan*_pred.ply' -printf '%T@ %p\n' 2>/dev/null \
|
|
| sort -rn | head -1 | cut -d' ' -f2-)
|
|
[ -n "$PRED" ] || { echo "no seosan prediction found"; exit 1; }
|
|
|
|
echo "prediction : $PRED"
|
|
cp "$PRED" "$DEST/seosan_pred_viewer.ply"
|
|
echo " -> $DEST/seosan_pred_viewer.ply"
|
|
|
|
SRC="$HOME/sum-parts/data/korea_poc/seosan_BlockYBA_tile0.ply"
|
|
if [ -f "$SRC" ]; then
|
|
python "$SCRIPTS/ply_for_viewer.py" "$SRC" "$DEST/seosan_rgb_viewer.ply"
|
|
fi
|
|
|
|
echo
|
|
echo "=== class colour legend ==="
|
|
python - <<'PY'
|
|
CLASSES = ['unclassified', 'terrain', 'high_vegetation', 'facade_surface',
|
|
'water', 'car', 'boat', 'roof_surface', 'chimney', 'dormer',
|
|
'balcony', 'roof_installation', 'wall']
|
|
COLORS = [(0,0,0), (170,85,0), (0,255,0), (255,255,0), (0,255,255),
|
|
(255,0,255), (0,0,153), (85,85,127), (255,50,50), (85,0,127),
|
|
(50,125,150), (50,0,50), (215,160,140)]
|
|
for i, (n, c) in enumerate(zip(CLASSES, COLORS)):
|
|
print(f" {i:>2} {n:<20} RGB {c}")
|
|
PY
|
|
|
|
echo
|
|
ls -lh "$DEST"/seosan_*viewer.ply
|