Record results so far and fix the memory blowup in the palette decode
Adds STATUS.md as the handoff document: benchmark numbers, the bare-earth
metrics that actually matter for this project, the Korean-data domain gap that
retraining will not fix, and what to do on the 24 GB machine.
The memory problem was in how a prediction's colours were turned back into
class indices. Every consumer built an (N, 13, 3) float64 temporary:
d = ((rgb[:, None, :] - COLOR_MAP[None, :, :]) ** 2).sum(axis=2)
That is ~250 MB of intermediates per 800k-point tile, several live at once, and
a full 4.7M-point block pushes it into gigabytes. main.py writes exact palette
entries, so an exact hash lookup resolves nearly every point with no large
temporary; only leftovers fall back to a chunked distance search. Peak RSS on a
470k-point tile drops to 61 MB. Extracted to sumparts_palette.py and shared by
coarse_eval.py and split_by_class.py.
Also from this round:
- patch_cm_mutation.sh: ConfusionMatrix.update() rewrote the caller's pred
tensor in place, folding every ignore_index point into class num_classes-1.
test() saves its visualization from that same tensor afterwards, so an
unlabelled tile came out 100% wall and the model looked degenerate when it
was not.
- patch_class_mask.sh: SUMPARTS_MASK_CLASSES drops known-absent classes from
the argmax. Measured on Seosan and it does not help - the runner-up for
"water" is "wall", not "terrain" - but the experiment is worth keeping.
- split_by_class.py now writes .ply alongside .obj. A vertex-only OBJ has zero
faces and most viewers render nothing, which is why the first export looked
broken.
- verify_outputs.sh reads exported files back with a parser, so "here are your
files" can be checked rather than asserted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,81 @@
|
||||
#!/usr/bin/env bash
|
||||
# SUM Parts - let inference rule out classes we know are absent
|
||||
#
|
||||
# The network is a closed-set classifier: 13 logits, softmax, argmax. There is
|
||||
# no "none of these". A point whose true class was never in the training
|
||||
# vocabulary still gets a label - whichever learned concept sits closest in
|
||||
# feature space.
|
||||
#
|
||||
# On the Seosan road tiles that produces 21% water and 5% boat. Neither exists
|
||||
# there. What the model learned as "water" in Helsinki - dark, flat, smooth,
|
||||
# horizontal - describes asphalt exactly, and boats are what sits on water, so
|
||||
# the hallucination is internally consistent.
|
||||
#
|
||||
# We know those classes are absent. The model does not. Masking their logits
|
||||
# before the argmax hands each of those points to its runner-up class instead.
|
||||
#
|
||||
# This is not a fix for the domain gap; it is telling the model something we
|
||||
# already know. Whether it helps depends entirely on what the runner-up is,
|
||||
# which is why it is worth measuring rather than assuming.
|
||||
#
|
||||
# Usage:
|
||||
# bash scripts/patch_class_mask.sh
|
||||
# SUMPARTS_MASK_CLASSES=4,6 <run inference> # 4=water, 6=boat
|
||||
#
|
||||
# Idempotent.
|
||||
set -euo pipefail
|
||||
|
||||
source "$HOME/miniconda3/etc/profile.d/conda.sh"
|
||||
conda activate sumparts
|
||||
|
||||
REPO="${1:-$HOME/sum-parts/semantic_segmentation/PointNeXt_bundle}"
|
||||
MAIN="$REPO/examples/segmentation/main.py"
|
||||
|
||||
[ -f "$MAIN" ] || { echo "error: $MAIN not found" >&2; exit 1; }
|
||||
|
||||
if grep -q 'SUMPARTS-CLASS-MASK' "$MAIN"; then
|
||||
echo "already patched"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cp -n "$MAIN" "$MAIN.orig" 2>/dev/null || true
|
||||
|
||||
python - "$MAIN" <<'PY'
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
p = Path(sys.argv[1])
|
||||
src = p.read_text(encoding="utf-8")
|
||||
|
||||
old = """ pred = all_logits.argmax(dim=1)
|
||||
if label is not None:
|
||||
cm.update(pred, label)"""
|
||||
|
||||
new = """ # SUMPARTS-CLASS-MASK: drop classes we know cannot occur in this scene
|
||||
# before taking the argmax, so their points fall through to the
|
||||
# runner-up instead. Set SUMPARTS_MASK_CLASSES to a comma-separated
|
||||
# list of class indices, e.g. "4,6" for water and boat.
|
||||
_mask = os.environ.get('SUMPARTS_MASK_CLASSES', '').strip()
|
||||
if _mask:
|
||||
_idx = [int(x) for x in _mask.split(',') if x.strip() != '']
|
||||
if cloud_idx == 0:
|
||||
logging.info(f' masking classes {_idx} out of the argmax')
|
||||
all_logits[:, _idx] = float('-inf')
|
||||
|
||||
pred = all_logits.argmax(dim=1)
|
||||
if label is not None:
|
||||
cm.update(pred, label)"""
|
||||
|
||||
if old not in src:
|
||||
print("PATTERN NOT FOUND -- main.py differs from what this patch expects",
|
||||
file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
|
||||
p.write_text(src.replace(old, new), encoding="utf-8")
|
||||
print("patched:", p)
|
||||
PY
|
||||
|
||||
python -c "import ast,sys; ast.parse(open(sys.argv[1], encoding='utf-8').read())" "$MAIN" \
|
||||
&& echo "syntax OK"
|
||||
|
||||
echo "PATCH DONE (original kept at $MAIN.orig)"
|
||||
Reference in New Issue
Block a user