nbrightandClaude Opus 5
609d9a6972
Reproduces the SUM Parts (CVPR 2025) face-labeling benchmark on a single consumer GPU, then applies it to drone-photogrammetry road survey meshes. Verified on RTX 3060 12GB / WSL2 Ubuntu 22.04 / CUDA 11.8 / torch 2.0.1: - CUDA extensions build (pointnet2_batch, pointops, chamfer_dist, emd, subsampling) - PointNet 100 epochs reaches mIoU 17.19, matching the paper's reported 15.1 - OBJ -> PLY conversion round-trips through the model and yields per-point predictions Four upstream source patches, all idempotent, originals preserved: - numpy aliases removed in 1.24 (np.long etc.) and collections ABCs moved in python 3.10 - the blind test split ships label = -1, which crashed ConfusionMatrix - mode=val referenced `epoch` before assignment Documents the traps that cost the most time, including VRAM overflow silently falling back to host RAM on WSL2 (25-100x slowdown, no OOM) and the colour scale mismatch between r/g/b float32 and red/green/blue uint8. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
36 lines
1.1 KiB
Bash
36 lines
1.1 KiB
Bash
#!/usr/bin/env bash
|
|
# SUM Parts - compare early vs recent per-iteration speed in a running train log
|
|
#
|
|
# Separates "each iteration got slower" from "something between epochs got
|
|
# slower" (validation, checkpointing, dataloader restart).
|
|
set -uo pipefail
|
|
|
|
RUNROOT="$HOME/sum-parts/runs"
|
|
WORKDIR="${1:-$(cat "$RUNROOT/.latest" 2>/dev/null)}"
|
|
LOG="$WORKDIR/train.log"
|
|
|
|
[ -f "$LOG" ] || { echo "no train.log at ${WORKDIR:-<unset>}"; exit 1; }
|
|
|
|
echo "log: $LOG"
|
|
echo
|
|
|
|
echo "=== earliest 15 rate readings ==="
|
|
grep -oE '[0-9.]+(s/it|it/s)' "$LOG" | head -15 | tr '\n' ' '
|
|
echo
|
|
echo
|
|
|
|
echo "=== most recent 15 rate readings ==="
|
|
grep -oE '[0-9.]+(s/it|it/s)' "$LOG" | tail -15 | tr '\n' ' '
|
|
echo
|
|
echo
|
|
|
|
echo "=== Train Epoch progress lines (first / last) ==="
|
|
grep -oE 'Train Epoch \[[0-9]+/[0-9]+\] Loss [0-9.]+ Acc [0-9.]+' "$LOG" | head -3
|
|
echo " ..."
|
|
grep -oE 'Train Epoch \[[0-9]+/[0-9]+\] Loss [0-9.]+ Acc [0-9.]+' "$LOG" | tail -3
|
|
echo
|
|
|
|
echo "=== log growth ==="
|
|
echo " size : $(du -h "$LOG" | cut -f1)"
|
|
echo " last write: $(date -r "$LOG" '+%F %T') (now $(date '+%F %T'))"
|