nbrightandClaude Opus 5
609d9a6972
Reproduces the SUM Parts (CVPR 2025) face-labeling benchmark on a single consumer GPU, then applies it to drone-photogrammetry road survey meshes. Verified on RTX 3060 12GB / WSL2 Ubuntu 22.04 / CUDA 11.8 / torch 2.0.1: - CUDA extensions build (pointnet2_batch, pointops, chamfer_dist, emd, subsampling) - PointNet 100 epochs reaches mIoU 17.19, matching the paper's reported 15.1 - OBJ -> PLY conversion round-trips through the model and yields per-point predictions Four upstream source patches, all idempotent, originals preserved: - numpy aliases removed in 1.24 (np.long etc.) and collections ABCs moved in python 3.10 - the blind test split ships label = -1, which crashed ConfusionMatrix - mode=val referenced `epoch` before assignment Documents the traps that cost the most time, including VRAM overflow silently falling back to host RAM on WSL2 (25-100x slowdown, no OOM) and the colour scale mismatch between r/g/b float32 and red/green/blue uint8. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
49 lines
1.5 KiB
Bash
49 lines
1.5 KiB
Bash
#!/usr/bin/env bash
|
|
# SUM Parts - start an unattended training run and return immediately
|
|
#
|
|
# setsid detaches the watchdog into its own session, so it keeps running after
|
|
# the launching shell, the Claude Code session, or the Windows terminal goes
|
|
# away. It does NOT survive `wsl --shutdown` or a Windows reboot -- nothing run
|
|
# inside WSL does.
|
|
#
|
|
# Usage:
|
|
# bash launch_overnight.sh # pointnet, 100 epochs
|
|
# EPOCHS=100 bash launch_overnight.sh pointnet
|
|
#
|
|
# Then check on it any time with:
|
|
# bash check_training.sh
|
|
set -euo pipefail
|
|
|
|
SCRIPTS="/mnt/d/MYCLAUDE_PROJECT/sum-parts-test/scripts"
|
|
CFG="${1:-pointnet}"
|
|
STAMP=$(date +%Y%m%d-%H%M%S)
|
|
WORKDIR="$HOME/sum-parts/runs/${CFG}_${STAMP}"
|
|
|
|
mkdir -p "$WORKDIR"
|
|
echo "$WORKDIR" > "$HOME/sum-parts/runs/.latest"
|
|
|
|
# already running?
|
|
if pgrep -f "train_watchdog.sh" > /dev/null; then
|
|
echo "a watchdog is already running:"
|
|
pgrep -af "train_watchdog.sh"
|
|
echo
|
|
echo "stop it first with: bash $SCRIPTS/stop_training.sh"
|
|
exit 1
|
|
fi
|
|
|
|
STAMP="$STAMP" EPOCHS="${EPOCHS:-100}" VAL_FREQ="${VAL_FREQ:-5}" \
|
|
MAX_RETRIES="${MAX_RETRIES:-8}" CFG_VOXEL_MAX="${CFG_VOXEL_MAX:-}" \
|
|
RESUME_CKPT="${RESUME_CKPT:-}" \
|
|
setsid nohup bash "$SCRIPTS/train_watchdog.sh" "$CFG" \
|
|
> "$WORKDIR/nohup.out" 2>&1 < /dev/null &
|
|
|
|
sleep 3
|
|
|
|
echo "launched: $CFG"
|
|
echo "workdir : $WORKDIR"
|
|
echo
|
|
pgrep -af "train_watchdog.sh" || echo "WARNING: watchdog not visible in process list"
|
|
echo
|
|
echo "check progress: bash $SCRIPTS/check_training.sh"
|
|
echo "stop : bash $SCRIPTS/stop_training.sh"
|