Skip to content

Releases

Release v0.2.2

Choose a tag to compare

@ln2697 ln2697 released this 21 Oct 16:02

Overview

  • Minor pipeline fix.

Checkpoints

Download:

hf download kesai-labs/py123d_garage_pretrained_checkpoints \
  --revision 56edbe90a71a3fd4baee2d5f2f2da0bb7e722364 \
  --include 'v0.2.0/*' \
  --local-dir outputs/checkpoints

Drive the recommended seed in AlpaSim:

  • Alpamayo baselines. Alpamayo 1 and 1.5 are now in the Physical AI results table, evaluated in their native setting with the same AlpaSim commit.
  • Progress metric. Progress is now the mean progress along the logged drive, from 0 to 100, higher is better. It replaces the share of scenes under 80 % progress, in the README and on the model cards.
  • Kernel check fix. The kernel check at the start of training no longer fails at random on gradients that are zero by design. Such gradients get a looser limit; all other gradients keep the strict one.
  • Checkpoints unchanged. Same weights as v0.2.0; only the model cards were updated.

Checkpoints

Download:

hf download kesai-labs/py123d_garage_pretrained_checkpoints \
  --revision 56edbe90a71a3fd4baee2d5f2f2da0bb7e722364 \
  --include 'v0.2.0/*' \
  --local-dir outputs/checkpoints

Drive the recommended seed in AlpaSim:

CHECKPOINT_FILE=outputs/checkpoints/v0.2.0/ltf-base-joint64-68M/posttrain/model_0.pth scripts/evaluation/alpasim_pai.sh

nuPlan dataset

To reproduce our pipeline, only

CHECKPOINT_FILE=outputs/checkpoints/v0.2.0/ltf-base-joint64-68M/posttrain/model_0.pth scripts/evaluation/alpasim_pai.sh

nuPlan dataset

To reproduce our pipeline, only the front camera is required:

hf download kesai-labs/nuplan \
  --repo-type dataset \
  --revision 484d9d14e18fdf712dbc70962997ba2d53617bce \
  --local-dir data/nuplan/123D \
  --include 'logs/*' 'maps/*' \
  --exclude '*/camera.pcam_b0.arrow' \
            '*/camera.pcam_l0.arrow' \
            '*/camera.pcam_l1.arrow' \
            '*/camera.pcam_l2.arrow' \
            '*/camera.pcam_r0.arrow' \
            '*/camera.pcam_r1.arrow' \
            '*/camera.pcam_r2.arrow'

Physical AI AV and NuRec datasets

The NVIDIA AV Dataset License does not allow redistribution, so no 123D version is hosted. Download the raw data from nvidia/PhysicalAI-Autonomous-Vehicles and nvidia/PhysicalAI-Autonomous-Vehicles-NuRec and convert them yourself. See the Physical AI AV and NuRec docs.

CARLA dataset

The normal view and the maps are enough for training:

hf download ln2697/lead-123d \
  --repo-type dataset \
  --revision b9c4f6234a3e967981af8d8168f8670ee4348440 \
  --local-dir data/lead/123D \
  --include 'logs/normal_view/*' 'maps/*' 'config.yaml'

Documentation

Further notes

  • The leaderboard submissions before this release replan at 10 Hz. We now switched to 2Hz replanning.

Release v0.2.1

Choose a tag to compare

@ln2697 ln2697 released this 21 Oct 08:19

Overview

  • Alpamayo baselines. Alpamayo 1 and 1.5 are now in the Physical AI results table, evaluated in their native setting with the same AlpaSim commit.
  • Progress metric. Progress is now the mean progress along the logged drive, from 0 to 100, higher is better. It replaces the share of scenes under 80 % progress, in the README and on the model cards.
  • Kernel check fix. The kernel check at the start of training no longer fails at random on gradients that are zero by design. Such gradients get a looser limit; all other gradients keep the strict one.
  • Checkpoints unchanged. Same weights as v0.2.0; only the model cards were updated.

Checkpoints

Download:

hf download kesai-labs/py123d_garage_pretrained_checkpoints \
  --revision 56edbe90a71a3fd4baee2d5f2f2da0bb7e722364 \
  --include 'v0.2.0/*' \
  --local-dir outputs/checkpoints

Drive the recommended seed in AlpaSim:

CHECKPOINT_FILE=outputs/checkpoints/v0.2.0/ltf-base-joint64-68M/posttrain/model_0.pth scripts/evaluation/alpasim_pai.sh

nuPlan dataset

To reproduce our pipeline, only the front camera is required:

hf download kesai-labs/nuplan \
  --repo-type dataset \
  --revision 484d9d14e18fdf712dbc70962997ba2d53617bce \
  --local-dir data/nuplan/123D \
  --include 'logs/*' 'maps/*' \
  --exclude '*/camera.pcam_b0.arrow' \
            '*/camera.pcam_l0.arrow' \
            '*/camera.pcam_l1.arrow' \
            '*/camera.pcam_l2.arrow' \
            '*/camera.pcam_r0.arrow' \
            '*/camera.pcam_r1.arrow' \
            '*/camera.pcam_r2.arrow'

Physical AI AV and NuRec datasets

The NVIDIA AV Dataset License does not allow redistribution, so no 123D version is hosted. Download the raw data from nvidia/PhysicalAI-Autonomous-Vehicles and nvidia/PhysicalAI-Autonomous-Vehicles-NuRec and convert them yourself. See the Physical AI AV and NuRec docs.

CARLA dataset

The normal view and the maps are enough for training:

hf download ln2697/lead-123d \
  --repo-type dataset \
  --revision b9c4f6234a3e967981af8d8168f8670ee4348440 \
  --local-dir data/lead/123D \
  --include 'logs/normal_view/*' 'maps/*' 'config.yaml'

Documentation

Further notes

  • The leaderboard submissions before this release replan at 10 Hz. We now switched to 2Hz replanning.

Release v0.2.0

Choose a tag to compare

@ln2697 ln2697 released this 20 Oct 21:52

Overview

  • Four new checkpoints. Camera-only Latent TransFuser, 68M parameters, trained on Physical AI AV, NuRec and nuPlan, three seeds each, with model cards and per-scene AlpaSim results. See the model overview.
  • Flow-matching motion planner. Plans waypoints, actions and a path jointly.
  • Three-stage training recipe. Basic training recipe to tackle short-cut learnings.
  • Streaming backbone. A backbone variant that encodes each camera frame once at inference time, suitable for closed-loop training. See TransFuser.
  • AlpaSim on both challenge tracks. Evaluation scripts for the Physical AI and nuPlan tracks, ensembles, and a submission pipeline. See Evaluation.
  • NuRec dataset. Conversion and splits of the NVIDIA NuRec clips.
  • Research logs. See here.
  • AlpaSim pinned to commit e121e59 of the e2e_challenge branch, with three patches applied by the setup script.

Checkpoints

Download:

hf download kesai-labs/py123d_garage_pretrained_checkpoints \
  --revision afb6b51fce5c9965589dc4776777f90e72594622 \
  --include 'v0.2.0/*' \
  --local-dir outputs/checkpoints

Drive the recommended seed in AlpaSim:

CHECKPOINT_FILE=outputs/checkpoints/v0.2.0/ltf-base-joint64-68M/posttrain/model_0.pth scripts/evaluation/alpasim_pai.sh

nuPlan dataset

To reproduce our pipeline, only the front camera is required:

hf download kesai-labs/nuplan \
  --repo-type dataset \
  --revision 484d9d14e18fdf712dbc70962997ba2d53617bce \
  --local-dir data/nuplan/123D \
  --include 'logs/*' 'maps/*' \
  --exclude '*/camera.pcam_b0.arrow' \
            '*/camera.pcam_l0.arrow' \
            '*/camera.pcam_l1.arrow' \
            '*/camera.pcam_l2.arrow' \
            '*/camera.pcam_r0.arrow' \
            '*/camera.pcam_r1.arrow' \
            '*/camera.pcam_r2.arrow'

Physical AI AV and NuRec datasets

The NVIDIA AV Dataset License does not allow redistribution, so no 123D version is hosted. Download the raw data from nvidia/PhysicalAI-Autonomous-Vehicles and nvidia/PhysicalAI-Autonomous-Vehicles-NuRec and convert them yourself. See the Physical AI AV and NuRec docs.

CARLA dataset

The normal view and the maps are enough for training:

hf download ln2697/lead-123d \
  --repo-type dataset \
  --revision b9c4f6234a3e967981af8d8168f8670ee4348440 \
  --local-dir data/lead/123D \
  --include 'logs/normal_view/*' 'maps/*' 'config.yaml'

Documentation

Further notes

  • The leaderboard submissions before this release replan at 10 Hz. We now switched to 2Hz replanning.

Release v0.1.1

Choose a tag to compare

@ln2697 ln2697 released this 10 Sep 15:08

Overview

Checkpoints

Download:

hf download kesai-labs/py123d_garage_pretrained_checkpoints \
  --revision 5516eddafcfa4b622c3df59200e5b189ef328182 \
  --local-dir outputs/checkpoints

nuPlan dataset

To reproduce our pipeline, only three cameras are required:

hf download kesai-labs/nuplan \
  --repo-type dataset \
  --revision 484d9d14e18fdf712dbc70962997ba2d53617bce \
  --local-dir data/nuplan/123D \
  --include 'logs/*' 'maps/*' \
  --exclude '*/camera.pcam_b0.arrow' \
            '*/camera.pcam_l1.arrow' \
            '*/camera.pcam_l2.arrow' \
            '*/camera.pcam_r1.arrow' \
            '*/camera.pcam_r2.arrow'

CARLA dataset

The normal view and the maps are enough for training:

hf download ln2697/lead-123d \
  --repo-type dataset \
  --revision b9c4f6234a3e967981af8d8168f8670ee4348440 \
  --local-dir data/lead/123D \
  --include 'logs/normal_view/*' 'maps/*' 'config.yaml'

Physical AI AV dataset

The NVIDIA AV Dataset License does not allow redistribution, so no 123D version is hosted. Download the raw data from nvidia/PhysicalAI-Autonomous-Vehicles and convert it yourself. See doc.

Documentation

Release v0.1.0

Choose a tag to compare

@ln2697 ln2697 released this 08 Sep 21:34

Overview

  • Initial release of code, dataset and checkpoints.
  • Pin Alpasim to commit 54952f4a3a58d6a8b917fa61b52785c93a954619.

Checkpoints

Download:

hf download kesai-labs/py123d_garage_pretrained_checkpoints \
  --revision 5516eddafcfa4b622c3df59200e5b189ef328182 \
  --local-dir outputs/checkpoints

nuPlan dataset

To reproduce our pipeline, only three cameras are required:

hf download kesai-labs/nuplan \
  --repo-type dataset \
  --revision 484d9d14e18fdf712dbc70962997ba2d53617bce \
  --local-dir data/nuplan/123D \
  --include 'logs/*' 'maps/*' \
  --exclude '*/camera.pcam_b0.arrow' \
            '*/camera.pcam_l1.arrow' \
            '*/camera.pcam_l2.arrow' \
            '*/camera.pcam_r1.arrow' \
            '*/camera.pcam_r2.arrow'

CARLA dataset

The normal view and the maps are enough for training:

hf download ln2697/lead-123d \
  --repo-type dataset \
  --revision b9c4f6234a3e967981af8d8168f8670ee4348440 \
  --local-dir data/lead/123D \
  --include 'logs/normal_view/*' 'maps/*' 'config.yaml'

Physical AI AV dataset

The NVIDIA AV Dataset License does not allow redistribution, so no 123D version is hosted. Download the raw data from nvidia/PhysicalAI-Autonomous-Vehicles and convert it yourself. See doc.

Documentation