Repository navigation
Releases
Release v0.2.2
Overview
- Minor pipeline fix.
Checkpoints
Download:
hf download kesai-labs/py123d_garage_pretrained_checkpoints \
--revision 56edbe90a71a3fd4baee2d5f2f2da0bb7e722364 \
--include 'v0.2.0/*' \
--local-dir outputs/checkpointsDrive the recommended seed in AlpaSim:
- Alpamayo baselines. Alpamayo 1 and 1.5 are now in the Physical AI results table, evaluated in their native setting with the same AlpaSim commit.
- Progress metric. Progress is now the mean progress along the logged drive, from 0 to 100, higher is better. It replaces the share of scenes under 80 % progress, in the README and on the model cards.
- Kernel check fix. The kernel check at the start of training no longer fails at random on gradients that are zero by design. Such gradients get a looser limit; all other gradients keep the strict one.
- Checkpoints unchanged. Same weights as v0.2.0; only the model cards were updated.
Checkpoints
Download:
hf download kesai-labs/py123d_garage_pretrained_checkpoints \
--revision 56edbe90a71a3fd4baee2d5f2f2da0bb7e722364 \
--include 'v0.2.0/*' \
--local-dir outputs/checkpointsDrive the recommended seed in AlpaSim:
CHECKPOINT_FILE=outputs/checkpoints/v0.2.0/ltf-base-joint64-68M/posttrain/model_0.pth scripts/evaluation/alpasim_pai.shnuPlan dataset
To reproduce our pipeline, only
CHECKPOINT_FILE=outputs/checkpoints/v0.2.0/ltf-base-joint64-68M/posttrain/model_0.pth scripts/evaluation/alpasim_pai.shnuPlan dataset
To reproduce our pipeline, only the front camera is required:
hf download kesai-labs/nuplan \
--repo-type dataset \
--revision 484d9d14e18fdf712dbc70962997ba2d53617bce \
--local-dir data/nuplan/123D \
--include 'logs/*' 'maps/*' \
--exclude '*/camera.pcam_b0.arrow' \
'*/camera.pcam_l0.arrow' \
'*/camera.pcam_l1.arrow' \
'*/camera.pcam_l2.arrow' \
'*/camera.pcam_r0.arrow' \
'*/camera.pcam_r1.arrow' \
'*/camera.pcam_r2.arrow'Physical AI AV and NuRec datasets
The NVIDIA AV Dataset License does not allow redistribution, so no 123D version is hosted. Download the raw data from nvidia/PhysicalAI-Autonomous-Vehicles and nvidia/PhysicalAI-Autonomous-Vehicles-NuRec and convert them yourself. See the Physical AI AV and NuRec docs.
CARLA dataset
The normal view and the maps are enough for training:
hf download ln2697/lead-123d \
--repo-type dataset \
--revision b9c4f6234a3e967981af8d8168f8670ee4348440 \
--local-dir data/lead/123D \
--include 'logs/normal_view/*' 'maps/*' 'config.yaml'Documentation
- See here.
Further notes
- The leaderboard submissions before this release replan at 10 Hz. We now switched to 2Hz replanning.
Release v0.2.1
Overview
- Alpamayo baselines. Alpamayo 1 and 1.5 are now in the Physical AI results table, evaluated in their native setting with the same AlpaSim commit.
- Progress metric. Progress is now the mean progress along the logged drive, from 0 to 100, higher is better. It replaces the share of scenes under 80 % progress, in the README and on the model cards.
- Kernel check fix. The kernel check at the start of training no longer fails at random on gradients that are zero by design. Such gradients get a looser limit; all other gradients keep the strict one.
- Checkpoints unchanged. Same weights as v0.2.0; only the model cards were updated.
Checkpoints
Download:
hf download kesai-labs/py123d_garage_pretrained_checkpoints \
--revision 56edbe90a71a3fd4baee2d5f2f2da0bb7e722364 \
--include 'v0.2.0/*' \
--local-dir outputs/checkpointsDrive the recommended seed in AlpaSim:
CHECKPOINT_FILE=outputs/checkpoints/v0.2.0/ltf-base-joint64-68M/posttrain/model_0.pth scripts/evaluation/alpasim_pai.shnuPlan dataset
To reproduce our pipeline, only the front camera is required:
hf download kesai-labs/nuplan \
--repo-type dataset \
--revision 484d9d14e18fdf712dbc70962997ba2d53617bce \
--local-dir data/nuplan/123D \
--include 'logs/*' 'maps/*' \
--exclude '*/camera.pcam_b0.arrow' \
'*/camera.pcam_l0.arrow' \
'*/camera.pcam_l1.arrow' \
'*/camera.pcam_l2.arrow' \
'*/camera.pcam_r0.arrow' \
'*/camera.pcam_r1.arrow' \
'*/camera.pcam_r2.arrow'Physical AI AV and NuRec datasets
The NVIDIA AV Dataset License does not allow redistribution, so no 123D version is hosted. Download the raw data from nvidia/PhysicalAI-Autonomous-Vehicles and nvidia/PhysicalAI-Autonomous-Vehicles-NuRec and convert them yourself. See the Physical AI AV and NuRec docs.
CARLA dataset
The normal view and the maps are enough for training:
hf download ln2697/lead-123d \
--repo-type dataset \
--revision b9c4f6234a3e967981af8d8168f8670ee4348440 \
--local-dir data/lead/123D \
--include 'logs/normal_view/*' 'maps/*' 'config.yaml'Documentation
- See here.
Further notes
- The leaderboard submissions before this release replan at 10 Hz. We now switched to 2Hz replanning.
Release v0.2.0
Overview
- Four new checkpoints. Camera-only Latent TransFuser, 68M parameters, trained on Physical AI AV, NuRec and nuPlan, three seeds each, with model cards and per-scene AlpaSim results. See the model overview.
- Flow-matching motion planner. Plans waypoints, actions and a path jointly.
- Three-stage training recipe. Basic training recipe to tackle short-cut learnings.
- Streaming backbone. A backbone variant that encodes each camera frame once at inference time, suitable for closed-loop training. See TransFuser.
- AlpaSim on both challenge tracks. Evaluation scripts for the Physical AI and nuPlan tracks, ensembles, and a submission pipeline. See Evaluation.
- NuRec dataset. Conversion and splits of the NVIDIA NuRec clips.
- Research logs. See here.
- AlpaSim pinned to commit e121e59 of the
e2e_challengebranch, with three patches applied by the setup script.
Checkpoints
Download:
hf download kesai-labs/py123d_garage_pretrained_checkpoints \
--revision afb6b51fce5c9965589dc4776777f90e72594622 \
--include 'v0.2.0/*' \
--local-dir outputs/checkpointsDrive the recommended seed in AlpaSim:
CHECKPOINT_FILE=outputs/checkpoints/v0.2.0/ltf-base-joint64-68M/posttrain/model_0.pth scripts/evaluation/alpasim_pai.shnuPlan dataset
To reproduce our pipeline, only the front camera is required:
hf download kesai-labs/nuplan \
--repo-type dataset \
--revision 484d9d14e18fdf712dbc70962997ba2d53617bce \
--local-dir data/nuplan/123D \
--include 'logs/*' 'maps/*' \
--exclude '*/camera.pcam_b0.arrow' \
'*/camera.pcam_l0.arrow' \
'*/camera.pcam_l1.arrow' \
'*/camera.pcam_l2.arrow' \
'*/camera.pcam_r0.arrow' \
'*/camera.pcam_r1.arrow' \
'*/camera.pcam_r2.arrow'Physical AI AV and NuRec datasets
The NVIDIA AV Dataset License does not allow redistribution, so no 123D version is hosted. Download the raw data from nvidia/PhysicalAI-Autonomous-Vehicles and nvidia/PhysicalAI-Autonomous-Vehicles-NuRec and convert them yourself. See the Physical AI AV and NuRec docs.
CARLA dataset
The normal view and the maps are enough for training:
hf download ln2697/lead-123d \
--repo-type dataset \
--revision b9c4f6234a3e967981af8d8168f8670ee4348440 \
--local-dir data/lead/123D \
--include 'logs/normal_view/*' 'maps/*' 'config.yaml'Documentation
- See here.
Further notes
- The leaderboard submissions before this release replan at 10 Hz. We now switched to 2Hz replanning.
Release v0.1.1
Overview
- Pump Alpasim to commit cd713e0d0563352ce45b99d0a53a7173d581bde2.
Checkpoints
Download:
hf download kesai-labs/py123d_garage_pretrained_checkpoints \
--revision 5516eddafcfa4b622c3df59200e5b189ef328182 \
--local-dir outputs/checkpointsnuPlan dataset
To reproduce our pipeline, only three cameras are required:
hf download kesai-labs/nuplan \
--repo-type dataset \
--revision 484d9d14e18fdf712dbc70962997ba2d53617bce \
--local-dir data/nuplan/123D \
--include 'logs/*' 'maps/*' \
--exclude '*/camera.pcam_b0.arrow' \
'*/camera.pcam_l1.arrow' \
'*/camera.pcam_l2.arrow' \
'*/camera.pcam_r1.arrow' \
'*/camera.pcam_r2.arrow'CARLA dataset
The normal view and the maps are enough for training:
hf download ln2697/lead-123d \
--repo-type dataset \
--revision b9c4f6234a3e967981af8d8168f8670ee4348440 \
--local-dir data/lead/123D \
--include 'logs/normal_view/*' 'maps/*' 'config.yaml'Physical AI AV dataset
The NVIDIA AV Dataset License does not allow redistribution, so no 123D version is hosted. Download the raw data from nvidia/PhysicalAI-Autonomous-Vehicles and convert it yourself. See doc.
Documentation
- See here.
Release v0.1.0
Overview
- Initial release of code, dataset and checkpoints.
- Pin Alpasim to commit
54952f4a3a58d6a8b917fa61b52785c93a954619.
Checkpoints
Download:
hf download kesai-labs/py123d_garage_pretrained_checkpoints \
--revision 5516eddafcfa4b622c3df59200e5b189ef328182 \
--local-dir outputs/checkpointsnuPlan dataset
To reproduce our pipeline, only three cameras are required:
hf download kesai-labs/nuplan \
--repo-type dataset \
--revision 484d9d14e18fdf712dbc70962997ba2d53617bce \
--local-dir data/nuplan/123D \
--include 'logs/*' 'maps/*' \
--exclude '*/camera.pcam_b0.arrow' \
'*/camera.pcam_l1.arrow' \
'*/camera.pcam_l2.arrow' \
'*/camera.pcam_r1.arrow' \
'*/camera.pcam_r2.arrow'CARLA dataset
The normal view and the maps are enough for training:
hf download ln2697/lead-123d \
--repo-type dataset \
--revision b9c4f6234a3e967981af8d8168f8670ee4348440 \
--local-dir data/lead/123D \
--include 'logs/normal_view/*' 'maps/*' 'config.yaml'Physical AI AV dataset
The NVIDIA AV Dataset License does not allow redistribution, so no 123D version is hosted. Download the raw data from nvidia/PhysicalAI-Autonomous-Vehicles and convert it yourself. See doc.
Documentation
- See here.