-
Notifications
You must be signed in to change notification settings - Fork 457
add smolvla to model-libraries.ts #1529
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 24 commits
0167612
2343fee
39d5f50
bab9770
85e2ce1
00b8578
ebe7910
0f9d592
75517d1
8ded89d
923df2d
a68bbf6
7f370c3
f78b651
64a8e3b
15dc85d
39edaa6
5ed372b
17773dc
d88c4f1
6c805cb
400c41f
3bf9671
b6e6919
9a42a0d
301e374
f817655
a21ee89
d50d2ce
64b4a7d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -1735,4 +1735,46 @@ audio = model.autoencoder.decode(codes)[0].cpu() | |
torchaudio.save("sample.wav", audio, model.autoencoder.sampling_rate) | ||
`, | ||
]; | ||
|
||
export const lerobot = (model: ModelData): string[] => { | ||
if (model.tags.includes("smolvla")) { | ||
const smolvlaSnippets = [ | ||
// Installation snippet | ||
`# Install LeRobot from source | ||
git clone https://github.com/huggingface/lerobot.git | ||
cd lerobot | ||
pip install -e .[smolvla]`, | ||
// Finetune snippet | ||
`# Launch finetuning on your dataset | ||
python lerobot/scripts/train.py \\ | ||
--policy.path=${model.id} \\ | ||
--dataset.repo_id=lerobot/svla_so101_pickplace \\ | ||
--batch_size=64 \\ | ||
--steps=20000 \\ | ||
--output_dir=outputs/train/my_smolvla \\ | ||
--job_name=my_smolvla_training \\ | ||
--policy.device=cuda \\ | ||
--wandb.enable=true`, | ||
]; | ||
if (model.id !== "lerobot/smolvla_base") { | ||
// Inference snippet (only if not base model) | ||
smolvlaSnippets.push( | ||
`# Run the policy using the record function | ||
python -m lerobot.record \ | ||
--robot.type=so101_follower \ | ||
--robot.port=/dev/ttyACM0 \ # <- Use your port | ||
--robot.id=my_blue_follower_arm \ # <- Use your robot id | ||
--robot.cameras="{ front: {type: opencv, index_or_path: 8, width: 640, height: 480, fps: 30}}" \ # <- Use your cameras | ||
--dataset.single_task="Grasp a lego block and put it in the bin." \ # <- Use the same task description you used in your dataset recording | ||
--dataset.repo_id=HF_USER/dataset_name \ # <- This will be the dataset name on HF Hub | ||
--dataset.episode_time_s=50 \ | ||
--dataset.num_episodes=10 \ | ||
--policy.path=${model.id}` | ||
); | ||
} | ||
return smolvlaSnippets; | ||
} | ||
return []; | ||
}; | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We usually demonstrate simple inference snippets. Does that make sense for smolvla, or not really? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. agree with @pcuenca that an inference example feels more natural at this stage since if a user has pushed their model, it means it has already been trained (at least I expect). FYI, There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, I see what you mean. However, unfortunately we cannot use smolvla_base for inference directly, the only use case for smolvla_base for now is fine-tuning. We could have pushed the inference snippet for any model fine-tuned on top of smolvla_base however, but that is not really applicable here I think. wdyt? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. (originally posted in DMs on slack)
Since I see there are 380+ models on https://huggingface.co/models?other=lerobot and only 1 is not finetuned (if I understand correctly?) so it would be a shame not to show the inference command for all of them. What do you think? |
||
//#endregion |
Uh oh!
There was an error while loading. Please reload this page.