Tensorflow with ftc‐ml toolchain tips
You should be referring to this link to train your model. It will contain the vast majority of the instructions on how to train the model; these are tips that the team has accumulated from past experience, and now share with you.
https://ftc-docs.firstinspires.org/en/latest/ftc_ml/index.html
Here are some general tips:
To work with the ftc-ml toolchain, you have to be added to the team on FTC dashboard and given the Team Member role. Create a FTC dashboard account first, and then be relentless with this; talk to him about it every chance you get. Don't patiently wait for him until a week before the deadline like I did. Make your emails short and snappy to increase the chances of him reading it, and email him again if he doesn't do it.
These are drastic measures, but it's better than not having a TFOD model until 2 days before the deadline.
Also, only Mentors can upload the videos to the ftc-ml toolchain platform. So send him a link to a Google Drive with the videos, and do this as soon as possible, because he won't upload them very fast.
Make your videos as soon as possible, according to Optimizing Videos for Increased Tensorflow Model Performance, and give them to a senior for review just to make sure you haven't messed up somewhere.
Read all of the pages before starting to train, especially Optimizing Videos for Increased Tensorflow Model Performance, which is easy to miss.
This can be found in https://ftc-docs.firstinspires.org/en/latest/ftc_ml/managing_tool/training_models/training-models.html
- Epoches is 100 (can be adjusted according to model metrics, but unclear what those are)
- Batch size is 32
- Steps is 6000 (it takes 60 minutes to train for 3,000 steps, you can train for 600 minutes, so the maximum steps is 30000)
- This gives us how many frames we can actually have in a video: 9600.
To be safe, keep it at 9500 so you don't go overboard.
For reference, that is around 150 seconds of 60 fps, or 300 seconds of 30 fps. If given a choice, go for 30 fps, as the lower the fps the better for training (due to less duplicate frames and less time wasted for the model).
There is a screenshot on the ftc-ml toolchain saying that there is only 120 minutes of training time. This is, at the date of writing (2023 to 2024 season), wrong. You can train for 600 minutes (10 hours).
ftc-ml toolchain uses a object tracker to automatically label your videos. If you speed it up by 8x, that'll break. 2x could work, but just in case, don't.