Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tutorial about using Lightning and Fastai callbacks for model checkpoint by metric #1145

Open
medphisiker opened this issue Oct 16, 2022 · 1 comment
Labels
documentation Improvements or additions to documentation example request good first issue Good for newcomers help wanted Extra attention is needed

Comments

@medphisiker
Copy link

medphisiker commented Oct 16, 2022

📓 New <Tutorial/Example>

Is this a request for a tutorial or for an example?
I think it will be very great to see example about using Lightning and Fastai callbacks for model check point by metric.
For example, we train some model like in tutorial https://airctic.com/0.7.0/getting_started_object_detection/.
And we have a great result by COCO metric on 10th epoch, and bad on 25th epoch.
If we use callback by metric we will be able to save checkpoints on great metric values.
I think it will be great to add code example in this tutorial https://airctic.com/0.7.0/getting_started_object_detection/.

I use them in pure Lightning. To do it we need to create special metric loggers in Lightning model class like train_on_epoch() and ect.
But in Ice Vision framework we get predefined class.
Unfortunately, I didn't understand how to write methods for collecting metrics and using callbacks for Ice Vision Lightning Callbacks.
Or Lightning function for finding learning rate. Example with fastai backend show how to do that.

What is the task?
First of all I am intresting in Object detection. But it looks like it is a general opportunity for all ice vision tasks.
As I understand ice vision wokrs with fastai and Lightning backend for all of its CV tasks.

Is this example for a specific model?
No, it is a general example. I think for all models in tutorial https://airctic.com/0.7.0/getting_started_object_detection/

Is this example for a specific dataset?
No.


Don't remove
Main issue for examples: #39

@medphisiker medphisiker added documentation Improvements or additions to documentation example request good first issue Good for newcomers help wanted Extra attention is needed labels Oct 16, 2022
@m-ali-awan
Copy link

Hi @medphisiker
You can use in fastai as follows:

from icevision.all import *
from fastai.callback.tracker import EarlyStoppingCallback,ReduceLROnPlateau,SaveModelCallback

learn = model_type.fastai.learner(dls=[train_dl, valid_dl], model=model,
                                    metrics=metrics,model_dir = icevision_models_pth)

learn.fine_tune(5, 1e-3, freeze_epochs = 1,
                  cbs=[EarlyStoppingCallback(min_delta = 0.01,patience = 3),
                       ReduceLROnPlateau(factor = 5),
                       SaveModelCallback(fname='only-scratch-with-3tasks-data')])

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation example request good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants