Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Major refactoring of ModelTrainer #3182

Merged
merged 87 commits into from Apr 14, 2023
Merged

Conversation

alanakbik
Copy link
Collaborator

@alanakbik alanakbik commented Apr 3, 2023

This PR is mostly based on #3084 by @plonerma and executes a major refactoring of the ModelTrainer.

Why? Since Flair 0.1 a very large amount of features were added to the ModelTrainer, causing the class to become very bloated and difficult to extend/debug.

This PR refactors the trainer in several ways:

  • it adds a Plugin logic for all non-essential features. Users can write new plugins that attach to the trainer through callbacks in different parts of the training. This gives users hopefully more flexibility in adding special features to the Flair trainer (like specific logic frameworks) .
  • after years of "feature bloat", some little-used features were removed. This also slims down the number of parameters the trainer can take.
  • similar to before, the train and fine_tune methods are "best practice" configurations for model training. A new train_custom method is added to this that gives users maximal flexibility when doing model training.

We hope that a more structured ModelTrainer will allow us to better integrate new features in the near future, such as multi-GPU and more logging support.

@helpmefindaname helpmefindaname marked this pull request as draft April 3, 2023 11:10
@alanakbik alanakbik marked this pull request as ready for review April 11, 2023 09:24
Copy link
Collaborator

@plonerma plonerma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The introduction of train_custom in combination with the other functions which offer a set of presets makes the Trainer more flexible. Overall, the train functions are much more compact.

I am a bit unsure about the events which are not currently used in any of the plugins. I can see that one might only want to keep those which are referenced by plugins in the repository. However, I already started using some of them in my experiments and think they offer a much greater potential for extending the train function (without the need to derive a trainer class with an overwritten train function).

flair/datasets/base.py Show resolved Hide resolved
flair/trainers/plugins/loggers/log_file.py Outdated Show resolved Hide resolved
flair/trainers/plugins/loggers/loss_file.py Outdated Show resolved Hide resolved
flair/trainers/trainer.py Outdated Show resolved Hide resolved
flair/trainers/trainer.py Outdated Show resolved Hide resolved
flair/trainers/trainer.py Outdated Show resolved Hide resolved
@alanakbik alanakbik merged commit a94682e into master Apr 14, 2023
1 check passed
@alanakbik alanakbik deleted the pluggable_trainer_detached branch April 14, 2023 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants