Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parameter metadata to the trainer schema. #2224

Merged
merged 7 commits into from
Jul 5, 2022

Conversation

justinxzhao
Copy link
Contributor

No description provided.

@justinxzhao justinxzhao marked this pull request as ready for review June 30, 2022 19:55
@github-actions
Copy link

github-actions bot commented Jun 30, 2022

Unit Test Results

       6 files  ±0         6 suites  ±0   2h 40m 38s ⏱️ + 3m 28s
2 908 tests ±0  2 862 ✔️ ±0    46 💤 ±0  0 ±0 
8 724 runs  ±0  8 582 ✔️ ±0  142 💤 ±0  0 ±0 

Results for commit 48023a3. ± Comparison against base commit 9d7da5f.

♻️ This comment has been updated with latest results.

ludwig/schema/metadata/trainer_metadata.py Show resolved Hide resolved
ludwig/schema/metadata/trainer_metadata.py Outdated Show resolved Hide resolved
ludwig/schema/optimizers.py Outdated Show resolved Hide resolved
ludwig/schema/trainer.py Outdated Show resolved Hide resolved
ludwig/schema/utils.py Show resolved Hide resolved
@justinxzhao justinxzhao requested a review from ksbrar June 30, 2022 22:11
@@ -354,13 +355,14 @@ class GradientClippingConfig(BaseMarshmallowConfig):
clipvalue: Optional[float] = FloatRange(default=None, allow_none=True, description="")


def GradientClippingDataclassField(default={}, allow_none=True, description="TODO"):
def GradientClippingDataclassField(description: str, default: Dict):
Copy link
Collaborator

@ksbrar ksbrar Jun 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would still keep {} as the default value for default so it's visually clear that in the case of GradientClippingDataclassField an empty dict is valid (whereas e.g. for OptimizerDataclassField you have to at least provide a type inside the dict)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added back the default value for default for now.

Outside of this PR, we should consider removing the default values for marshmallow field function arguments. This will make some of the marshmallow unit tests more verbose, but it ensures that the field's parameters (default, description, etc.) are defined in one location, at the point of instantiation. For example, right now for OptimizerDataclassField, we declare the default in two places -- as a function parameter default via the marshmallow field definition, as well as at the point of instantiation in ECDTrainerConfig.

Here's what I'm thinking:

  • Always include description and default as required arguments so that it's impossible to instantiate a marshmallow field without explicitly specifying the description and default. This setup ensures that the defaults and descriptions are concisely listed where parameters are declared, i.e. ECDTrainerConfig.
  • Include other arguments like allow_none in the marshmallow field only if we want this to be able to be configured differently across different instances of the field. For example, we should remove allow_none from the list of parameters for GradientClippingDataclassField since it's always True.
  • Use ParameterMetadata to contain all other "optional" information pertaining to the field. Eventually, this will also be a required argument.

See also, why default arguments are bad.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed on all counts!

ludwig/schema/trainer.py Show resolved Hide resolved
@justinxzhao justinxzhao merged commit f61ad2d into master Jul 5, 2022
@justinxzhao justinxzhao deleted the trainer_parameter_metadata branch July 5, 2022 21:17
@justinxzhao justinxzhao linked an issue Jul 5, 2022 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[IterML] Add parameter metadata for Trainer parameters.
3 participants