Add support for global layer options #563
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This allows to configure model layers at Axon-compilation time. This is useful if we want to make some compilation-time decisions when calling the model, without building a different model altogether.
API-wise the options are passed as
Those options are merged into layer options, but only when declared explicitly, like so:
We could pass them nested too and access as
opts[:global_options][:output_hidden_states]
, but this would break all existing layers that have options allowlist usingkeyword!
. I actually think that merging them in a flat manner is better, because they can be validated withkeyword!
in the exact same way.Naming suggestions are welcome, for example I was considering
:global_layer_options -> :layer_options
and:global_options -> :expose_options
, though not sure if "expose" is clear enough.