PyTorch implementation of ml-params CLI API
pip install -r requirements.txt
pip install .
After installing as above, follow usage from https://github.com/SamuelMarks/ml-params
| Other vendors | |
|---|---|
| tensorflow | pytorch |
| keras | skorch |
| flax | sklearn |
| trax | xgboost |
| jax | cntk |
To make the development of ml-params-pytorch type safer and maintain consistency with the other ml-params implementing projects, the cdd was created.
When PyTorch itself changes—i.e., a new major version of PyTorch is released—then run the sync_properties, as shown in the module-level docstring here ml_params_pytorch/ml_params/type_generators.py;
To synchronise all the various other APIs, edit one and it'll translate to the others, but make sure you select which one is the gold-standard.
As an example, using the class TorchTrainer methods as truth, this will update the CLI parsers and config classes:
python -m cdd sync --class 'ml_params_pytorch/ml_params/config.py' \
--class-name 'TrainConfig' \
--function 'ml_params_pytorch/ml_params/trainer.py' \
--function-name 'TorchTrainer.train' \
--argparse-function 'ml_params_pytorch/ml_params/cli.py' \
--argparse-function-name 'train_parser' \
--truth 'function'
python -m cdd sync --class 'ml_params_pytorch/ml_params/config.py' \
--class-name 'LoadDataConfig' \
--function 'ml_params_pytorch/ml_params/trainer.py' \
--function-name 'TorchTrainer.load_data' \
--argparse-function 'ml_params_pytorch/ml_params/cli.py' \
--argparse-function-name 'load_data_parser' \
--truth 'function'
python -m cdd sync --class 'ml_params_pytorch/ml_params/config.py' \
--class-name 'LoadModelConfig' \
--function 'ml_params_pytorch/ml_params/trainer.py' \
--function-name 'TorchTrainer.load_model' \
--argparse-function 'ml_params_pytorch/ml_params/cli.py' \
--argparse-function-name 'load_model_parser' \
--truth 'function'
To generate custom config CLI parsers, run this before ^:
$ for name in 'activations' 'losses' 'optimizer_lr_schedulers' 'optimizers'; do
rm 'ml_params_pytorch/ml_params/'"$name"'.py';
python -m ml_params_pytorch.ml_params.cdd_cli_gen "$name" 2>/dev/null | xargs python -m cdd gen;
done
To see what this is doing, here it is expanded for datasets:
$ python -m cdd gen --name-tpl '{name}Config'
--input-mapping 'ml_params_pytorch.ml_params.type_generators.exposed_datasets'
--emit 'argparse'
--output-filename 'ml_params_pytorch/ml_params/datasets.py'
Cleanup the code everywhere, removing unused imports and autolinting/autoformatting:
$ fd -epy -x autoflake --remove-all-unused-imports -i {} \;
$ isort --atomic .
$ python -m black .
Licensed under either of
- Apache License, Version 2.0 (LICENSE-APACHE or https://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or https://opensource.org/licenses/MIT)
at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.