Sparse updates is a technique that train neural networks only updating few parameters.
We follow the black format.
It's recommended to install in conda or virtual enviroment.
cd sparse_update
pip install -e .
Please refer to scripts/train.py
python scripts/train.py
The script will also generate predictions on test data (save as [Dataset].tsv), which is used to submit to the Glue server for testing.
- Support sparsed update (diff pruning)
- Support all Glue tasks (Add corresponding classes in
datamodules.pyandmodules.py) - Support pretraining tasks
The repo base on the following projects