NeuroSurgeon is a python toolkit built to enable deep learning researchers to easily uncover and manipulate subnetworks within trained models. NeuroSurgeon provides a simple API to inject differentiable binary masks techniques into linear, attention, and convolution layers in BERT, GPT, ResNet, and ViT-style models within Huggingface Transformers. Differentiable masking has a variety of use cases for deep learning research, such as:
- Pruning to uncover functional subnetworks
- Relevant Papers:
- Subnetwork Probing
- Relevant Papers:
- Training with L0 Regularization
- Relevant Papers:
Read the NeuroSurgeon Documentation
To get started with NeuroSurgeon, check out the tutorial here. This covers the basic workflow for using NeuroSurgeon to uncover functional subnetworks within a trained model.
NeuroSurgeon requires python 3.9 or higher and several libraries, including Transformers and PyTorch. Installation can be done using PyPi:
pip install NeuroSurgeon
NeuroSurgeon's logo was created with the help of DALL-E 2, using the prompt "A cute cartoon robot doctor smiling with a stethoscope".