Repository navigation
Dependencies and Environment
The original requirements.txt includes torch, transformers, datasets, accelerate, PEFT, TRL, bitsandbytes, numpy, pandas, scikit-learn, and Jupyter packages. The package also includes requirements_distilbert.txt for the classifier-only workflow.
py -3.12 -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -r requirements_distilbert.txt
python -m pip checkCheck CUDA configuration separately:
python -c "import torch; print(torch.__version__, torch.version.cuda, torch.cuda.is_available())"A GPU driver alone is not sufficient if PyTorch was installed as CPU-only. For GPU training, use the official PyTorch installation selector. Package compatibility must be verified in the target environment; no downloads or full training runs were executed while producing this Wiki.
Transformer Training Wiki · Implementation status: DistilBERT source available.This project is licensed under the Apache License, Version 2.0.© 2026 Sudhanshu Shukla · Licensed under the Apache License, Version 2.0
Overview
Architecture
Implemented classifier
- DistilBERT-Workflow
- Dataset-and-Labels
- Training-Configuration
- Evaluation-and-Inference
- Frozen-Backbone-Comparison
- Artifacts-and-Checkpoints
Experimental alignment draft
Operations