first commit

.gitignore

@@ -0,0 +1,20 @@
+data/
+**/__pycache__/
+logs/*
+experiments/logs
+!logs/.gitkeep
+datasets/* 
+!datasets/*.sh
+.vscode/
+*.egg-info/
+eggs/
+.eggs/
+*.egg
+**.egg
+build/
+_build/
+**/build/
+outputs/
+log.txt
+**/DeltaHub/
+*beans

.readthedocs.yaml

@@ -0,0 +1,29 @@
+# .readthedocs.yaml
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 1
+
+# Set the version of Python and other tools you might need
+build:
+  os: ubuntu-20.04
+  tools:
+    python: "3.9"
+    # You can also specify other tool versions:
+    # nodejs: "16"
+    # rust: "1.55"
+    # golang: "1.17"
+
+# Build documentation in the docs/ directory with Sphinx
+sphinx:
+   configuration: docs/conf.py
+
+# If using Sphinx, optionally build your docs in additional formats such as PDF
+# formats:
+#    - pdf
+
+# Optionally declare the Python requirements required to build your docs
+python:
+   install:
+   - requirements: docs/requirements.txt

README.md

@@ -0,0 +1,94 @@
+<div align="center">
+
+
+<img src="https://s4.ax1x.com/2022/02/14/Hy7lAf.png" width="350px">
+
+**An Open-Source Framework for Paramter Efficient Tuning.**
+
+------
+
+<p align="center">
+  <a href="#Overview">Overview</a> •
+  <a href="#installation">Installation</a> •
+  <a href="#Supported-Models">Supported Models</a> •
+  <a href="https://opendelta.readthedocs.io/">Docs</a> • 
+  <a href="https://docs.google.com/spreadsheets/d/1BIVa8ocAPga-u7rBOXLYaTfaJSjI1dWfwohmLjmFDrY/edit?usp=sharing">Performance</a> •
+
+
+</p>
+
+</div>
+
+![version](https://img.shields.io/badge/version-v0.1.0-blue)
+
+## Overview
+
+OpenDelta is a toolkit for parameter efficient methods (we dub it as *delta tuning*), by which users could flexibly assign (or add) a small amount parameters to update while keeping the most paramters frozen. By using OpenDelta, users could easily implement prefix-tuning, adapters, Lora, or any other types of delta tuning with preferred PTMs.
+
+## Installation
+create a virtualenv (optional)
+```shell
+conda create -n opendelta_env python=3.8
+conda activate opendelta_env
+```
+
+### Using Pip
+
+Our repo is tested on Python 3.6+ and PyTorch 1.8.1+, install OpenDelta using pip as follows:
+
+```shell
+pip install opendelta
+```
+
+To play with the latest features, you can also install OpenDelta from the source.
+
+### Build from Source
+
+```shell
+git clone https://github.com/thunlp/OpenDelta.git
+cd OpenDelta
+``` 
+
+#### Option 1: If you won't modify the code, run
+```shell
+python setup.py install
+```
+
+#### Option 2:  If you want to modify the code, run
+```shell
+python setup.py develop
+```
+
+
+
+### Verified Supported Models
+
+** You can try to use OpenDelta on any backbone models based on PyTorch.** However, with small chances that
+The interface of the submodules of the backbone model is not supported. Therefore we verified some commonly
+used models that OpenDelta are sure to support.
+
+We will keep testing more and more emerging models.
+
+Pull requests are welcomed when you successfully apply OpenDelta on your own backbone model.
+
+
+|            | Lora | Bias<br>Tuning  | Adapter<br>Houstbly | Adapter<br>Preffier  | Adapter<br>Drop  | Adapater<br> Low-Rank   | Compactor  |Prefix<br> Tuning      | Prompt <br> Tuning |
+| --------- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ----- | ----- | 
+| T5             | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  |
+| GPT-2          | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  |     |
+| BART           | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  |     | 
+| DistilBERT     | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  |     | 
+| RoBERTa        | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  |     |
+| BERT           | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  |
+| T5-3b(parallel)| ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  |
+| Deberta-v2     | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  |     |     |
+| CTRL           | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  | ✅  |     |     |
+| ViT            | ✅  |     |    |     |     |      |   |     |     |
+
+
+### Performance Checked Combination
+
+Google sheet [here](https://docs.google.com/spreadsheets/d/1BIVa8ocAPga-u7rBOXLYaTfaJSjI1dWfwohmLjmFDrY/edit?usp=sharing)
+
+
+

docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.https://www.sphinx-doc.org/
+	exit /b 1
+)
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd

docs/readme.md

@@ -0,0 +1,20 @@
+# OpenDelta Documentation
+
+To build this doc locally, please firstly install [sphinx](https://www.sphinx-doc.org/en/master/) packages.
+
+```
+pip install sphinx
+pip install sphinx_rtd_theme
+pip install sphinx_copybutton
+pip install sphinx_toolbox
+pip install myst_parser
+```
+
+Then install opendelta either from source, or from pip. After that,
+
+```
+cd docs
+make html
+```
+
+Then open the generated `docs/build/html/index.html` in your local browser. 

docs/requirements.txt

@@ -0,0 +1,13 @@
+sphinx_copybutton
+sphinx_rtd_theme
+sphinx_toolbox
+torch
+transformers
+sentencepiece==0.1.96
+tqdm==4.62.2
+openprompt
+loralib
+decorator
+rich
+myst_parser
+web.py