.. _readme:

.. |project| replace:: trident
.. _project: https://www.github.com/fdschmidt93/trident/


Notices
=======

|project| currently undergoes heavy development. The README and associated documentation needs to be vastly updated to reflect the changes in the code. That said, the documentation for individual functions in many cases still works as expected.

Using |project|
***************

|project| is generic framework to train and evaluate (deep learning) models and represents a thin layer of convenience on top of

* `Pytorch`
* `Lightning`
* `Hydra`
* `datasets`
* `transformers`
* `TorchMetrics`

In particular, |project| intends to abstract any boilerplate away in a highly modular fashion:

* Data & Preprocessing pipeline powered by `datasets <https://huggingface.co/docs/datasets/>`_ and :obj:`lightning.LightningDataModule`: user setup required
* Training and Evaluation wrappers constitute convenience layers around `Pytorch-Lightning <https://pytorch-lightning.readthedocs.io/>`_

You can opt-out of any component and replace it with your implementation with ease (see :ref:`customization`). It is quintessential that you familiarize yourself with Hydra as it constitutes the backbone for composing experiments in |project|.

Quick Start
===========

.. include:: ./docs/installation.rst

Paradigms
---------

* **Simplicity:** follow existing patterns (e.g. naming conventions) of our stack as much as possible for easy understanding and extending
* **Low code:** trident is a wrapper first -- prefer implementations from well-maintained frameworks
* **Out-of-the-box:** provide readily available pipelines for popular benchmark datasets

That said, you are encouraged to check the source code of the primary modules :doc:`src.modules` and :doc:`src.datamodules` which are thin wrappers around `LightningModule <https://pytorch-lightning.readthedocs.io/en/latest/common/lightning_module.html>`_ and `LightningDataModule <https://pytorch-lightning.readthedocs.io/en/latest/extensions/datamodules.html>`_\, respectively.

Usage
-----

Typical usage of trident follows the below schema:

1. Clone the repo
2. Write a configuration for your model (see also :ref:`customization`)
3. Train on an existing experiment with `python run.py experiment=mnli module=my_model`

You can find existing pipelines at :repo:`experiments configs <configs/experiment/>`. A full experiment (incl. `module`) is defined in the :repo:`MNLI-TinyBert config <configs/experiment/mnli_tinybert.yaml>`.

Project Structure
-----------------

**Important Concepts**:

* `module` encapsulates a :obj:`lightning.LightningModule` in a Hydra config (example: :repo:`sequence classification <configs/module/sequence_classification.yaml>`) and comprises the following keys:
    
    - model: your model that returns a container with loss and required attributes
    - :ref:`evaluation <evaluation>`: use and existing or customized evaluation config
    - optimizer: the optimizer for training
    - scheduler: the scheduler for optimization
    - :ref:`overrides <function-override>`: override and add single methods of the :obj:`TridentModule`
    - :ref:`mixins <mixins>`: override and add groups of methods of the :obj:`TridentModule`

    Please see more details in :ref:`trident_module`.

* `datamodule` represents a :obj:`lightning.LightningDataModule` in a Hydra config (example: :repo:`MNLI <configs/datamodule/mnli.yaml>`)

* `experiment` constitutes a set of configuration that typically defines everything except for the `model`

.. code-block::
    
    ├── configs             <- Hydra configuration files
    │   ├── callbacks         <- Model training callbacks
    │   ├── datamodule        <- Datamodule configs
    │   ├── experiment        <- Experiment configs encapsulate **at least** everything except model (but may include models)
    │   ├── hparams_search    <- Hyperparameter search configs
    │   ├── hydra             <- Hydra related configs
    │   ├── logger            <- Logger configs
    │   ├── module            <- Module configs: model (mixins & overrides), optimizer, scheduler
    │   ├── trainer           <- Trainer configs
    │   │
    │   └── config.yaml       <- Main project configuration file
    │
    ├── logs                <- Logs generated by Hydra & PyTorch Lightning loggers
    │
    ├── tests               <- Tests of any kind
    │   ├── helpers           <- A couple of testing utilities
    │   ├── shell             <- Shell/command based tests
    │   └── unit              <- Unit tests
    │
    ├── src
    │   ├── callbacks           
    │   ├── datamodules         
    │   ├── modules             
    │   │  ├──── base         <- Definition of TridentModule
    │   │  ├──── mixins       <- Extend TridentModule with methods
    │   │  └──── functional   <- Extend TridentModule functionally
    │   ├── utils               
    │   │
    │   └── train.py          <- Training pipeline


.. _trident_module:

(Trident)Module
---------------

Most importantly, |project| allows you to:

- Use your own LightningModules
- Embed your own model into the TridentModule
- Easily leverage existing frameworks like `transformers <https://huggingface.co/transformers/>`_ (see below)

A `TridentModule` is analogous to what a :obj:`lightning.LightningDataModule` comprises (model, optimizer, scheduler, ...) and commonly defined with **shared and individual flags**. Here's an example for sequence classification.

First, the included model configurations inherit **shared options** from the :repo:`base module config <configs/module/base.yaml>`.

.. code-block:: yaml

    _recursive_: false
    _target_: src.modules.base.TridentModule
    defaults:
    - /optimizer: ${optimizer}  # global default (adamw: /configs/optimizer/adamw.yaml
    - /scheduler: ${scheduler}  # global default (linear warmup: /configs/scheduler/linear_warm_up.yaml)

`/configs/module/sequence_classification.yaml`

For instance, you can then 

.. code-block:: bash

    # change the learning rate
    python run.py experiment=mnli optimizer.lr=0.0001
    # set a different optimizer
    python run.py experiment=mnli optimizer=adam  # see /configs/optimizer/
    # no lr scheduler
    python run.py experiment=mnli scheduler=null



.. code-block:: yaml

    defaults:
    - base # inherit config from /configs/module/base.yaml
    - /evaluation: classification # 
    # optionally
    - /overrides: ...
    
    # you can set instructions here on how to embed your own model
    model:
      _target_: transformers.AutoModelForSequenceClassification.from_pretrained
      num_labels: ???
      pretrained_model_name_or_path: ???

Should you embed your own model, a TridentModule requires the model to `forward` be analogous to Huggingface transformers models, see the :py:meth:`src.modules.base.TridentModule.training_step`. Your model output should contain all required attributes for evaluation.

The `TridentModule` by default incorporates mixins for optimizer configuration and custom evaluation loops for the respective tasks (see :ref:`evaluation`).

DataModule
----------

TODO

Extensibility
-------------

trident allows you to integrate your existing projects via:

* Extend Python path: add an existing project to scope of trident easily
* Hook required functionality into training via
    1. Linking your own functions
    2. Pytorch-Lightning
    3. Explicit method overrides
    4. Incorporating custom mixins


For details on how to incorporate custom models or datamodules into the framework, see :ref:`customization`.

Contributing
------------

Please see :ref:`Contributing <./docs/source/CONTRIBUTING.rst>`!


Credits
-------

* This project is was largely inspired by and is based on https://github.com/ashleve/lightning-hydra-template
* A related project is: https://github.com/PyTorchLightning/lightning-transformers

Author
-------

| Name: Fabian David Schmidt
| Mail: fabian.schmidt@uni-wuerzburg.de
| Affiliation: University of Würzburg