test: Switched to pytest suite (#131)

* refactor: Removed OneCycle

* docs: Updated README & docs

* test: Switched to pytest

* chore: Updated CI job

* fix: Fixed loss implementation

* style: Fixed lint

* docs: Updated CONTRIBUTING

* docs: Updated README

* test: Fixed typo in unittests

.github/workflows/main.yml

@@ -57,7 +57,7 @@ jobs:
 
       - name: Run unittests
         run: |
-          coverage run -m unittest discover test/
+          coverage run -m pytest test/
           coverage xml
 
       - name: Upload coverage to Codecov

.github/workflows/requirements.txt

@@ -1 +1,4 @@
+pytest>=5.3.2
 coverage>=4.5.4
+torchvision>=0.9.0
+

CONTRIBUTING.md

@@ -8,6 +8,8 @@ Everything you need to know to contribute efficiently to the project.
 
 - [holocron](https://github.com/frgfm/Holocron/blob/master/holocron) - The actual holocron library
 - [test](https://github.com/frgfm/Holocron/blob/master/test) - Python unit tests
+- [docs](https://github.com/frgfm/Holocron/blob/master/docs) - Sphinx documentation building
+- [references](https://github.com/frgfm/Holocron/blob/master/references) - Reference training scripts
 
 
 
@@ -36,3 +38,31 @@ Use Github [issues](https://github.com/frgfm/Holocron/issues) for feature reques
 
 - **Code**: ensure to provide docstrings to your Python code. In doing so, please follow [Google-style](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html) so it can ease the process of documentation later.
 - **Commit message**: please follow [Udacity guide](http://udacity.github.io/git-styleguide/)
+
+### Running CI verifications locally
+
+#### Unit tests
+
+In order to run the same unit tests as the CI workflows, you can run unittests locally:
+
+```shell
+pytest test/
+```
+
+#### Lint verification
+
+To ensure that your incoming PR complies with the lint settings, you need to install [flake8](https://flake8.pycqa.org/en/latest/) and run the following command from the repository's root folder:
+
+```shell
+flake8 ./
+```
+This will read the `.flake8` setting file and let you know whether your commits need some adjustments.
+
+#### Annotation typing
+
+Additionally, to catch type-related issues and have a cleaner codebase, annotation typing are expected. After installing [mypy](https://github.com/python/mypy), you can run the verifications as follows:
+
+```shell
+mypy --config-file mypy.ini
+```
+The `mypy.ini` file will be read to check your typing.

README.md

@@ -1,111 +1,87 @@
 # Holocron
 
-[![License: Apache 2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE) [![Codacy Badge](https://api.codacy.com/project/badge/Grade/5713eafaf8074e27a4013dbfcfad9d69)](https://www.codacy.com/manual/fg/Holocron?utm_source=github.com&utm_medium=referral&utm_content=frgfm/Holocron&utm_campaign=Badge_Grade) ![Build Status](https://github.com/frgfm/Holocron/workflows/python-package/badge.svg) [![codecov](https://codecov.io/gh/frgfm/Holocron/branch/master/graph/badge.svg)](https://codecov.io/gh/frgfm/Holocron) [![Docs](https://img.shields.io/badge/docs-available-blue.svg)](https://frgfm.github.io/Holocron)
+[![License: Apache 2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE) [![Codacy Badge](https://api.codacy.com/project/badge/Grade/5713eafaf8074e27a4013dbfcfad9d69)](https://www.codacy.com/manual/fg/Holocron?utm_source=github.com&utm_medium=referral&utm_content=frgfm/Holocron&utm_campaign=Badge_Grade) ![Build Status](https://github.com/frgfm/Holocron/workflows/python-package/badge.svg) [![codecov](https://codecov.io/gh/frgfm/Holocron/branch/master/graph/badge.svg)](https://codecov.io/gh/frgfm/Holocron) [![Docs](https://img.shields.io/badge/docs-available-blue.svg)](https://frgfm.github.io/Holocron)   [![Pypi](https://img.shields.io/badge/pypi-v0.1.3-blue.svg)](https://pypi.org/project/pylocron/) 
 
 Implementations of recent Deep Learning tricks in Computer Vision, easily paired up with your favorite framework and model zoo.
 
 > **Holocrons** were information-storage [datacron](https://starwars.fandom.com/wiki/Datacron) devices used by both the [Jedi Order](https://starwars.fandom.com/wiki/Jedi_Order) and the [Sith](https://starwars.fandom.com/wiki/Sith) that contained ancient lessons or valuable information in [holographic](https://starwars.fandom.com/wiki/Hologram) form.
 
 *Source: Wookieepedia*
 
-
-
-## Table of Contents
-
-- [Getting Started](#getting-started)
-  - [Prerequisites](#prerequisites)
-  - [Installation](#installation)
-- [Usage](#usage)
-- [Technical Roadmap](#technical-roadmap)
-- [Documentation](#documentation)
-- [Contributing](#contributing)
-- [License](#license)
-
-
-
 *Note: support of activation mapper and model summary has been dropped and outsourced to independent packages ([torch-cam](https://github.com/frgfm/torch-cam) & [torch-scan](https://github.com/frgfm/torch-scan)) to clarify project scope.*
 
+## Quick Tour
 
-
-## Getting started
-
-### Prerequisites
-
-- Python 3.6 (or more recent)
-- [pip](https://pip.pypa.io/en/stable/)
-
-### Installation
-
-You can install the package using [pypi](https://pypi.org/project/pylocronn/) as follows:
-
-```bash
-pip install pylocron
-```
-
-or using [conda](https://anaconda.org/frgfm/pylocron):
-
-```bash
-conda install -c frgfm pylocron
-```
-
-
-
-## Usage
-
-### nn
-
-##### Main features
-
+### PyTorch layers for every need
 - Activation: [SiLU/Swish](https://arxiv.org/abs/1606.08415), [Mish](https://arxiv.org/abs/1908.08681), [HardMish](https://github.com/digantamisra98/H-Mish), [NLReLU](https://arxiv.org/abs/1908.03682), [FReLU](https://arxiv.org/abs/2007.11824)
 - Loss: [Focal Loss](https://arxiv.org/abs/1708.02002), MultiLabelCrossEntropy, [LabelSmoothingCrossEntropy](https://arxiv.org/pdf/1706.03762.pdf), [MixupLoss](https://arxiv.org/pdf/1710.09412.pdf), [ClassBalancedWrapper](https://arxiv.org/abs/1901.05555), [ComplementCrossEntropy](https://arxiv.org/abs/2009.02189), [MutualChannelLoss](https://arxiv.org/abs/2002.04264)
 - Convolutions: [NormConv2d](https://arxiv.org/pdf/2005.05274v2.pdf), [Add2d](https://arxiv.org/pdf/1912.13200.pdf), [SlimConv2d](https://arxiv.org/pdf/2003.07469.pdf), [PyConv2d](https://arxiv.org/abs/2006.11538)
 - Regularization: [DropBlock](https://arxiv.org/abs/1810.12890)
 - Pooling: [BlurPool2d](https://arxiv.org/abs/1904.11486), [SPP](https://arxiv.org/abs/1406.4729), [ZPool](https://arxiv.org/abs/2010.03045)
 - Attention: [SAM](https://arxiv.org/abs/1807.06521), [LambdaLayer](https://openreview.net/forum?id=xTJEN-ggl1b), [TripletAttention](https://arxiv.org/abs/2010.03045)
 
-### models
-
-##### Main features
-
+### Models for vision tasks
 - Classification: [Res2Net](https://arxiv.org/abs/1904.01169) (based on the great [implementation](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/res2net.py) from Ross Wightman), [Darknet-24](https://pjreddie.com/media/files/papers/yolo_1.pdf), [Darknet-19](https://pjreddie.com/media/files/papers/YOLO9000.pdf), [Darknet-53](https://pjreddie.com/media/files/papers/YOLOv3.pdf), [CSPDarknet-53](<https://arxiv.org/abs/1911.11929>), [ResNet](https://arxiv.org/abs/1512.03385), [ResNeXt](https://arxiv.org/abs/1611.05431), [TridentNet](https://arxiv.org/abs/1901.01892), [PyConvResNet](https://arxiv.org/abs/2006.11538), [ReXNet](https://arxiv.org/abs/2007.00992), [SKNet](https://arxiv.org/abs/1903.06586), [RepVGG](https://arxiv.org/abs/2101.03697).
 - Detection: [YOLOv1](https://pjreddie.com/media/files/papers/yolo_1.pdf), [YOLOv2](https://pjreddie.com/media/files/papers/YOLO9000.pdf), [YOLOv4](https://arxiv.org/abs/2004.10934)
 - Segmentation: [U-Net](https://arxiv.org/abs/1505.04597), [UNet++](https://arxiv.org/abs/1807.10165), [UNet3+](https://arxiv.org/abs/2004.08790)
 
-### ops
+### Vision-related operations
+- boxes: [Distance-IoU & Complete-IoU losses](https://arxiv.org/abs/1911.08287)
 
-##### Main features
+### Trying something else than Adam
+- Optimizer: [LARS](https://arxiv.org/abs/1708.03888), [Lamb](https://arxiv.org/abs/1904.00962), [RAdam](https://arxiv.org/abs/1908.03265), [TAdam](https://arxiv.org/pdf/2003.00179.pdf), [AdamP](https://arxiv.org/pdf/2006.08217), [AdaBelief](https://arxiv.org/abs/2010.07468), and customized versions (RaLars)
+- Optimizer wrapper: [Lookahead](https://arxiv.org/abs/1907.08610), Scout (experimental)
 
-- boxes: [Distance-IoU & Complete-IoU losses](https://arxiv.org/abs/1911.08287)
 
-### optim
 
-##### Main features
+## Setup
 
-- Optimizer: [LARS](https://arxiv.org/abs/1708.03888), [Lamb](https://arxiv.org/abs/1904.00962), [RAdam](https://arxiv.org/abs/1908.03265), [TAdam](https://arxiv.org/pdf/2003.00179.pdf), [AdamP](https://arxiv.org/pdf/2006.08217), [AdaBelief](https://arxiv.org/abs/2010.07468), and customized versions (RaLars)
-- Optimizer wrapper: [Lookahead](https://arxiv.org/abs/1907.08610), Scout (experimental)
-- Scheduler: [OneCycleScheduler](https://arxiv.org/abs/1803.09820) *(this implementation was made before PyTorch officially had an [implementation](https://pytorch.org/docs/stable/optim.html#torch.optim.lr_scheduler.OneCycleLR), for better support it is advised to consider the official PyTorch version)*
+Python 3.6 (or higher) and [pip](https://pip.pypa.io/en/stable/)/[conda](https://docs.conda.io/en/latest/miniconda.html) are required to install TorchCAM.
+
+### Stable release
 
+You can install the last stable release of the package using [pypi](https://pypi.org/project/pylocron/) as follows:
 
+```shell
+pip install pylocron
+```
 
-## Technical roadmap
+or using [conda](https://anaconda.org/frgfm/pylocron):
 
-The project is currently under development, here are the objectives for the next releases:
+```shell
+conda install -c frgfm pylocron
+```
 
-- [x] Standardize models: standardize models by task.
-- [x] Reference scripts: add reference training scripts
-- [ ] Speed benchmark: compare `holocron.nn` functions execution speed.
+### Developer installation
 
+Alternatively, if you wish to use the latest features of the project that haven't made their way to a release yet, you can install the package from source:
 
+```shell
+git clone https://github.com/frgfm/Holocron.git
+pip install -e Holocron/.
+```
 
-## Documentation
 
-The full package documentation is available [here](<https://frgfm.github.io/Holocron/>) for detailed specifications. The documentation was built with [Sphinx](sphinx-doc.org) using a [theme](github.com/readthedocs/sphinx_rtd_theme) provided by [Read the Docs](readthedocs.org) 
+## What else
 
+### Documentation
+
+The full package documentation is available [here](https://frgfm.github.io/holocron/) for detailed specifications.
+
+
+### Reference scripts
+
+Reference scripts are provided to train your models using holocron on famous public datasets. Those scripts currently support the following vision tasks:
+- [Image classification](references/classification)
+- [Object detection](references/detection)
+- [Semantic segmentation](references/segmentation)
 
 
 ## Contributing
 
-Please refer to `CONTRIBUTING` if you wish to contribute to this project.
+Any sort of contribution is greatly appreciated!
+
+You can find a short guide in [`CONTRIBUTING`](CONTRIBUTING) to help grow this project!
 
 
 

docs/source/optim.rst

@@ -42,23 +42,3 @@ should write your code this way:
 .. autoclass:: holocron.optim.wrapper.Lookahead
 
 .. autoclass:: holocron.optim.wrapper.Scout
-
-
-Learning rate schedulers
----------------------------
-
-:mod:`holocron.optim.lr_scheduler` provides several methods to adjust the learning
-rate based on the number of epochs. :class:`holocron.optim.lr_scheduler.OneCycleScheduler`
-allows dynamic learning rate reducing based on some validation measurements.
-
-Learning rate scheduling should be applied after optimizer's update; e.g., you
-should write your code this way:
-
-    >>> scheduler = ...
-    >>> for epoch in range(100):
-    >>>     train(...)
-    >>>     validate(...)
-    >>>     scheduler.step()
-
-.. autoclass:: holocron.optim.lr_scheduler.OneCycleScheduler
-

holocron/nn/functional.py

@@ -292,14 +292,16 @@ def complement_cross_entropy(
         torch.Tensor: loss reduced with `reduction` method
     """
 
+    ce_loss = F.cross_entropy(x, target, weight, ignore_index=ignore_index, reduction=reduction)
+
     if gamma == 0:
-        return F.cross_entropy(x, target, weight, ignore_index=ignore_index, reduction=reduction)
+        return ce_loss
 
     # log(P[class]) = log_softmax(score)[class]
     # logpt = F.log_softmax(x, dim=1)
 
     pt = F.softmax(x, dim=1)
-    pt.div_(1 - pt.transpose(0, 1).gather(0, target.unsqueeze(0)).transpose(0, 1))
+    pt = pt / (1 - pt.transpose(0, 1).gather(0, target.unsqueeze(0)).transpose(0, 1))
 
     loss = - 1 / (x.shape[1] - 1) * pt * torch.log(pt)
 
@@ -309,8 +311,9 @@ def complement_cross_entropy(
         loss[:, class_idx][target == class_idx] = 0.
 
     # Ignore index (set loss contribution to 0)
+    valid_idxs = torch.ones(loss.shape[1], dtype=torch.bool, device=x.device)
     if ignore_index >= 0 and ignore_index < x.shape[1]:
-        loss[:, ignore_index] = 0.
+        valid_idxs[ignore_index] = False
 
     # Weight
     if weight is not None:
@@ -321,14 +324,14 @@ def complement_cross_entropy(
 
     # Loss reduction
     if reduction == 'sum':
-        loss = loss.sum()
+        loss = loss[:, valid_idxs].sum()
     else:
-        loss = loss.sum(dim=1)
+        loss = loss[:, valid_idxs].sum(dim=1)
         if reduction == 'mean':
             loss = loss.mean()
 
     # Smooth the labels
-    return F.cross_entropy(x, target, weight, ignore_index=ignore_index, reduction=reduction) + gamma * loss
+    return ce_loss + gamma * loss
 
 
 def mutual_channel_loss(

holocron/optim/__init__.py

@@ -6,4 +6,3 @@
 from .adabelief import AdaBelief
 from .adamp import AdamP
 from . import wrapper
-from . import lr_scheduler