-
Notifications
You must be signed in to change notification settings - Fork 18
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
7ebd950
commit d670de1
Showing
1 changed file
with
41 additions
and
73 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,136 +1,104 @@ | ||
| <p align="center"><img src="https://raw.githubusercontent.com/francois-rozet/piqa/master/banner.svg" width="80%"></p> | ||
|
|
||
| > PIQA is not endorsed by Facebook, Inc.; PyTorch, the PyTorch logo and any related marks are trademarks of Facebook, Inc. | ||
| <p align="center"><img src="https://raw.githubusercontent.com/francois-rozet/piqa/master/docs/images/banner.svg" width="80%"></p> | ||
|
|
||
| # PyTorch Image Quality Assessment | ||
|
|
||
| The `piqa` package is a collection of measures and metrics for image quality assessment in various image processing tasks such as denoising, super-resolution, image interpolation, etc. It relies only on [PyTorch](https://github.com/pytorch/pytorch) and takes advantage of its efficiency and automatic differentiation. | ||
|
|
||
| PIQA is directly inspired from the [`piq`](https://github.com/photosynthesis-team/piq) project, but focuses on the conciseness, readability and understandability of its (sub-)modules, such that anyone can easily reuse and/or adapt them to its needs. | ||
|
|
||
| However, conciseness should never be at the expense of efficiency; PIQA's implementations are up to 3 times faster than those of other IQA PyTorch packages like [`kornia`](https://github.com/kornia/kornia), [`piq`](https://github.com/photosynthesis-team/piq) and [`IQA-pytorch`](https://github.com/dingkeyan93/IQA-optimization). | ||
| PIQA is a collection of PyTorch metrics for image quality assessment in various image processing tasks such as generation, denoising, super-resolution, interpolation, etc. It focuses on the efficiency, conciseness and understandability of its (sub-)modules, such that anyone can easily reuse and/or adapt them to its needs. | ||
|
|
||
| > PIQA should be pronounced *pika* (like Pikachu ⚡️) | ||
| ## Installation | ||
|
|
||
| The `piqa` package is available on [PyPI](https://pypi.org/project/piqa/), which means it is installable with `pip`: | ||
| The `piqa` package is available on [PyPI](https://pypi.org/project/piqa), which means it is installable via `pip`. | ||
|
|
||
| ```bash | ||
| ``` | ||
| pip install piqa | ||
| ``` | ||
|
|
||
| Alternatively, if you need the latest features, you can install it using | ||
| Alternatively, if you need the latest features, you can install it from the repository. | ||
|
|
||
| ```bash | ||
| pip install git+https://github.com/francois-rozet/piqa | ||
| ``` | ||
|
|
||
| or copy the package directly to your project, with | ||
|
|
||
| ```bash | ||
| git clone https://github.com/francois-rozet/piqa | ||
| cp -R piqa/piqa <path/to/project>/piqa | ||
| pip install git+https://github.com/francois-rozet/piqa | ||
| ``` | ||
|
|
||
| ## Getting started | ||
|
|
||
| In `piqa`, each metric is associated to a class, child of `torch.nn.Module`, which has to be instantiated to evaluate the metric. | ||
| In `piqa`, each metric is associated to a class, child of `torch.nn.Module`, which has to be instantiated to evaluate the metric. All metrics are differentiable and support CPU and GPU (CUDA). | ||
|
|
||
| ```python | ||
| import torch | ||
| import piqa | ||
|
|
||
| # PSNR | ||
| from piqa import PSNR | ||
|
|
||
| x = torch.rand(5, 3, 256, 256) | ||
| y = torch.rand(5, 3, 256, 256) | ||
|
|
||
| psnr = PSNR() | ||
| psnr = piqa.PSNR() | ||
| l = psnr(x, y) | ||
|
|
||
| # SSIM | ||
| from piqa import SSIM | ||
|
|
||
| x = torch.rand(5, 3, 256, 256, requires_grad=True).cuda() | ||
| y = torch.rand(5, 3, 256, 256).cuda() | ||
|
|
||
| ssim = SSIM().cuda() | ||
| ssim = piqa.SSIM().cuda() | ||
| l = 1 - ssim(x, y) | ||
| l.backward() | ||
| ``` | ||
|
|
||
| Like `torch.nn` built-in components, these classes are based on functional definitions of the metrics, which are less user-friendly, but more versatile. | ||
|
|
||
| ```python | ||
| import torch | ||
|
|
||
| from piqa.ssim import ssim | ||
| from piqa.utils.functional import gaussian_kernel | ||
|
|
||
| x = torch.rand(5, 3, 256, 256) | ||
| y = torch.rand(5, 3, 256, 256) | ||
| kernel = gaussian_kernel(11, sigma=1.5).expand(3, 11, 11) | ||
|
|
||
| kernel = gaussian_kernel(11, sigma=1.5).repeat(3, 1, 1) | ||
|
|
||
| l = ssim(x, y, kernel=kernel, channel_avg=False) | ||
| l = 1 - ssim(x, y, kernel=kernel) | ||
| ``` | ||
|
|
||
| For more information about PIQA's features, check out the documentation at [francois-rozet.github.io/piqa/](https://francois-rozet.github.io/piqa/). | ||
|
|
||
| ### Metrics | ||
| For more information, check out the documentation at [piqa.readthedocs.io](https://piqa.readthedocs.io). | ||
|
|
||
| | Acronym | Class | Range | Objective | Year | Metric | | ||
| |:-------:|:---------:|:--------:|:---------:|:----:|------------------------------------------------------------------------------------------------------| | ||
| | TV | `TV` | `[0, ∞]` | / | 1937 | [Total Variation](https://en.wikipedia.org/wiki/Total_variation) | | ||
| | PSNR | `PSNR` | `[0, ∞]` | max | / | [Peak Signal-to-Noise Ratio](https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio) | | ||
| | SSIM | `SSIM` | `[0, 1]` | max | 2004 | [Structural Similarity](https://en.wikipedia.org/wiki/Structural_similarity) | | ||
| | MS-SSIM | `MS_SSIM` | `[0, 1]` | max | 2004 | [Multi-Scale Structural Similarity](https://ieeexplore.ieee.org/document/1292216/) | | ||
| | LPIPS | `LPIPS` | `[0, ∞]` | min | 2018 | [Learned Perceptual Image Patch Similarity](https://arxiv.org/abs/1801.03924) | | ||
| | GMSD | `GMSD` | `[0, ∞]` | min | 2013 | [Gradient Magnitude Similarity Deviation](https://arxiv.org/abs/1308.3052) | | ||
| | MS-GMSD | `MS_GMSD` | `[0, ∞]` | min | 2017 | [Multi-Scale Gradient Magnitude Similarity Deviation](https://ieeexplore.ieee.org/document/7952357) | | ||
| | MDSI | `MDSI` | `[0, ∞]` | min | 2016 | [Mean Deviation Similarity Index](https://arxiv.org/abs/1608.07433) | | ||
| | HaarPSI | `HaarPSI` | `[0, 1]` | max | 2018 | [Haar Perceptual Similarity Index](https://arxiv.org/abs/1607.06140) | | ||
| | VSI | `VSI` | `[0, 1]` | max | 2014 | [Visual Saliency-based Index](https://ieeexplore.ieee.org/document/6873260) | | ||
| | FSIM | `FSIM` | `[0, 1]` | max | 2011 | [Feature Similarity](https://ieeexplore.ieee.org/document/5705575) | | ||
| ### Available metrics | ||
|
|
||
| ### JIT | ||
| | Class | Range | Objective | Year | Metric | | ||
| |:---------:|:------:|:---------:|:----:|------------------------------------------------------------------------------------------------------| | ||
| | `TV` | [0, ∞] | / | 1937 | [Total Variation](https://en.wikipedia.org/wiki/Total_variation) | | ||
| | `PSNR` | [0, ∞] | max | / | [Peak Signal-to-Noise Ratio](https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio) | | ||
| | `SSIM` | [0, 1] | max | 2004 | [Structural Similarity](https://en.wikipedia.org/wiki/Structural_similarity) | | ||
| | `MS_SSIM` | [0, 1] | max | 2004 | [Multi-Scale Structural Similarity](https://ieeexplore.ieee.org/document/1292216/) | | ||
| | `LPIPS` | [0, ∞] | min | 2018 | [Learned Perceptual Image Patch Similarity](https://arxiv.org/abs/1801.03924) | | ||
| | `GMSD` | [0, ∞] | min | 2013 | [Gradient Magnitude Similarity Deviation](https://arxiv.org/abs/1308.3052) | | ||
| | `MS_GMSD` | [0, ∞] | min | 2017 | [Multi-Scale Gradient Magnitude Similarity Deviation](https://ieeexplore.ieee.org/document/7952357) | | ||
| | `MDSI` | [0, ∞] | min | 2016 | [Mean Deviation Similarity Index](https://arxiv.org/abs/1608.07433) | | ||
| | `HaarPSI` | [0, 1] | max | 2018 | [Haar Perceptual Similarity Index](https://arxiv.org/abs/1607.06140) | | ||
| | `VSI` | [0, 1] | max | 2014 | [Visual Saliency-based Index](https://ieeexplore.ieee.org/document/6873260) | | ||
| | `FSIM` | [0, 1] | max | 2011 | [Feature Similarity](https://ieeexplore.ieee.org/document/5705575) | | ||
| | `FID` | [0, ∞] | min | 2017 | [Fréchet Inception Distance](https://arxiv.org/abs/1706.08500) | | ||
|
|
||
| Most functional components of `piqa` support PyTorch's JIT, *i.e.* [TorchScript](https://pytorch.org/docs/stable/jit.html), which is a way to create serializable and optimizable functions from PyTorch code. | ||
| ### Tracing | ||
|
|
||
| By default, jitting is disabled for those components. To enable it, the `PIQA_JIT` environment variable has to be set to `1`. To do so temporarily, | ||
| All metrics of `piqa` support [PyTorch's tracing](https://pytorch.org/docs/stable/generated/torch.jit.trace.html), which optimizes their execution, especially on GPU. | ||
|
|
||
| * UNIX-like `bash` | ||
|
|
||
| ```bash | ||
| export PIQA_JIT=1 | ||
| ``` | ||
|
|
||
| * Windows `cmd` | ||
|
|
||
| ```cmd | ||
| set PIQA_JIT=1 | ||
| ``` | ||
|
|
||
| * Microsoft `PowerShell` | ||
| ```python | ||
| ssim = piqa.SSIM().cuda() | ||
| ssim_traced = torch.jit.trace(ssim, (x, y)) | ||
|
|
||
| ```powershell | ||
| $env:PIQA_JIT=1 | ||
| l = 1 - ssim_traced(x, y) # should be faster ¯\_(ツ)_/¯ | ||
| ``` | ||
|
|
||
| ### Assert | ||
|
|
||
| PIQA uses type assertions to raise meaningful messages when an object-oriented component doesn't receive an input of the expected type. This feature eases a lot early prototyping and debugging, but it might hurt a little the performances. | ||
|
|
||
| If you need the absolute best performances, the assertions can be disabled with the Python flag [`-O`](https://docs.python.org/3/using/cmdline.html#cmdoption-o). For example, | ||
| PIQA uses type assertions to raise meaningful messages when a metric doesn't receive an input of the expected type. This feature eases a lot early prototyping and debugging, but it might hurt a little the performances. If you need the absolute best performances, the assertions can be disabled with the Python flag [`-O`](https://docs.python.org/3/using/cmdline.html#cmdoption-o). For example, | ||
|
|
||
| ```bash | ||
| ``` | ||
| python -O your_awesome_code_using_piqa.py | ||
| ``` | ||
|
|
||
| Alternatively, you can disable PIQA's type assertions within your code with | ||
|
|
||
| ```python | ||
| from piqa.utils import set_debug | ||
| set_debug(False) | ||
| piqa.utils.set_debug(False) | ||
| ``` | ||
|
|
||
| ## Contributing | ||
|
|
||
| If you have a question, an issue or would like to contribute, please read our [contributing guidelines](https://github.com/francois-rozet/piqa/blob/master/CONTRIBUTING.md). |