Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Visual Saliency-induced Index (VSI) #119

Merged
merged 9 commits into from
Jul 6, 2020
Merged

Add Visual Saliency-induced Index (VSI) #119

merged 9 commits into from
Jul 6, 2020

Conversation

denproc
Copy link
Collaborator

@denproc denproc commented Jun 30, 2020 •
edited
Loading

Closes #115

Proposed Change

  • Added VSI measure
  • Added VSI Loss function
  • Both support back propagation
  • Both support GPU
  • Tests to check functionality

Let's merge #116 first.

@denproc denproc added the feature New feature or request label Jun 30, 2020
@denproc denproc requested review from zakajd and snk4tr June 30, 2020 09:40
@codecov
Copy link

codecov bot commented Jul 1, 2020 •
edited
Loading

Codecov Report

Merging #119 into master will increase coverage by 1.04%.
The diff coverage is 97.97%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #119      +/-   ##
==========================================
+ Coverage   87.76%   88.81%   +1.04%     
==========================================
  Files          15       17       +2     
  Lines        1275     1421     +146     
==========================================
+ Hits         1119     1262     +143     
- Misses        156      159       +3
Flag Coverage Δ
#unittests 88.81% <97.97%> (+1.04%) ⬆️
Impacted Files Coverage Δ
piq/functional.py 94.33% <94.33%> (ø)
piq/brisque.py 100.00% <100.00%> (ø)
piq/vsi.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 451329b...584638d. Read the comment docs.

@zakajd
Copy link
Collaborator

zakajd commented Jul 1, 2020

Nice job!
Have you compared results from your implementation with MATLAB?
Just for test I run your code on 2 images from TID2013 dataset (they are attached in zip archive) and got score 0.9521

Code that I used

import torch
from skimage.io import imread
from piq import vsi

I01 = torch.tensor(imread('data/I01.BMP')).permute(2, 0, 1)
i1_01_5 = torch.tensor(imread('data/i01_01_5.bmp')).permute(2, 0, 1)
print(vsi(i1_01_5, I01, data_range=255.))

While computing VSI score for those 2 images in OCTAVE (I don't have MATLAB installed) produces score 0.96405.
I'm using octave-online, after loading VSI.m file and 2 images code to compute score is simple:

I01 = imread("I01.BMP")
i01_01_5 = imread("i01_01_5.bmp")
score = VSI(i01_01_5, I01)

Difference is quite significant. Can you comment on that?

test_images.zip

zakajd
zakajd requested changes Jul 1, 2020
View reviewed changes
piq/vsi.py Outdated Show resolved Hide resolved
piq/vsi.py Outdated Show resolved Hide resolved
piq/vsi.py Outdated Show resolved Hide resolved
piq/vsi.py Outdated Show resolved Hide resolved
piq/vsi.py Outdated Show resolved Hide resolved
piq/vsi.py Outdated Show resolved Hide resolved
piq/vsi.py Outdated Show resolved Hide resolved
zakajd
zakajd requested changes Jul 1, 2020
View reviewed changes
piq/vsi.py Outdated Show resolved Hide resolved
piq/vsi.py Outdated Show resolved Hide resolved
zakajd
zakajd reviewed Jul 1, 2020
View reviewed changes
piq/vsi.py Outdated Show resolved Hide resolved
snk4tr
snk4tr reviewed Jul 2, 2020
View reviewed changes
Copy link
Contributor

@snk4tr snk4tr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one @denproc 👍
Only little comments from me 😄

piq/vsi.py Outdated Show resolved Hide resolved
piq/vsi.py Outdated Show resolved Hide resolved
piq/vsi.py Outdated Show resolved Hide resolved
piq/vsi.py Outdated Show resolved Hide resolved
piq/vsi.py Outdated Show resolved Hide resolved
piq/vsi.py Outdated Show resolved Hide resolved
piq/vsi.py Outdated Show resolved Hide resolved
piq/vsi.py Outdated Show resolved Hide resolved
@denproc
feature(vsi): Changes proposed by @zakajd and @snk4tr
2d0afc3 
- added functional.py from #109
- added new colour spaces to convert images
- debugging
- updated tests
@denproc
assets(vsi): Images to run tests
7f26889
@denproc
docs(vsi): Small fix
0c589a0
snk4tr
snk4tr reviewed Jul 4, 2020
View reviewed changes
Copy link
Contributor

@snk4tr snk4tr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Can be merged unless @zakajd has some additional comments

tests/test_vsi.py
Comment on lines +37 to +43
def test_vsi_compare_with_matlab(device) -> None:
prediction = torch.tensor(np.array(Image.open('tests/assets/I01.BMP'))).permute(2, 0, 1)
target = torch.tensor(np.array(Image.open('tests/assets/i01_01_5.bmp'))).permute(2, 0, 1)
predicted_score = vsi(prediction, target, data_range=255, reduction='none')
target_score = torch.tensor([0.96405]).to(predicted_score)
assert torch.allclose(predicted_score, target_score), f'Expected result similar to MATLAB,' \
f'got diff{predicted_score - target_score}'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@sonarcloud
Copy link

sonarcloud bot commented Jul 4, 2020

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities (and Security Hotspot 0 Security Hotspots to review)
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

zakajd
zakajd approved these changes Jul 6, 2020
View reviewed changes
Copy link
Collaborator

@zakajd zakajd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me 👍
Merging

@zakajd zakajd merged commit 43f8c76 into master Jul 6, 2020
@zakajd zakajd deleted the feature/VSI branch July 6, 2020 07:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@snk4tr snk4tr snk4tr left review comments

@zakajd zakajd zakajd approved these changes

Assignees

@snk4tr snk4tr

Labels
feature New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Metric: VSI (Visual Saliency-induced Index)
3 participants
@denproc @zakajd @snk4tr