Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Derive quantizer index offsets for segments from distortion scales #2694

Merged
merged 2 commits into from
Aug 21, 2022

Conversation

barrbrain
Copy link
Collaborator

@barrbrain barrbrain commented Mar 14, 2021

Runs k-means on logarithms of distortion scale, with k chosen by variance in spacing of the result.
Now with more comments and AWCY results.

@coveralls
Copy link
Collaborator

coveralls commented Mar 14, 2021

Coverage Status

Coverage decreased (-0.4%) to 83.547% when pulling a3df721 on barrbrain:segment-opt into b0cb667 on xiph:master.

src/rate.rs Outdated
@@ -229,7 +229,7 @@ pub(crate) fn bexp64(logq57: i64) -> i64 {
// input: a 64-bit integer in Q0 (no fraction).
// output: a 64-bit log in Q57.
// TODO: Mark const once we can use local variables in a const function.
fn blog64(w: i64) -> i64 {
pub fn blog64(w: i64) -> i64 {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note to self, once it lands refactor the maths function from rate.rs to a stand alone file.

src/segmentation.rs Outdated Show resolved Hide resolved
@barrbrain barrbrain force-pushed the segment-opt branch 4 times, most recently from 7568f6a to cb290cb Compare March 17, 2021 09:23
@barrbrain
Copy link
Collaborator Author

AWCY results at the default speed indicate more work is needed from this initial version:

PSNR Y PSNR Cb PSNR Cr CIEDE2000 SSIM MS-SSIM PSNR-HVS Y PSNR-HVS Cb PSNR-HVS Cr PSNR-HVS VMAF VMAF-NEG
1.6103 1.8378 1.7304 1.8383 1.5591 1.7372 1.8687 1.8090 1.4279 1.8676 2.1277 2.1061

@codecov-commenter
Copy link

codecov-commenter commented Aug 15, 2022

Codecov Report

Base: 86.44% // Head: 86.44% // Decreases project coverage by -0.00% ⚠️

Coverage data is based on head (74976b9) compared to base (c0b1f56).
Patch coverage: 99.26% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2694      +/-   ##
==========================================
- Coverage   86.44%   86.44%   -0.01%     
==========================================
  Files          87       88       +1     
  Lines       33942    33957      +15     
==========================================
+ Hits        29342    29354      +12     
- Misses       4600     4603       +3     
Impacted Files Coverage Δ
src/util/kmeans.rs 98.85% <98.85%> (ø)
src/segmentation.rs 97.90% <100.00%> (+2.03%) ⬆️
src/lib.rs 41.93% <0.00%> (-4.50%) ⬇️
src/asm/x86/lrf.rs 92.22% <0.00%> (-1.90%) ⬇️
src/me.rs 95.20% <0.00%> (-0.40%) ⬇️
src/predict.rs 81.47% <0.00%> (-0.10%) ⬇️
src/cpu_features/x86.rs 39.58% <0.00%> (ø)
src/encoder.rs 87.33% <0.00%> (+0.03%) ⬆️
src/rdo.rs 85.92% <0.00%> (+0.09%) ⬆️
... and 3 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@barrbrain barrbrain force-pushed the segment-opt branch 3 times, most recently from fd93a89 to c72d409 Compare August 16, 2022 14:10
@shssoichiro
Copy link
Collaborator

Wow, this seems much cleaner. Nice!

@barrbrain barrbrain force-pushed the segment-opt branch 2 times, most recently from 11d2ddd to 85ebf73 Compare August 17, 2022 05:15
@barrbrain
Copy link
Collaborator Author

barrbrain commented Aug 17, 2022

Update: Rebased on top of recent changes.
AWCY results with number of segments fixed at 3 and dynamic offsets on objective-fast-1 at default speed:

PSNR Y PSNR Cb PSNR Cr CIEDE2000 SSIM MS-SSIM PSNR-HVS Y PSNR-HVS Cb PSNR-HVS Cr PSNR-HVS VMAF VMAF-NEG
-0.4592 -0.2004 -0.4580 -0.3037 -0.3615 -0.2910 -0.4767 0.0709 -0.3100 -0.4787 -0.5725 -0.5436

@barrbrain barrbrain force-pushed the segment-opt branch 3 times, most recently from 03d1bc3 to 0784825 Compare August 19, 2022 18:26
@shssoichiro
Copy link
Collaborator

shssoichiro commented Aug 19, 2022

I noticed the testing between different numbers of segments--the reasoning for the current version using a dynamic number of segments based on the variance of scores within the frame is because there is some amount of filesize overhead in using a higher number of segments (because they have to be coded). Therefore, we wanted to avoid coding segments that were not useful.

If it's not possible to dynamically choose the segment count based on score variance, maybe a workaround could be to use 5 segments with low-to-mid quantizers, where the segmentation coding overhead is less significant, and 3 segments with higher quantizers where the segmentation coding would affect the overall filesize more.

@barrbrain
Copy link
Collaborator Author

The goal is to retain a dynamic number of segments. The current testing is to measure the performance of an 'oracle' that picks the best number of segments for each clip.

@barrbrain barrbrain force-pushed the segment-opt branch 2 times, most recently from 5c4ec67 to a752a33 Compare August 20, 2022 00:04
@barrbrain barrbrain force-pushed the segment-opt branch 5 times, most recently from b276af9 to e6c596e Compare August 21, 2022 07:09
@barrbrain barrbrain marked this pull request as ready for review August 21, 2022 08:15
Choose the value of k with lowest variance in spacing between the
logs of the selected quantizers.

AWCY results on objective-1-fast at the default speed:

 PSNR Y | PSNR Cb | PSNR Cr | CIEDE2000 |    SSIM | MS-SSIM
-0.3646 | -0.0796 | -0.3117 |   -0.1695 | -0.2051 | -0.1797

PSNR-HVS Y | PSNR-HVS Cb | PSNR-HVS Cr | PSNR-HVS |    VMAF | VMAF-NEG
   -0.3030 |      0.2454 |     -0.2721 |  -0.3020 | -0.4099 |  -0.3882
@barrbrain
Copy link
Collaborator Author

AWCY results for objective-1-fast at default speed

PSNR Y PSNR Cb PSNR Cr CIEDE2000 SSIM MS-SSIM PSNR-HVS Y PSNR-HVS Cb PSNR-HVS Cr PSNR-HVS VMAF VMAF-NEG
-0.4139 -0.1869 -0.2631 -0.2345 -0.2842 -0.2500 -0.3600 0.1368 -0.2013 -0.3594 -0.3614 -0.4027

@barrbrain barrbrain merged commit 27997eb into xiph:master Aug 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants