Skip to content

JpegEncoderTests are not reliable #1550

Open
@antonfirsov

Description

@antonfirsov

Prerequisites

  • I have written a descriptive issue title
  • I have verified that I am running the latest version of ImageSharp
  • I have verified if the problem exist in both DEBUG and RELEASE mode
  • I have searched open and closed issues to ensure it has not already been reported

Description

Our current validation logic in JpegEncoderTests is too tolerant, which caused #1549 to go through the quality gate completely unnoticed. The method for calculating the tolerance percentage is result of trial-and error experiments (dealing with platform differences if memory serves well):

/// <summary>
/// Anton's SUPER-SCIENTIFIC tolerance threshold calculation
/// </summary>
private static ImageComparer GetComparer(int quality, JpegSubsample subsample)
{
float tolerance = 0.015f; // ~1.5%
if (quality < 50)
{
tolerance *= 10f;
}
else if (quality < 75 || subsample == JpegSubsample.Ratio420)
{
tolerance *= 5f;
if (subsample == JpegSubsample.Ratio420)
{
tolerance *= 2f;
}
}
return ImageComparer.Tolerant(tolerance);
}

The method above returns a comparer of 15% tolerance, for subsample=420, quality=100, which doesn't really make sense. The image in #1549 (comment) has ~4% difference compared to the image before encoding.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions