New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic metadata-only image parsing API #292

Merged
merged 27 commits into from Jan 19, 2018

Conversation

Projects
None yet
7 participants
@xakep139
Copy link
Contributor

xakep139 commented Aug 10, 2017

Prerequisites

  • I have written a descriptive pull-request title
  • I have verified that there are no overlapping pull-requests open
  • I have verified that I am following matches the existing coding patterns and practise as demonstrated in the repository. These follow strict Stylecop rules 馃懏.
  • I have provided test coverage for my change (where applicable)

Description

Implemented feature request #258

@codecov

This comment has been minimized.

Copy link

codecov bot commented Aug 10, 2017

Codecov Report

Merging #292 into master will decrease coverage by 0.02%.
The diff coverage is 69.49%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #292      +/-   ##
==========================================
- Coverage   80.66%   80.63%   -0.03%     
==========================================
  Files         512      513       +1     
  Lines       20141    20199      +58     
  Branches     2197     2206       +9     
==========================================
+ Hits        16246    16287      +41     
- Misses       3218     3228      +10     
- Partials      677      684       +7
Impacted Files Coverage 螖
...geSharp/Formats/Jpeg/PdfJsPort/PdfJsJpegDecoder.cs 83.33% <0%> (-16.67%) 猬囷笍
...geSharp/Formats/Jpeg/GolangPort/OrigJpegDecoder.cs 55.55% <0%> (-44.45%) 猬囷笍
src/ImageSharp/Formats/Png/PngDecoder.cs 100% <100%> (酶) 猬嗭笍
src/ImageSharp/Formats/Gif/GifDecoder.cs 100% <100%> (酶) 猬嗭笍
src/ImageSharp/Image/Image.FromStream.cs 57.57% <100%> (+2.73%) 猬嗭笍
src/ImageSharp/Formats/Jpeg/JpegDecoder.cs 100% <100%> (酶) 猬嗭笍
src/ImageSharp/Formats/Bmp/BmpDecoder.cs 100% <100%> (酶) 猬嗭笍
src/ImageSharp/Formats/PixelTypeInfo.cs 100% <100%> (酶)
src/ImageSharp/Image/Image.Decode.cs 75% <50%> (-2.78%) 猬囷笍
src/ImageSharp/Formats/Png/PngDecoderCore.cs 69.51% <57.69%> (-0.68%) 猬囷笍
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
螖 = absolute <relative> (impact), 酶 = not affected, ? = missing data
Powered by Codecov. Last update f49bd9b...d2c0338. Read the comment docs.

@@ -22,7 +22,7 @@ public class TestFileSystem : ImageSharp.IO.IFileSystem

public static TestFileSystem Global { get; } = new TestFileSystem();

public static void RegisterGloablTestFormat()
public static void RegisterGlobalTestFormat()

This comment has been minimized.

@xakep139

xakep139 Aug 10, 2017

Contributor

Fix typo

@@ -23,15 +23,15 @@ public class TestFormat : IConfigurationModule, IImageFormat
{
public static TestFormat GlobalTestFormat { get; } = new TestFormat();

public static void RegisterGloablTestFormat()
public static void RegisterGlobalTestFormat()

This comment has been minimized.

@xakep139

xakep139 Aug 10, 2017

Contributor

Fix typo

return this.testFormat.Sample<TPixel>();
}

public int DetectPixelSize(Configuration configuration, Stream stream)
{
throw new NotImplementedException();

This comment has been minimized.

@xakep139

xakep139 Aug 10, 2017

Contributor

Not implemented for test format

@@ -1238,7 +1254,7 @@ private void ProcessStartOfFrameMarker(int remaining)

if (h == 3 || v == 3)
{
throw new ImageFormatException("Lnsupported subsampling ratio");
throw new ImageFormatException("Unsupported subsampling ratio");

This comment has been minimized.

@xakep139

xakep139 Aug 11, 2017

Contributor

Fix typo


[Theory]
[InlineData(TestImages.Gif.Cheers, 8)]
[InlineData(TestImages.Gif.Giphy, 8)]

This comment has been minimized.

@xakep139

xakep139 Aug 11, 2017

Contributor

For these two GIFs Image.GetPixelFormatSize() returns 32 which is not correct

TestFileSystem.Global.AddFile(this.FilePath, this.DataStream);
}

[Fact]
public void LoadFromStream()
{
Image<Rgba32> img = Image.Load<Rgba32>(this.DataStream);

This comment has been minimized.

@xakep139

xakep139 Aug 11, 2017

Contributor

Indentation here

@antonfirsov

This comment has been minimized.

Copy link
Member

antonfirsov commented Aug 13, 2017

@xakep139 nice job!
I wonder if instead of having a dedicated DetectPixelSize() method, wouldn't it be better to introduce and return a PixelTypeInfo object instead? We could bridge it to our PixelFormats later.

@xakep139

This comment has been minimized.

Copy link
Contributor

xakep139 commented Aug 18, 2017

@antonfirsov I apologize for delayed response. Your idea is pretty good, I'll implement it.
Initially I thought about detecting TPixel, but some bpp's couldn't be converted into it.

xakep139 added some commits Aug 18, 2017

@xakep139

This comment has been minimized.

Copy link
Contributor

xakep139 commented Aug 24, 2017

@antonfirsov do you have other suggestions?

@@ -25,6 +25,9 @@ public static class Png
public const string Powerpoint = "Png/pp.png";
public const string SplashInterlaced = "Png/splash-interlaced.png";
public const string Interlaced = "Png/interlaced.png";
public const string Palette8Bpp = "Png/Palette-8bpp.png";

This comment has been minimized.

@antonfirsov

antonfirsov Aug 24, 2017

Member

Unix systems are case-sensitive, so you have to rename this to palette-8pp.png, to match the filename. There is a test failure on travis because of that.

This comment has been minimized.

@xakep139

xakep139 Aug 24, 2017

Contributor

fixed in d6a2d40

@antonfirsov

This comment has been minimized.

Copy link
Member

antonfirsov commented Aug 24, 2017

@xakep139 no, the PixelTypeInfo thing is the only one, but we are releasing our first beta soon, which means that we need to be very strict defining our public API. I think we need to implement this with the PixelTypeInfo stuff now, because there is no chance to change it later.

@JimBobSquarePants any suggestions to manage this?

xakep139 added some commits Aug 24, 2017

@CLAassistant

This comment has been minimized.

Copy link

CLAassistant commented Sep 26, 2017

CLA assistant check
All committers have signed the CLA.

@xakep139

This comment has been minimized.

Copy link
Contributor

xakep139 commented Sep 26, 2017

@antonfirsov I've synced the branch with upstream

@antonfirsov

This comment has been minimized.

Copy link
Member

antonfirsov commented Sep 26, 2017

@xakep139 thanks! (And also thanks for your patience!)

Someone has to refactor your work to the PixelTypeInfo style so we can merge it. I'll try to be that someone ...
Can I push right to your branch?

@xakep139

This comment has been minimized.

Copy link
Contributor

xakep139 commented Sep 27, 2017

@antonfirsov, yes, sure

@antonfirsov antonfirsov referenced this pull request Jan 11, 2018

Closed

interfaces for Image #430

4 of 4 tasks complete
@antonfirsov

This comment has been minimized.

Copy link
Member

antonfirsov commented Jan 14, 2018

@xakep139 Sorry for the late response! I gonna deal with this during my ImageSharp hackaton in Feburary. It could be related to #430, because we should probably use the same PixelFormatInfo class on both API-s.

I would probably move the method DetectPixelType() into another interface to reduce API pollution, because not every IImageDecoder can implement it now:

internal interface IPixelTypeDetector
{
    PixelTypeInfo DetectPixelType(Configuration configuration, Stream stream)
}

public class PngDecoder : IImageDecoder, IPixelTypeDetector
{
   ...
}

@JimBobSquarePants I really think this is a valuable feature, and we should merge this. Thoughts about my ideas?

@JimBobSquarePants

This comment has been minimized.

Copy link
Member

JimBobSquarePants commented Jan 14, 2018

@xakep139 @antonfirsov Refresh my memory. What information are we seeking from this API and what is it's use case?

@xakep139

This comment has been minimized.

Copy link
Contributor

xakep139 commented Jan 15, 2018

In our system we should check bit depth of uploaded images, it's technical requirement of another system

@xakep139

This comment has been minimized.

Copy link
Contributor

xakep139 commented Jan 15, 2018

In general it could be usefull to get metadata without loading (decoding) the whole image in memory. E.g. to check image size (width, height), get pixel type (RGB, RGBA, etc.) and bit depth (or a pixel size).
For now we could only get image format with Image.DetectFormat() without loading.
Such feature (like magick identify in ImageMagick) could be very helpfull.

@JimBobSquarePants

This comment has been minimized.

Copy link
Member

JimBobSquarePants commented Jan 15, 2018

Thanks @xakep139
Ok that makes sense, as long as we are not trying to use it to automatically determine the PixelFormat to use.

@antonfirsov Would we then use reflection to determine which decoders support what we need?

@tocsoft

This comment has been minimized.

Copy link
Member

tocsoft commented Jan 15, 2018

we would register them the same we we do with the encoders/decoders i.e. we would add Configuration.SetPixelTypeInfoProvider(ImageFormat format, IPixelTypeDetector detectorInstance)

@denisivan0v denisivan0v referenced this pull request Jan 16, 2018

Merged

PixelDataPool<T>: reduce maximum pooled array size #436

4 of 4 tasks complete
@antonfirsov

This comment has been minimized.

Copy link
Member

antonfirsov commented Jan 16, 2018

Actually, I wanted to keep things as dumb as:

public static class Image
{
    ...

    public static bool TryDetectPixelType(Configuration config, Stream stream, out PixelTypeInfo pixelTypeInfo)
    {
        IImageDecoder decoder = DiscoverDecoder(stream, Configuration config);
        IPixelTypeDetector detector = decoder as IPixelTypeDetector;
        if (detector == null) return false;
        pixelTypeInfo = detector.DetectPixelType(config, stream);
        return true;
    }

    ...
}

I'm afraid that going any further, and exposing IPixelTypeDetector publicly + registering it into Configuration really means starting the design of a public metadata API, which I think is outside the scope of this PR.

@JimBobSquarePants

This comment has been minimized.

Copy link
Member

JimBobSquarePants commented Jan 17, 2018

Yeah let's leave it simple there and try not to expose another public API yet.

xakep139 and others added some commits Jan 17, 2018

Added an API to read base image information without decoding it
- intoduced base IImage interface
- introduced IImageInfoDetector interface
- Image.DetectPixelType method expanded to Identify method that returns IImage
@denisivan0v

This comment has been minimized.

Copy link
Contributor

denisivan0v commented Jan 17, 2018

@antonfirsov @JimBobSquarePants Me and @xakep139 are working on the same project and now we're extremely need an API that would provide base image information without decoding it.
So, I've made changes that was mentioned in #292 (comment):

Hope it will also help to resolve #430.

I'd be much appreciated for any suggestions and comments.

@antonfirsov

This comment has been minimized.

Copy link
Member

antonfirsov commented Jan 17, 2018

@denisivan0v thanks for joining in!

I really hope we'll be able to figure out how to provide the feature you need. But let's start with a very important disclaimer:
For the ImageSharp team, maintaining a consistent & clean API has a higher priority than individual features! So expect us to be very strict at the review process.

The major issue is that ImageSharp has not been designed to support metadata-only scenarios. I understand however that many users need it, and I think it's worth for us to find a way for supporting them, so they don't have to look for other libraries. (@JimBobSquarePants do you agree?)

However, to avoid the design lavine I foresee here, this feature set should be limited and/or isolated at this point, with future-proof API modifications only.

/// <summary>
/// Gets information about pixel.
/// </summary>
PixelTypeInfo PixelType { get; }

This comment has been minimized.

@antonfirsov

antonfirsov Jan 17, 2018

Member

I suppose this value holds the native pixel type in the stream, after Identify().
What's the deal with Image.Load<TPixel>()? What if typeof(TPixel)the native pixel type in the stream ?

This comment has been minimized.

@JimBobSquarePants

JimBobSquarePants Jan 17, 2018

Member

That situation will happen more often than not since 3 out of 4 of our currently supported image formats are commonly RGB.

If it's going to be part of the interface then both The property and its type should be renamed as it's just confusing now. What would the equivalent information be called in other libraries?

This comment has been minimized.

@antonfirsov

antonfirsov Jan 17, 2018

Member

However, if we rename it to something like OriginalPixelType, while keeping it in IImage means that we added another source stream specific, metadata-like property to the Image<T> class, which is mostly intended to represent raw image/pixel data.

My suggestion:
Let's name it OriginalPixelType and move it into the ImageMetaData class.

This comment has been minimized.

@tocsoft

tocsoft Jan 17, 2018

Member

Image shouldn't have any knowledge of its original loaded data.. this is why we removed the original image format from Image I think we need this API to return an ImageMetadata object instead that exposes all this info and not the based IImage interface.

This comment has been minimized.

@antonfirsov

antonfirsov Jan 17, 2018

Member

Okkay, let's sum up the situation, so we can figure it out:

  • @tocsoft Not sure if I'm comfortable with this fact, but we already have MetaData exposed on Image<TPixel>.
  • We need to include Width+Height (or maybe SixLabors.Size?) into the metadata-only result. Currently it's not exposed on ImageMetadata.
    • @denisivan0v solved this by defining ImageInfo : IImage as the metadata-only result
    • But maybe these are just different things, and sharing a common interface is an LSP violation.

@denisivan0v @xakep139 do you actually need the ImageInfo/Image<T> polymorphism? Eg: do you want to mix them in heterogenous containers? For me it feels like an unrealistic scenario.

This comment has been minimized.

@antonfirsov

antonfirsov Jan 17, 2018

Member

@JimBobSquarePants I'm not sure if I'm following you correctly.
Do you suggest to keep the PixelType property on the IImage interface + keep it in sync with TPixel in case of Image<TPixel>? Or the opposite: always exposing the raw type after Image.Load<TPixel>() regardless of TPixel?

This comment has been minimized.

@JimBobSquarePants

JimBobSquarePants Jan 17, 2018

Member

@antonfirsov The former.

For Image<TPixel> the PixelType property represents TPixel

For ImageInfo the PixelType property represents bits per pixel of the source image.

Keeping it in sync for Image<TPixel> is simple since it can be calculated in the constructor.

This comment has been minimized.

@antonfirsov

antonfirsov Jan 18, 2018

Member

Makes sense, and looks like being in sync with my vision about the role of PixelTypeInfo as a bridge between TPixel and raw formats, and future pixel-type agnostic Image API-s. (System.Drawing is decoding images into their native formats, we might implement this feature with IImage.)

However I'm still unsure if the result of Identify() is actually an image. Maybe we should rename to IImageInfo, and reserve IImage for future use cases:
IImage should represent an image that actually has pixels, so it could be used with processors, .Mutate(), .Clone() etc.

I think we should implement the following hierarchy in long term:

// Has Dimensions + PixelTypeInfo + Metadata
public interface IImageInfo { ... }

// Also has pixels, could be used with ImageProcessor-s.
public interface IImage : IImageInfo { ... }

internal class ImageInfo : IImageInfo { ... }

public class Image<TPixel> : IImage { ... }

This comment has been minimized.

@JimBobSquarePants

JimBobSquarePants Jan 18, 2018

Member

Yeah, let's do the rename. 馃憤

This comment has been minimized.

@denisivan0v

denisivan0v Jan 19, 2018

Contributor

Great design review, thanks!

I also wasn't fully satisfied that Image<T> and internal ImageInfo shares the same IImage interface. But IImageInfo and suggested hierarchy solved the problem and made things clear.

Done in 61c9caf.

@JimBobSquarePants
Copy link
Member

JimBobSquarePants left a comment

Overall this is very good! 馃

Just some minor changes and I'm happy for it to go in. Thanks for persisting with us! 馃槃

/// <summary>
/// Gets information about pixel.
/// </summary>
PixelTypeInfo PixelType { get; }

This comment has been minimized.

@JimBobSquarePants

JimBobSquarePants Jan 17, 2018

Member

That situation will happen more often than not since 3 out of 4 of our currently supported image formats are commonly RGB.

If it's going to be part of the interface then both The property and its type should be renamed as it's just confusing now. What would the equivalent information be called in other libraries?

public interface IImage
{
/// <summary>
/// Gets information about pixel.

This comment has been minimized.

@JimBobSquarePants

JimBobSquarePants Jan 17, 2018

Member

Lets flesh out the property descriptors here. This doesn't tell me anything and wont help with docs.

This comment has been minimized.

@denisivan0v

denisivan0v Jan 19, 2018

Contributor

Done in 61c9caf.

@@ -242,6 +295,7 @@ private void ReadLogicalScreenDescriptor()
{
Width = BitConverter.ToInt16(this.buffer, 0),
Height = BitConverter.ToInt16(this.buffer, 2),
BitsPerPixel = (this.buffer[4] & 0x07) + 1, // The lowest 3 bits represent the bit depth minus 1

This comment has been minimized.

@JimBobSquarePants

JimBobSquarePants Jan 17, 2018

Member

Was hoping this wouldn't trip you up. Good research. 馃憤

This comment has been minimized.

@denisivan0v

denisivan0v Jan 19, 2018

Contributor

Actually, this was initially done by @xakep139 馃憤

This comment has been minimized.

@xakep139
@@ -327,7 +385,7 @@ private void ReadFrame()
indices = Buffer<byte>.CreateClean(imageDescriptor.Width * imageDescriptor.Height);

this.ReadFrameIndices(imageDescriptor, indices);
this.ReadFrameColors(indices, localColorTable ?? this.globalColorTable, imageDescriptor);
this.ReadFrameColors(ref image, ref previousFrame, indices, localColorTable ?? this.globalColorTable, imageDescriptor);

This comment has been minimized.

@JimBobSquarePants

JimBobSquarePants Jan 17, 2018

Member

Image<T> and ImageFrame<T> are already reference types?

This comment has been minimized.

@denisivan0v

denisivan0v Jan 19, 2018

Contributor

I had to remove TPixel type parameter from this class and move it Decode method. So Image<TPixel> image and ImageFrame<TPixel> previousFrame fields also removed and became local variables in Decode<TPixel> method. They marked with ref because they are reassigning down the call stack.


namespace SixLabors.ImageSharp
{
internal sealed class ImageInfo : IImage

This comment has been minimized.

@JimBobSquarePants

JimBobSquarePants Jan 17, 2018

Member

Please add xml docs

This comment has been minimized.

@denisivan0v

denisivan0v Jan 19, 2018

Contributor

Done in 61c9caf

namespace SixLabors.ImageSharp.Formats
{
/// <summary>
/// Stores information about pixel.

This comment has been minimized.

@JimBobSquarePants

JimBobSquarePants Jan 17, 2018

Member

We need a better description than this.

I don't like the name. RawPixelTypeInfo or RawPixelFormatInfo perhaps? Naming is hard!

Whatever we choose the property name on the interface should match.

This comment has been minimized.

@antonfirsov

antonfirsov Jan 17, 2018

Member

@JimBobSquarePants I proposed this class and naming in order to make it extendable in a way like this:

public class PixelTypeInfo
{
    public System.Type PixelType { get; }
    public int BitsPerPixel { get; }

    internal PixelTypeInfo(int bitsPerPixel, System.Type pixelType = null)
    {
        this.BitsPerPixel = bitsPerPixel;
    }

    public static PixelTypeInfo Create<TPixel>() where TPixel 
        :  IPixel<TPixel> => 
        new PixelTypeInfo(Unsafe.SizeOf<TPixel>(), typeof(TPixel));
}

This will enable integrating it deeper with our pixel type infrastructure, and provide non-generic pixel-type related API-s. As a bridge between native (image format/stream bound) pixel type data and our internal pixel types, this class will not necessarily represent "raw"/native pixel type information only.

This comment has been minimized.

@JimBobSquarePants

JimBobSquarePants Jan 17, 2018

Member

You'd need to keep it in sync with changes to TPixel in that case and the PR utilizes it purely for the raw format. It can be one or the other, not both.

This comment has been minimized.

@denisivan0v
namespace SixLabors.ImageSharp.Tests.TestUtilities.ReferenceCodecs
{
public class SystemDrawingReferenceDecoder : IImageDecoder
using SixLabors.ImageSharp.MetaData;

This comment has been minimized.

@JimBobSquarePants

JimBobSquarePants Jan 17, 2018

Member

Why were the namespaces altered in this class?

This comment has been minimized.

@denisivan0v

denisivan0v Jan 19, 2018

Contributor

That wasn't my intension. I guess it's because of my incorrect tooling settings (I'm using JetBrains Rider on Mac). Reverted it in 61c9caf.

@antonfirsov antonfirsov changed the title Add an equivalent for System.Drawing.Image.GetPixelFormatSize() (#258) Basic metadata-only image parsing API Jan 18, 2018

@denisivan0v

This comment has been minimized.

Copy link
Contributor

denisivan0v commented Jan 18, 2018

@antonfirsov @JimBobSquarePants huge thanks for your review and suggestions! I'm going to make changes asap to reduce time to the next review cycle.

@@ -17,6 +17,7 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="CoreCompat.System.Drawing" Version="1.0.0-beta006" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.5.0" />

This comment has been minimized.

@JimBobSquarePants

JimBobSquarePants Jan 19, 2018

Member

Do we need this?

This comment has been minimized.

@denisivan0v

denisivan0v Jan 19, 2018

Contributor

This enables unit tests execution in JetBrains Rider, but it's not necessary.

This comment has been minimized.

@JimBobSquarePants

JimBobSquarePants Jan 19, 2018

Member

Let's keep it then. I want everyone to be able to test the project in their desired IDE.

@JimBobSquarePants JimBobSquarePants merged commit 61c9caf into SixLabors:master Jan 19, 2018

3 of 5 checks passed

codecov/patch 79.64% of diff hit (target 80.72%)
Details
codecov/project 80.7% (-0.02%) compared to 814d4ac
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
license/cla Contributor License Agreement is signed.
Details
@JimBobSquarePants

This comment has been minimized.

Copy link
Member

JimBobSquarePants commented Jan 19, 2018

Many, many thanks @xakep139 and @denisivan0v ! 馃

This is an incredibly valuable addition to the library and we're so happy you stuck with us to get it merged in!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment