Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Full System.Drawing Functionality on .NET Core #21980

Closed
mellinoe opened this issue May 25, 2017 · 103 comments
Closed

Support Full System.Drawing Functionality on .NET Core #21980

mellinoe opened this issue May 25, 2017 · 103 comments

Comments

@mellinoe
Copy link

@mellinoe mellinoe commented May 25, 2017

Update: The following is the current plan.

  • Port the .NET Framework implementation to .NET Core. WIP, See dotnet/corefx#20593 .
  • Move mono's libgdiplus-based managed implementation from mono to corefx
  • Ensure that there is a code-sharing story between mono and corefx. This likely means removing the source code from mono and importing code from the submodule link that already exists. In progress. Mono has begun using some portions of the Windows implementation already.
  • Port mono's tests for System.Drawing, making any modifications necessary to ensure that they pass on the Windows implementation, which is our baseline for compatibility. In progress
  • Consolidate differences between the Windows and Unix versions of the library. Document all differences in behavior and mark test cases appropriately.

System.Drawing has been identified as one of the top most-used assemblies that are missing from .NET Core. Although there are 3rd-party alternatives that provide more modern versions of the same functionality (ImageSharp, SkiaSharp, etc.), we believe there is still value in providing a binary-compatible implementation of System.Drawing itself. This issue is intended to document and track the work items involved. There are still lots of open questions; feedback and suggestions are encouraged.

Priorities:

  • Simplify migration from .NET Framework to .NET Core. We need to be binary compatible (no recompilation needed for existing assets), and we need to match the behavior as closely as possible.
  • Support System.Drawing where mono does, and facilitate code sharing between our implementations. As part of this, we will be open-sourcing the original Windows implementation from .NET Framework. This should allow us to improve mono's implementation where it diverges, and hopefully reduce the maintenance burden between us by consolidating code and effort.
  • NON-GOALS: We do not view this library as an opportunity for innovation, nor will we invest significant amounts of effort into optimizing or refactoring it. High performance, rich OS-specific functionality, and a redesigned, modern API are not areas we are interested in investing in as part of this effort. There are alternative libraries which target all of those issues and more.

Plan

System.Drawing is, fundamentally, a very thin wrapper over the GDI+ component on Windows. Worse yet, it is not available for UWP applications, so support is limited strictly to Win32 applications. This makes cross-platform implementations difficult, and the largest chunk of work involved with this effort will be figuring out how we can support the library on all of our platforms, and maintaining compatibility between them. There are several existing implementations that we can leverage, but none will support all of our platforms by themselves. We may need several distinct backends in order to get full cross-platform support, although some options could avoid that. The below chart maps implementation approaches (rows) with platform applicability (columns):

Win32 UWP macOS Linux / Others
.NET Framework Version (GDI+) X X X
Direct2D/Win2D X X
Mono/libgdiplus X
Xamarin/CoreGraphics X X X
Managed

.NET Framework Version (GDI+)

This is the existing implementation from the .NET Framework on Windows. It is a very thin wrapper around GDI+.

  • ✔️ Baseline implementation. Well-tested, stable.
  • Can only be used in non-UWP Windows apps (Win32).

Direct2D/Win2D

In order to get UWP support, we could implement a compatibility layer over Direct2D, or Win2D.

  • ✔️ GPU acceleration
  • Only buys us UWP support over the existing .NET Framework implementation.
  • High implementation cost
  • Can be shared between Win32 and UWP applications (Windows 8+ only).

Mono/libgdiplus

This is the implementation used in mono. It uses a complicated native shim called "libgdiplus", which relies on a large set of native binaries.

  • ✔️ libgdiplus is available in a large number of package managers across the Unix ecosystem already. This means we can treat the dependency the same way we do the other native dependencies for .NET Core.
  • Native code adds a lot of complexity for new devs.

Xamarin/CoreGraphics

This is a specialized implementation used by Xamarin, in support of macOS and iOS. Unlike the "libgdiplus" mono version, it only uses CoreGraphics (a system component).

  • ✔️ No extra native dependencies; only uses system libraries.
  • macOS-specific. Could be used for iOS (not supported by .NET Core yet).

Managed (ImageSharp or custom)

One option is to implement the library in managed code. We can call into an existing library, like ImageSharp, for image-processing and graphics functionality. This is very desirable for a lot of reasons, but it does have a higher implementation cost.

  • ✔️ Can be used everywhere.
  • ✔️ Can achieve uniform behavior everywhere (at the expense of conformity with .NET Framework).
  • ✔️ Lower maintenance costs in the long-term.
  • High implementation cost.

Windows-Specific

One option is to treat the library as a Windows-specific component, similar to Registry. Attempting to install and load the library on other platforms would throw PlatformNotSupportedException's, as Registry does.

Tests

We do not have a very good existing test suite in the .NET Framework. The library is mainly covered indirectly, through other components like WinForms. Mono has a good set of conformance tests for their implementation, which we should be able to re-use. Where appropriate, we should also consider test cases which directly compare output from the .NET Framework implementation and other implementations (assuming we choose to support such things).

Location

An open question is where this library should live. If we take third-party dependencies to support some platforms, we may not want this code to live in corefx.

Printing Functionality

One thing skimmed over above is the fact that System.Drawing exposes a lot of functionality for printing images and documents. This poses a different problem from image-processing and graphics, which can be implemented portably. We may decide to exclude all printing-related portions of the library.

Future investments

Once the library is supported on all platforms, we will accept small, targeted improvements to the library, keeping in mind that legacy compatibility is the only important metric for the library. We will not want to merge any code that refactors, optimizes, or augments the Windows version until we have a fully-working cross-platform implementation. This matches our strategy with other compatibility libraries, like System.IO.SerialPort.

@marek-safar @JimBobSquarePants @ViktorHofer @qmfrederik @karelz

@qmfrederik
Copy link
Contributor

@qmfrederik qmfrederik commented May 25, 2017

@mellinoe Thanks for the additional information!

I actually have a good experience with using Mono/libgdiplus via CoreCompat.System.Drawing. Here's why:

  • Mono/libgdiplus actually does run on .NET Core on Windows. When on Windows, Mono's System.Drawing just uses GDI+ from Windows. So I think you can add a checkbox to 'Win32' for Mono's System.Drawing
  • I don't think the native dependencies need to be a problem. libgdiplus ships as a native component for most operating systems .NET Core supports. All Linux distros which .NET Core supports include libgdiplus in their mainline repositories: Ubuntu, Debian, Fedora, OpenSuse - even Alpine has it. Plus, libgidplus was recently added to Homebrew, too.
    So, I would suggest treating libgdiplus as a the single native dependency which would be acquired from the Linux distro or Homebrew - using the same acquisition mechanism for other .NET Core dependencies, such as curl or OpenSSL.
  • Mono's System.Drawing does include the Printing components, so you would get those for free, too.
  • Mono's System.Drawing runs natively on netstandard2.0 (and all unit tests pass!) and is very close to running on netstandard1.6.
  • Could you elaborate on "Native code adds a lot of complexity for new devs"?
  • Regarding "Behavior and performance issues", I'm not really aware of them, so it would be good to have them listed, too.
  • Since it's part of Mono, Mono's System.Drawing is still being maintained.

Another option would be to use the System.Drawing code from NetFX and have it consume libgdiplus instead of GDI+ on Linux. There's been some talk of open sourcing System.Drawing before, not sure if that's still an option? It would be fairly easy to implement, too.

Regarding UWP, do you currently have demand for System.Drawing on UWP?

/cc @akoeplinger

@mellinoe
Copy link
Author

@mellinoe mellinoe commented May 25, 2017

All Linux distros which .NET Core supports include libgdiplus in their mainline repositories: Ubuntu, Debian, Fedora, OpenSuse - even Alpine has it. Plus, libgidplus was recently added to Homebrew, too.

That is good to know; thanks for the info. If we are able to treat it as a fully external dependency, like the other native libs we use, then I would have less hesitation about using it. I'm not an expert on libgdiplus, so I would certainly like to understand more about it. The folks that I have talked with have been hesitant to readily suggest that we go ahead and use it, but it may be okay in the face of less-promising alternatives. In general, though, I think we should be really cautious about any new native dependencies we adopt in .NET Core. They make a lot of things more complicated for us and for customers.

Could you elaborate on "Native code adds a lot of complexity for new devs"?

I was trying to express the idea that folks trying to submit fixes for System.Drawing will have a harder time contributing to an external C library than a managed C# library here. Just a different skill-set, is all.

There's been some talk of open sourcing System.Drawing before, not sure if that's still an option? It would be fairly easy to implement, too.

At the very least, we will be using the .NET Framework version for Windows, so we will certainly open-source it. We can attempt to use that same implementation against libgdiplus on other platforms, but I fear that would involve reworking tons of the native code. Then again, that is just a guess on my part.

Regarding UWP, do you currently have demand for System.Drawing on UWP?

We view System.Drawing as one of the main libraries blocking old customers (and their code) from migrating to our new platforms, including UWP. That's not exactly the question you asked, but the end result is: we'd like to support System.Drawing everywhere, because it has proven to be an important dependency for existing libraries.

@qmfrederik
Copy link
Contributor

@qmfrederik qmfrederik commented May 26, 2017

At the very least, we will be using the .NET Framework version for Windows, so we will certainly open-source it. We can attempt to use that same implementation against libgdiplus on other platforms, but I fear that would involve reworking tons of the native code. Then again, that is just a guess on my part.

My guess is that the .NET Framework version will work well with libgdiplus. It's an assumption based on my experience with libgdiplus and how it tries to be binary compatible with GDI+.
The proof of the pudding is in the eating, though. If you can make the .NET Framework System.Drawing source code available under an open source license, I'd be happy to give it a spin, and run Mono's unit tests for System.Drawing on .NET Framework's System.Drawing backed by libgdiplus running on .NET Core on, say, Linux -- that would give the definitive answer.

I was trying to express the idea that folks trying to submit fixes for System.Drawing will have a harder time contributing to an external C library than a managed C# library here. Just a different skill-set, is all.

That's a fair point, although on the other hand the C libraries that libgdiplus uses - such as libpng, libjpeg,... have matured for a long time so there's some value to that, too. Graphics code requires a special skillset anyway, perhaps the bulk of people with that skillset are knowledgeable in C?

Perhaps another option is to start with libgdiplus and move code from unmanaged to managed. It could help kick start the System.Drawing implementation by reusing libgdiplus and move towards the goal of cross-platform managed code step by step.

@JimBobSquarePants
Copy link

@JimBobSquarePants JimBobSquarePants commented May 26, 2017

Perhaps another option is to start with libgdiplus and move code from unmanaged to managed. It could help kick start the System.Drawing implementation by reusing libgdiplus and move towards the goal of cross-platform managed code step by step.

If I was to build a managed 2D graphics library (I am) I wouldn't, by choice, re-implement the System.Drawing API.

That said @mellinoe I'm happy to offer and assist in migration if the decision was made to use ImageSharp as an underlying managed library. In Jamestopia though we wouldn't bother and would actively push developers to migrate to better API's. Seems a lot cheaper and easier for MS.

There are some fundamental differences in how the two libraries currently work which would have to be discussed and workarounds considered. We're still unfinished so we have great flexibility to alter the API to improve flexibility and fill gaps which would help a lot.

Pixel Formats
ImageSharp expands all image formats into one of 20 different pixel formats we bundle. System.Drawing stores images in Format8bppIndexed format and similar in their indexed form. There would have to be a workaround developed to do the same.

Image Formats

  • Gif - Our output for gif is different to System.Drawing. We support animated gif plus transparency, we also quantize the output using Octrees by default and dither the result. This is all configurable but we would have to limit our encoder to use the default System.Drawing palette.
  • Png - We use all the available filters to encode pngs to reduce file size plus we support indexed png. We can add a switch to not filter the output scanlines to make the types compatible.
  • Jpeg - Our output is very slightly different to System.Drawing (I think due to rounding errors on decode) and our decoder performance is not as good just now. We need help on that to make it up to scratch.
  • Bmp - Should be no issues there. We support more output types but that can be limited.
  • Tiff - We have a tiff format in the works. Great progress is being made but it is unfinished.

MetaData
System.Drawing bundles everything together into the PropertyItem class from EXIF metadata to gif loop count and frame delay and makes no effort to decode that data. We've split out metadata into generic, EXIF, and ICC components and offer full reading/writing for those data types. You would have to create Property items to match that data and store the raw bytes.

Affine Transforms - Resize, Crop etc
We would need to do some work to ensure that we can produce the same output as System.Drawing. I'm not sure what sampler it uses for the different quality enumerations. We also don't support matrix operations within resize operations so there would have to be work done there.

Drawing
We offer drawing with a very similar API to System.Drawing. There may be a few gaps but I imagine. I know off-hand we haven't implemented all the Hatch bushes @tocsoft is our resident expert there so he can fill you in on what is/isn't there.

I'm sure there's more...

Regarding "Behavior and performance issues", I'm not really aware of them, so it would be good to have them listed, too.

@qmfrederik I had a very, very quick scan of the libgdi codebase and immediately found unsupported areas 1, 2. I've no idea what is out there. I remember having issues with Image.FromStream(stream, useembeddedColorManagement) in the past but I don't know if that is still an issue.

Graphics code requires a special skillset anyway, perhaps the bulk of people with that skillset are knowledgeable in C?

Btw I couldn't read/write C if you held a gun to my head plus I'm teaching myself graphics programming as I write ImagSharp. Shocking 😝

@migueldeicaza
Copy link
Member

@migueldeicaza migueldeicaza commented May 26, 2017

It is worth updating the original issue with the new information provided in the comments.

Mono's System.Drawing implementation was used for Mono's Windows.Forms implementation and we spent a few years tuning it to work with various third party component vendors. We delegated the hard graphic problems to Cairo.

It was also built so that on Windows, it would use the system provided GDI+ library, and on other platforms, it uses Mono's libgdiplus, it achieves this by having libgdiplus surface the same API as the Windows libgdiplus.

This means a couple of things:

  • That Mono's System.Drawing is actually a very thin layer on top of libgdiplus.
  • that on Windows, you get to use the real GDI+, which is quite nice.

While the implementation works, the font handling capabilities are complicated on Unix, so we used FreeType to get this information. While this works great on Linux systems, it is very slow on first use on a Mac that typically lacks FreeType, so if I was going to do the work in this space, I would start by improving libgdiplus to have a swappable font backend and use Apple's CoreText on OSX to avoid the initial font cache creation problems.

Cairo and the underlying libraries are faster than any C# code you can write.

That said, if I was going to take on the long-term maintenance of System.Drawing, today I would take a different approach.

I would use SkiaSharp as the backing engine as it is actively maintained, actively developed, actively fine tuned and optimized for both Chrome and Android. It also addresses some of the design limitations of Cairo, both the bottlenecks of the API as well as issues with the coordinate space that happen due to loss of precision with huge matrix transformations (Mozilla went through this, and made essentially this switch as well).

I would start with the sysdrawing-coregraphics which is a project that moved most of the logic from libgdiplus from C to C# and replaced the calls to Cairo with calls to CoreGraphics.

And then I would replace the calls to CoreGraphics with calls to the Skia APIs.

Last year I did some work along those lines on my spare time, where I mostly butchered the codebase so it would compile without CoreGraphics and I was planning on adding the Skia support. The time to do that never materialized. If you are interested in that fork, you can find it here.

@JimBobSquarePants
Copy link

@JimBobSquarePants JimBobSquarePants commented May 26, 2017

Just had a thought. What about server support. What are the plans there? That should factor into any decision making.

@TheBlueSky
Copy link

@TheBlueSky TheBlueSky commented May 26, 2017

So what is the goal here? Only API compatibility? This is what I understood from what @mellinoe said.

If this is the case, @migueldeicaza comparison between native code and C# isn't very relevant, as long as C# performance is decent:

Cairo and the underlying libraries are faster than any C# code you can write.

I mean if building the API in managed code is an option, let's go for it.

On the other hand, if we settled on "let's just create something" option, let's not go with Win32-only option; this makes it a bit confusing.

@vibeeshan025
Copy link

@vibeeshan025 vibeeshan025 commented May 26, 2017

Great initiative!! Its better to have System.Drawing implemented on top of set of interfaces and let the community develop each implementation.
Example we can use DirectX on windows and OpenGL on linux and mac OS. but the System.Drawing will internally call the interfaces which is platform agnostic.
If we are going for windows only option then its not going to make any difference to existing .net frameworks.

@JimBobSquarePants
Copy link

@JimBobSquarePants JimBobSquarePants commented May 26, 2017

You need good APIs for that that to be a viable idea. This is a stopgap not a solution.

@bitapparat
Copy link

@bitapparat bitapparat commented May 26, 2017

I'd personally prefer a more forward-looking approach with an officially supported modern cross-platform graphics API for all .Net platforms, and a generic and optional System.Drawing implementation on top of it. But i'm aware that's a rather extensive and expensive solution and, thus, probably unrealistic.

@gulbanana
Copy link

@gulbanana gulbanana commented May 26, 2017

I would love to see something like an implementation on top of ImageSharp. This hypothetical System.Drawing port isn't for high performance or high quality use - it's just to remove a blocker for porting legacy code in which image handling might be one small function. It would be fine if it was slower than the original netfx System.Drawing or didn't produce bit-identical output in all ops- it might even be able to get away with throwing NotSupported for some rarely-used apis.

Building on top of a community library with a modern API would provide a path forward for people to move their code to make direct use of that implentation later on.

@stefanolson
Copy link

@stefanolson stefanolson commented May 26, 2017

Whilst I'm aware that a lot of people use System.Drawing, the WPF System.Windows.Media api is a lot more capable in many areas, and more similar to the UWP API so personally I would rather see a better, more flexible/modern API than System.Drawing available, although of course there may be valuable things in System.Drawing that may be worth implementing

@Rekeyea
Copy link

@Rekeyea Rekeyea commented May 26, 2017

IMHO System.Drawing should be left as Windows only. It is already being used in .Net Framework projects using ASP.Net Core, but those will still be using Windows since those are enterprise applications and I don't see them moving their whole infrastructure "because now they can".

@ViktorHofer
Copy link
Member

@ViktorHofer ViktorHofer commented May 26, 2017

IMHO System.Drawing should be left as Windows only. It is already being used in .Net Framework projects using ASP.Net Core, but those will still be using Windows since those are enterprise applications and I don't see them moving their whole infrastructure "because now they can".

I disagree. ASP.NET Core is being used by more and more open source projects and startups because of cross-platform support and its speed improvements in comparison to nodeJS. I think it's viable to support System.Drawing on all environments where .NET Core and ASP.NET Core is running.

I'm a little worried about binary compatibility here. I think that we definitely need a System.Drawing implementation which is compatible with .NET Framework but I also think that we should build something (together with the community) which is modern, scalable and fast. It might be a good idea to 1. port the existing lib to core (managed) and 2. build a successor of System.Drawing which has all the innovations and improvements which aren't possible with the current API spec.

@Rekeyea
Copy link

@Rekeyea Rekeyea commented May 26, 2017

@ViktorHofer it was said that there are other more modern alternatives like ImageSharp or SkiaSharp. That's why I think it's pointless since those open source projects and startups should use those other alternatives.

@JimBobSquarePants
Copy link

@JimBobSquarePants JimBobSquarePants commented May 26, 2017

@ViktorHofer I've literally spent the last two years of my life doing just that with ImageSharp. It's already faster than System.Drawing in many respects and has a much simpler/easier to extend API.

It needs the community to step up to help me push it over the line. Building something else would be a crushing blow to me.

@ViktorHofer
Copy link
Member

@ViktorHofer ViktorHofer commented May 26, 2017

@JimBobSquarePants Then we should definitely also talk about building upon existing libraries 👍 If you put a lot of effort into it we shouldn't exclude ImageSharp from the discussion.

@benaadams
Copy link
Member

@benaadams benaadams commented May 26, 2017

What's the goal?

  1. Is it so people can move their apps from Framework to Core with minimal changes?
  2. Is it so people can move their apps from Framework on Windows to Core on Linux?
  3. Is it so people can develop new server apps with an api that is remarked as:

Classes within the System.Drawing namespace are not supported for use within a Windows or ASP.NET service. Attempting to use these classes from within one of these application types may produce unexpected problems, such as diminished service performance and run-time exceptions.

I had initially imagined it was for (1) where people used System.Drawing in spite of the warnings; but maybe is for (2) and (3)?

@willdean
Copy link
Collaborator

@willdean willdean commented May 26, 2017

@JimBobSquarePants What needs to be done to remove the "in early stages (alpha). As such, we cannot support its use on production environments until the library reaches release candidate status." warning from your project?

It's rather off-putting, and given the way that the .NET Core team have redefined 'Release Candidate' to mean what 'Alpha Experimental Technology Preview' might have once meant, it's now anyone's guess in the world of .NET what "early stages (alpha)" might mean, but it does sound like "not something I want to invest time in right now".

If you want to mop-up the world of System.Drawing users - I'm one, with a important (to us) ASP.Net application which can't currently be ported to .NET Core because it depends on, inter alia, System.Drawing - then you perhaps need to work a little on the optics for that market.

  • Would it be feasible to have a big table of features with green check-marks and red crosses to show the current state of feature parity with System.Drawing and then get rid of the generic warning?
  • Can you get onto Nuget rather than MyGet? (albeit marked pre-release?)
  • Can you make it clear that it works with full .NET, rather than people needing to pick through the platform types?
  • Don't publicly admit that you taught yourself graphics programming while working on this library. :-)

I don't want to see anyone suffer 'a crushing blow', especially an unintended one from a myopic pachyderm like MS, but I went to look at ImageSharp this morning and came away discouraged.

@Kukkimonsuta
Copy link

@Kukkimonsuta Kukkimonsuta commented May 26, 2017

I'd go with path of least resistance - identify which scenario would be the cheapest and do that. Put warnings all around that it's just compatibility layer that should not be used for new projects. If feeling bad about leftover manpower invest it in improving ImageSharp :)

@qmfrederik
Copy link
Contributor

@qmfrederik qmfrederik commented May 26, 2017

@willdaen Out of curiosity, did you have a look at CoreCompat.System.Drawing (Mono's Sysyem.Drawing ported to Core)? Would that meet your requirements?

@qmfrederik
Copy link
Contributor

@qmfrederik qmfrederik commented May 26, 2017

@Kukkimonsuta Yeah, that kind of is what I thought this was about - a cross platform impkementation of System.Drawing which people can (temporarly) use as they move to .NET Core - basically removing one of the adoption blockers.

It may not be the most pure solution, but it kind of is what libgdiplus was built for.

@JimBobSquarePants
Copy link

@JimBobSquarePants JimBobSquarePants commented May 26, 2017

@willdean

I went to look at ImageSharp this morning and came away discouraged.

Really? Well that's terribly disappointing.

You didn't see the feature list linked to from the readme? I need to update that actually to add all our Porter Duff support and flesh out some other things.

Can you make it clear that it works with full .NET, rather than people needing to pick through the platform types?

It's in the readme.

Built against .Net Standard 1.1 ImageSharp can be used in device, cloud, and embedded/IoT scenarios

The message means exactly what it says. The API is subject to change. (Btw I don't think MS have redefined anything, they just rushed things certainly but alpha, beta etc is not defined by them)

What needs to be done to remove the "in early stages (alpha). As such, we cannot support its use on production environments until the library reaches release candidate status." warning from your project?

We need to do two things.

  1. Get the jpeg decoder refactored to reduce memory usage and speed it up.
  2. Change the IPixel interface to not pack/unpack from bytes and use predefined structs instead. RGBA32, RGB24, BGRA32, BGR24 and also use refs for the Vector4 packing/unpacking

After that I'm happy to move to beta and add the library to Nuget.

It's been a very slow alpha as I want to get this right. There's simply no point creating a half/baked graphics library. No one will use it and you'll end up wasting your time.

P.S. I've no shame in admitting I learned by doing. It's the absolute best way to thoroughly understand things IMO and I see it as an encouragement that others can do the same. 😄

@willdean
Copy link
Collaborator

@willdean willdean commented May 26, 2017

@JimBobSquarePants Thanks for the response - I am really reluctant to hijack this thread by extending discussion about ImageSharp specifically... I'll give it a try and file any issues on your repo.

@willdean
Copy link
Collaborator

@willdean willdean commented May 26, 2017

@qmfrederik I haven't looked at CoreCompat.System.Drawing - we're blocked on several things (SignalR is another) and I haven't really searched hard for alternatives to everything which is missing - I just keep a vague eye on what's happening to the things I know will cause us problems.

@karelz
Copy link
Member

@karelz karelz commented May 26, 2017

Lots of great feedback from anyone, thank you! (I didn't catch up on all of it)

Here's how I look at System.Drawing priorities and options:

Priorities

  1. Primary goal of System.Drawing is to simplify migration from Desktop to .NET Core (on Windows), i.e. high-compat between Desktop and Core implementation on Windows.
  2. Second-level goal is to support System.Drawing where it is needed -- currently Mono needs it on Mac & Linux (Mono is trying to reuse CoreFX libraries / source code, so synergy is desired).
    • Note: Mono has additional requirement on iOS -- size has to be small (i.e. it requires using Mac APIs - @marek-safar to confirm if I didn't mix it up with our discussion about HttpClient).
  3. Third-level (stretch) goal is to simplify migration from Windows to Linux/Mac on .NET Core.
    • Note: Easy migration does not necessarily imply great perf.
  4. Non-goals are high-perf, rich OS-specific capabilities, modern API surface, etc. (anything not listed in (1)-(3)).

Solutions

(2) Mono needs it on Mac & Linux

  • From our chats with @marek-safar we learned that Mono implementation on Linux is extremely involved - it requires special builds for specific versions of distros (not even just distros, but their versions).
  • We agreed with @marek-safar that Linux implementation's complexities are not something we should try to port to CoreFX (due to high cost, little value). We should rather prepare simple migration path (see (3)).
  • @migueldeicaza how does that align with your reply? Is libgdiplus the complex piece @marek-safar told me about? Is there a viable option to NOT migrate that part into CoreFX?

(3) simplify migration from Windows to Linux/Mac

  • Platforms which are easy to implement (e.g. Mac implementation from Mono), we should just implement.
  • For platforms which are not feasible to implement due to cost/benefit, we should recommend migration to 3rd party libraries (ImageSharp, SkiaSharp, etc.).
  • To make migration easier for devs, we could invest into shims which fake old System.Drawing API surface on top of the libraries. The shims would likely be in different namespace though (e.g. SkiaSharp.Drawing.Compat), so a small code change and/or cross-compilation would be required. The shims should probably live in the library repo, or in separate repo (not tied to CoreFX itself to avoid dependency of CoreFX on the libraries).
@qmfrederik
Copy link
Contributor

@qmfrederik qmfrederik commented May 26, 2017

@karelz Regarding the Mono implementation on Linux, I don't think it is involved at all. In fact, libgdiplus, the native part, ships with all Linux distros .NET Core supports. So you don't need to port it, the distros have already done that for you. You can declare a dependency on libgdiplus pretty much the same way you do for openssl or curl - acquire it via the distro instead of NuGet.

You can consider distributing libgdiplus and its dependencies as native components un NuGet packages. That does get complicated, fast - I tried with CoreCompat.System.Drawing.

Mono/libgdiplus gets you 1, 2 and 3 at almost no cost - most of the work has already been done. The result is in the CoreCompat.System.Drawing NuGet package.

@karelz
Copy link
Member

@karelz karelz commented May 26, 2017

@qmfrederik I was relying only on info from @marek-safar who works on Mono - so I let him chime in.

@migueldeicaza
Copy link
Member

@migueldeicaza migueldeicaza commented May 26, 2017

Mr @qmfrederik is correct, libgdiplus is available on pretty much every Linux distribution already and it only relies on the unmanaged pieces. If Marek had any feedback on dealing with build complexity, it would be around you trying to ship a forked version of yours, and then you would have the same issues that you have in shipping CoreCLR - namely that you will want to compile for each system.

So that should not be an issue if you want to take the fastest/easiest path to bring System.Drawing.

You do not need to worry about iOS, for historical reasons System.Drawing is not used there and we do not even distribute it (beyond the basic Rectangle, Point types).

On iOS, the world has gone with either CoreGraphics or our cross-platform SkiaSharp.

Given the outlined priorities from @karelz, I would change my recommendation. Given these priorities, really System.Drawing wont be a long-term foundation for graphics, and I would essentially just reuse the Mono code.

Rely on gdiplus, and use Mono's System.Drawing stack (and ifdef out parts that wont work on .NET Core 2).

You can certainly choose a different path, but given the above goals, it sounds like investing on rewriting this code is a waste of time.

@karelz
Copy link
Member

@karelz karelz commented May 26, 2017

Given these priorities, really System.Drawing wont be a long-term foundation for graphics, and I would essentially just reuse the Mono code.

That was my thinking as well. With the caveat that we might want to use Desktop implementation on Windows to have higher-confidence compat (I've been bitten by HttpListener Mono compat recently, so I'm a bit cautious now ;)).

given the above goals, it sounds like investing on rewriting this code is a waste of time.

Agreed. We didn't plan huge investment. And I am not fan of wasting effort.

Overall, I don't think System.Drawing is such a great API & implementation that we want to innovate on it -- other projects like SkiaSharp, ImageSharp already did that, so let's rely on them to innovate for future in the graphics space. We should not reinvent the wheel in CoreFX, unless there are really good reasons. One .NET ecosystem to rule them all! ;)

@karelz
Copy link
Member

@karelz karelz commented Sep 28, 2017

@FixBo which stub do you mean? Throwing PlatformNotSupported exceptions on platforms which are not supported?

@danmosemsft
Copy link
Member

@danmosemsft danmosemsft commented Sep 28, 2017

@FixBo you rae correct that we normally do that, eg in this case so code that references it could load on UAP (even if not use it). @FixBo what platform would you need this for -- what scenario?

@FixBo
Copy link

@FixBo FixBo commented Sep 28, 2017

@danmosemsft I'm in process of porting a set of libraries to netstandard. Some part's of them relies on Drawing, eg document exporting/printing. On the other hand document generation/pdf export/dataaccess can work on pure netstandart.
For now I have three options: change target to netcore, split dlls, use a lot of defines.

@qmfrederik
Copy link
Contributor

@qmfrederik qmfrederik commented Sep 28, 2017

@FixBo Is your issue that System.Drawing.Common targets netcoreapp2.0 but not netstandard?

@FixBo
Copy link

@FixBo FixBo commented Sep 28, 2017

@qmfrederik Correct.

@qmfrederik
Copy link
Contributor

@qmfrederik qmfrederik commented Sep 28, 2017

@FixBo I think the plan is to have System.Drawing.Common target netstandard but @mellinoe can confirm.

@mellinoe
Copy link
Author

@mellinoe mellinoe commented Sep 28, 2017

@qmfrederik At the moment, it actually relies on some private state in the Color structure which is not publically available in netstandard2.0, but are implemented (privately) in .NET Core 2.0. We would have to make some structural changes to the code to re-target to .NET Standard 2.0. Perhaps it is worth looking into after we are stable and solid.

@jmroyb
Copy link

@jmroyb jmroyb commented Oct 13, 2017

Will System.Drawing.ColorTranslator (I miss System.Drawing.ColorTranslator.FromHtml) be available at net core 2.1?

@karelz
Copy link
Member

@karelz karelz commented Oct 13, 2017

@safern can you please comment on FromHtml?

@VahidN
Copy link

@VahidN VahidN commented Oct 16, 2017

I can't use https://www.nuget.org/packages/System.Drawing.Common from a .NET Standard project:
error NU1202: Package System.Drawing.Common 4.5.0-preview1-25718-03 is not compatible with netstandard2.0 (.NETStandard,Version=v2.0). Package System.Drawing.Common 4.5.0-preview1-25718-03 supports: netcoreapp2.0 (.NETCoreApp,Version=v2.0)

Do you have any plan to make it compatible with .NET Standard projects too. Library authors would appreciate it.

@safern
Copy link
Member

@safern safern commented Oct 17, 2017

I can't use https://www.nuget.org/packages/System.Drawing.Common from a .NET Standard project:
error NU1202: Package System.Drawing.Common 4.5.0-preview1-25718-03 is not compatible with netstandard2.0 (.NETStandard,Version=v2.0). Package System.Drawing.Common 4.5.0-preview1-25718-03 supports: netcoreapp2.0 (.NETCoreApp,Version=v2.0)
Do you have any plan to make it compatible with .NET Standard projects too. Library authors would appreciate it.

Hi @VahidN thanks for bringing that up. We created issue: https://github.com/dotnet/corefx/issues/24675 and I'll be solving the issue in the upcoming days.

Will System.Drawing.ColorTranslator (I miss System.Drawing.ColorTranslator.FromHtml) be available at net core 2.1?

I will have to look why we didn't port it from the beginning. @jmroyb I'll update you on that.

@mqudsi
Copy link

@mqudsi mqudsi commented Dec 26, 2017

@mellinoe and others: it has taken me considerable time but I’ve read - carefully - through each article nd every comment here. Without weighing in on some of the implementation specific details, I have a question that was mentioned in passing but not addressed.

What is the benefit to providing this as a part of .NET Core? Please do not misunderstand me - I appreciate the importance of System.Drawing as a blocking dependency preventing any legacy codebases from migrating to .NET Core. But why must this be addressed by adding a legacy, deprecated, highly system-specific library to the actual .NET Core/Standard specification and implementation? To rephrase: how would simply providing a NuGet package that implements, via one or more of the means discussed above, System.Drawing on these specific platforms not be a sufficient stopgap to help these developers that do not have access to the needed resources to migrate to a better/more modern alternative?

By including this in .NET Core, we are not only committing to providing a solution for platforms currently supported by CoreFx but also severely restricting what platforms CoreFx can be easily ported to in the future.

It’s all well and good to discuss how awesome SkiaSharp support is on Linux and Android and whatnot, but what happens when someone wants to port CoreFx to SPARC or even just some archaic ARM variant not supported by Skia or libgdiplus or whatnot? What about FreeBSD? Or another OS not yet developed?

I am all for developing this library, but I think careful thought needs to be given to its repercussions. I am speaking as a die-hard SWF fan that shunned Silverlight, WPF, and XAML until Windows 10 Creators Update, when I finally embraced UWP. I deeply appreciate the difficulties in learning the subtleties and nuances of new toolkit’s and the difficulties in migrating to a platform with missing dependencies. But what is to be gained by bending over backwards to support System.Drawing?

It seems this discussion has almost derailed into a conversation about support SWF on CoreFx and lost sight of the initial goal: to make it easier for applications consuming SD to migrate to CoreFx. I would really appreciate if you would take a step back and perhaps reevaluate this. I do appreciate the time and effort already invested in this undertaking, but none of it need by lost, simply publishing everything discussed but under the context of a separate package than part of .NET Core would accomplish all the same goals.

Now to ask a different question that was previously asked but not answered: while SD is definitely a major blocking concern for users of legacy .NET FX apps on Windows blocking their migration to CoreFx, what is the actual number of projects that would benefit from this being a Windows-only NuGet package vs this giant effort to support it across all platforms? How many organizations currently using SD are expected to migrate to .NET Core if SD were available and expand to non-Windows platforms that haven’t already used a better alternative to SD/GDI+?

@benaadams
Copy link
Member

@benaadams benaadams commented Dec 26, 2017

@mqudsi I assume its being provided via the Windows Compatibility Pack for .NET Core rather than being part of .NET Standard

@safern
Copy link
Member

@safern safern commented Dec 26, 2017

If I understand clearly your concern is that by adding System.Drawing to .NET Core it would make really hard to extend/port this platform to another framework, platform using different libraries right?

If that is your concern and I understood correctly, System.Drawing is not inbox in .NET Core neither netstandard, which means that it is shipped as an independent package in NuGet and if people want to use it they just reference that package in their project. This was done as a compatibility effort to make migration from .NET Framework to .NET Core easier. So we are accomplishing both goals, compatibility and we’re not adding this as part of the framework inbox. Which for your concern (which is reasonable) doesn’t affect you either from what I understood in your comment. Please feel free to correct me if that wasn’t your main concern.

To answer your last question, the point of .NET Core is provide portability and extensibility to different platforms, if developers have a .NET Framework app that use System.Drawing but they don’t want to port it because they don’t gain much because it will be Windows-Only supported (the same as they have in their current app), what would be the pint of this effort vs if we support it in Unix also, they gain the portability of their app for free and is a good incentive to move to .NET Core.

@safern
Copy link
Member

@safern safern commented Dec 26, 2017

@mqudsi I assume its being provided via the Windows Compatibility Pack for .NET Core rather than being part of .NET Standard

Correct, and also as an individual package in NuGet.

@mqudsi
Copy link

@mqudsi mqudsi commented Dec 26, 2017

Thank you! This is my second forray into the github page for CoreFX (first was my PR from earlier today) and presumed the issue being here meant it was part of “core” CoreFx.

Thanks for clarifying!

@JimBobSquarePants
Copy link

@JimBobSquarePants JimBobSquarePants commented Dec 27, 2017

To answer your last question, the point of .NET Core is provide portability and extensibility to different platforms, if developers have a .NET Framework app that use System.Drawing but they don’t want to port it because they don’t gain much because it will be Windows-Only supported (the same as they have in their current app), what would be the pint of this effort vs if we support it in Unix also, they gain the portability of their app for free and is a good incentive to move to .NET Core.

I would argue that offering System.Drawing support on Unix would offer zero incentive to move to .NET Core

@qmfrederik
Copy link
Contributor

@qmfrederik qmfrederik commented Dec 27, 2017

I would argue that offering System.Drawing support on Unix would offer zero incentive to move to .NET Core

There seems to be demand for System.Drawing on .NET Core; CoreCompat.System.Drawing, CoreCompat.System.Drawing.v2 and System.Drawing.Common combined have over 300,000 downloads on NuGet.

To put the "giant effort to support it across all platforms" in some context: Mono has ported System.Drawing to Linux and macOS more than 10 years ago. They've also made libgdiplus compatible with ARM-based OSes such as Android and iOS. What CoreFX is doing is consolidating the Windows, Linux and macOS implementations of System.Drawing.

At the end of the day, it's another NuGet package which is available to you, and you're free to use any imaging library which meets your requirements 😄.

@clrjunkie
Copy link

@clrjunkie clrjunkie commented Dec 27, 2017

There seems to be demand for System.Drawing on .NET Core; CoreCompat.System.Drawing, CoreCompat.System.Drawing.v2 and System.Drawing.Common combined have over 300,000 downloads on NuGet.

Now that System.Drawing is out of the BCL (a good thing) I think it's important we have new basic (but quality) image manipulation support as part of the BCL, specifically for image resizing and rotation which does not mandate any external dependency.

@danmosemsft
Copy link
Member

@danmosemsft danmosemsft commented Feb 2, 2018

@safern is this issue still useful? We already have an issue to track marking the compat pack stable. If we want to track consolidating the Windows and Unix code further it would probably make sense to have a new more focused issue.

@safern
Copy link
Member

@safern safern commented Feb 2, 2018

Agree. We can close this issue.

@safern safern closed this Feb 2, 2018
@voltcode
Copy link

@voltcode voltcode commented Nov 5, 2018

I was researching a way to migrate our apps toward net core with Platform Extensions and noticed that System.Drawing.ImageConverter is not present in Platform Extensions. Is it going to be suppported? Is this the best place to report such thing, or should I raise a separate issue or report it somewhere else ?

@benaadams
Copy link
Member

@benaadams benaadams commented Nov 5, 2018

@voltcode probably should open a new issue; for tracking, if its not already in the Windows Compatibility Pack

@danmosemsft
Copy link
Member

@danmosemsft danmosemsft commented Nov 5, 2018

@voltcode this would be a new issue, but actually it is already being added in dotnet/corefx#33092 which you can track.

After that .NET Core should have everything that was in System.Drawing.dll in the System.Drawing namespace - I just checked.

@msftgits msftgits transferred this issue from dotnet/corefx Jan 31, 2020
@msftgits msftgits added this to the 2.1.0 milestone Jan 31, 2020
@hannespreishuber
Copy link

@hannespreishuber hannespreishuber commented Jun 11, 2020

doenst work with blazor Webassembly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.