While using the OpenCVSharp4 wasm runtime Cv2.ImDecode and Mat.FromImageBytes fail for jpg and png images, while bmp images work. #1512
Description
Summary of your issue
While using the OpenCVSharp4 wasm runtime Cv2.ImDecode and Mat.FromImageBytes fail for jpg and png images, while bmp images work.
Environment
Blazor wasm using OpenCVSharp4 and the OpenCVSharp4 wasm runtime
What did you do when you faced the problem?
I cloned your opencvsharp_blazor_sample repo and it loads the bitmap image "images/Mandrill.bmp" correctly. I then modified it to load a png and then jpg and both of those fail. The resulting Mats are empty.
I modified OpenCVSharpSample.razor method OnAfterRenderAsync with the below code:
Example code:
// bmp
var srcBytes = await httpClient.GetByteArrayAsync("images/Mandrill.bmp");
srcMat ??= Mat.FromImageData(srcBytes);
Console.WriteLine($"srcBytes.length == {srcBytes.Length} srcMat.Empty() == {srcMat.Empty()}");
// jpg
var jpgBytes = await httpClient.GetByteArrayAsync("images/test-image-1.jpg");
var jpgMat = Mat.FromImageData(jpgBytes);
Console.WriteLine($"jpgBytes.length == {jpgBytes.Length} jpgMat.Empty() == {jpgMat.Empty()}");
// png
var pngBytes = await httpClient.GetByteArrayAsync("images/test-image-2.png");
var pngMat = Mat.FromImageData(pngBytes);
Console.WriteLine($"pngBytes.length == {pngBytes.Length} pngMat.Empty() == {pngMat.Empty()}");
Output:
srcBytes.length == 196662 srcMat.Empty() == False
jpgBytes.length == 250239 jpgMat.Empty() == True
pngBytes.length == 3298424 pngMat.Empty() == True
What did you intend to be?
srcBytes.length == 196662 srcMat.Empty() == False
jpgBytes.length == 250239 jpgMat.Empty() == False
pngBytes.length == 3298424 pngMat.Empty() == False
Workaround:
Current workaround, in the browser, is to use an Image element to load the image, then draw it onto a canvas, read the canvas imageData, create a Mat of appropriate size and of the format CV_8UC4, and then write the image RGBA bytes to the Mat using Marshal.Copy. Quite a bit more effort than a single call but it works.