Saving a CanvasCommandList to file

[ratishphilip]

I am trying to apply some cropping to an image, captured using MediaCapture, using the CanvasCommandList and then save it to a file.

public async Task CapturePhotoAsync(CanvasAnimatedControl canvas)
{
    var file = await KnownFolders.PicturesLibrary.CreateFileAsync("Photo.jpg", CreationCollisionOption.GenerateUniqueName);

    using (var captureStream = new InMemoryRandomAccessStream())
    {
        await mediaCapture.CapturePhotoToStreamAsync(ImageEncodingProperties.CreateJpeg(), captureStream);

        captureStream.Seek(0);

        var inputBitmap = await CanvasBitmap.LoadAsync(canvas, captureStream);
        var cl = new CanvasCommandList(canvas);
        using (var clds = cl.CreateDrawingSession())
        {
            using (clds.CreateLayer(1f, new Rect(0,0,3264,1840)))
            {
                clds.Transform = Matrix3x2.CreateRotation((float)Math.PI/2f, TransformCenter) *
                        Matrix3x2.CreateScale(1.4f, TransformCenter);
                clds.DrawImage(inputBitmap);
            }
        }

        using (var stream = await file.OpenAsync(FileAccessMode.ReadWrite))
        {
            try
            {
                await CanvasImage.SaveAsync(cl, new Rect(0,0,3264,1840), 96, canvas, stream, CanvasBitmapFileFormat.Jpeg);
            }
            catch (Exception e)
            {
                var ex = e.ToString();
            }
        }
    }
}

When I am executing this method, it throws an exception at CanvasImage.SaveAsync call

System.Exception: The object was not in the correct state to process the method. (Exception from HRESULT: 0x88990001)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at CameraManager.<CapturePhotoAsync>d__56.MoveNext()

What could be the reason for this exception?

[shawnhar]

I don't see anything obviously wrong with this code sample.

You may be able to get more info (ideally a failure callstack) if you debug with Visual Studio set to native or mixed mode debugging, and with first chance C++ exceptions turned on?

[ratishphilip]

@shawnhar

Here is a detailed call stack

Microsoft C++ exception: HResultException at memory location 0x081FFCB0.

KernelBase.dll!RaiseException�() Unknown
vcruntime140_app.dll!_CxxThrowException�() Unknown
Microsoft.Graphics.Canvas.dll!ThrowHR(long) Unknown
Microsoft.Graphics.Canvas.dll!ABI::Microsoft::Graphics::Canvas::DefaultCanvasImageAdapter::SaveImage(struct ID2D1Image *,struct WICImageParameters const &,struct ID2D1Device *,struct IStream *,struct _GUID const &,float) Unknown
Microsoft.Graphics.Canvas.dll!std::_Func_impl<class <lambda_a6397a56ba7d9fa8cd76269d8f56ca81>,class std::allocator,void>::_Do_call(void) Unknown
Microsoft.Graphics.Canvas.dll!ExceptionBoundary<class <lambda_054de472c57785f392dd64f989f68c1d> >(class <lambda_054de472c57785f392dd64f989f68c1d> &&) Unknown
Microsoft.Graphics.Canvas.dll!Microsoft::WRL::Details::InvokeHelper<struct Microsoft::WRL::Implements<struct Microsoft::WRL::RuntimeClassFlags<2>,struct ABI::Windows::System::Threading::IWorkItemHandler,class Microsoft::WRL::FtmBase,class Microsoft::WRL::Details::Nil,class Microsoft::WRL::Details::Nil,class Microsoft::WRL::Details::Nil,class Microsoft::WRL::Details::Nil,class Microsoft::WRL::Details::Nil,class Microsoft::WRL::Details::Nil,class Microsoft::WRL::Details::Nil>,class <lambda_51ef2b7b49cffbefda5c493004cc8e5f>,1>::Invoke(struct ABI::Windows::Foundation::IAsyncAction *) Unknown
threadpoolwinrt.dll!Windows::System::Threading::CThreadPoolWorkItem::CommonWorkCallback(void) Unknown
threadpoolwinrt.dll!Windows::System::Threading::CThreadPoolWorkItem::BatchedCallback(struct _TP_CALLBACK_INSTANCE *,void *,struct _TP_WORK *) Unknown
ntdll.dll!TppWorkpExecuteCallback() Unknown
ntdll.dll!TppWorkerThread�() Unknown
ntdll.dll!RtlUserThreadStart�() Unknown

[shawnhar]

Interesting, that looks like it's made it past all the more obvious places I might expect things to fail.

What would it take to get us set up with a repro app we can debug?

[ratishphilip]

I have created a repro app from here and modified it to test this scenario.

Once you run the app, click on the '+' button at the bottom of the screen. In the next page that loads up, click on the button with the red dot. This will try to capture a photo and call the CapturePhotoAsync method in RecordingPage.xaml.cs

Here is the project
VideoCaptureWithEditing.zip

[ratishphilip]

@shawnhar
I am able to repro it on Lumia 640 and Lumia 950 XL. Both are on latest Insider builds.

[shawnhar]

Thanks for the repro. Unfortunately on my machine I can't get as far as the Win2D image saving code because this is failing somewhere in the media capture. Afraid I don't understand these media APIs well enough to figure out why :-( After I enable mixed mode debugging, when I click the red button I get a failure in StopPreviewAsync:

System.Runtime.InteropServices.COMException: 'The sample allocator is currently empty, due to outstanding requests.

Is it possible to separate the Win2D code you are having trouble with from the video capture stuff, or do you need both together to repro the failure?

[ratishphilip]

Can you connect a mobile to your machine and let it run in that?
I am just trying to obtain a IMemoryRandomAccessStream from the MediaCapture, convert it into a CanvasBitmap, then creating a CanvasComandList for rotating and cropping the output image by using a CreateLayer. So far so good.
It is when I try to save the CanvasComandList to a file, I get the exception.

[ratishphilip]

@shawnhar I have created a simple repro project containing the Win2d code only.

Just build and run the app and click on the Save button. In the handler I am loading a local image resource on to a CanvasBitmap and using CanvasCommandList to rotate and crop the output image.

I am then trying to save this CanvasCommandList to a StorageFile. While calling the CanvasImage.SaveAsync method the exception is thrown.

Here is the project
TestCanvasSave.zip

[shawnhar]

Thanks, I can repro this now, and understand the issue.

Workaround: wrap your command list in a no-op image effect, and pass that effect rather than the command list directly to SaveAsync.

Notes for my future reference: the problem is that CanvasImageFactory::SaveWithQualityAndBufferPrecisionAsync is calling GetWrappedResource, when it should be using GetD2DImage. The former is intended for interop, and skips behaviors such as closing active command lists that are necessary before the D2D interface can be used as an image.

[ratishphilip]

Thank you!

[ratishphilip]

It is working fine with the workaround.
Hopefully the bug will be fixed soon.

[shawnhar]

This is fixed in Win2D 1.21.0 (released today)