PsiStore Write & PsiStudio Live Visualization Issue

[xiangzhi]

I think these are somewhat related but not sure if its something unique to my machine's setup. I'm trying to record and live preview a Kinect 2 Stream using the following code:

            using (var pipeline = Pipeline.Create(true))
            {
                var store = Store.Create(pipeline, "testStore", @"C:\Data\Stores");
                var kinectSensor = new KinectSensor(pipeline, new KinectSensorConfiguration() { OutputBodies = true, OutputColor = true });

                var numGen = Generators.Range(pipeline, 0, 100, TimeSpan.FromSeconds(1));
                numGen.Write("test", store, largeMessages: true);
                numGen.Do(m => Console.WriteLine(m));

                kinectSensor.Bodies.Write("Body", store, largeMessages:true);
                kinectSensor.ColorImage.Write("Image", store, largeMessages:true);

                pipeline.Run();
            }

The first thing I notice is that PsiStore is being written in 32kb chunks instead of the large message formats (262MBs I think?)

If I close the application and then open the store in PsiStudio, it takes a while to repair the store, but I can still visualize it.

However, when I try to visualize it LIVE when it is running, I get the following exception from the store generating application.

System.AggregateException
  HResult=0x80131500
  Message=Pipeline 'default' was terminated because of one or more unexpected errors
  Source=Microsoft.Psi
  StackTrace:
   at Microsoft.Psi.Pipeline.ThrowIfError() in C:\Users\xiangzht\source\repos\psi\Sources\Runtime\Microsoft.Psi\Executive\Pipeline.cs:line 1265
   at Microsoft.Psi.Pipeline.Run(ReplayDescriptor descriptor) in C:\Users\xiangzht\source\repos\psi\Sources\Runtime\Microsoft.Psi\Executive\Pipeline.cs:line 472
   at Microsoft.Psi.Pipeline.Run(TimeInterval replayInterval, Boolean useOriginatingTime, Boolean enforceReplayClock, Single replaySpeedFactor) in C:\Users\xiangzht\source\repos\psi\Sources\Runtime\Microsoft.Psi\Executive\Pipeline.cs:line 484
   at SimpleTest.Program.Main(String[] args) in C:\Users\xiangzht\source\repos\psi\Playground\SimpleTest\Program.cs:line 29

Inner Exception 1:
IOException: Not enough memory resources are available to process this command.

The memory also went from about 200MB all the way to 1GB before crashing. PsiStudio also showed this message:

I feel like this is a known issue and I seen it before? 😜 Let me know how I could help with debugging this issue.

[chitsaw]

[posted on behalf of @sjdent]

Hi Xiangzhi

For the size of the chunks (extents) being written, from your screenshot it looks like you’re writing 32MB extents, not 32kB extents, and this is correct for a 32-bit application. If your app was 64-bit then the extents would be 256MB each. With 256MB extents, I’m seeing new extents being created about every second, and with 32MB extents that number would be much higher.

You’re generating a lot of data because you’re writing unencoded color images to the store, so it’s possible that the out of memory condition you’re seeing is correct. If you could write encoded JPEGs instead of raw HD images, it would really cut down on your data rate and I think the issue should go away. Switching to a 64 bit application may help as well.

[xiangzhi]

Thank you!
You are right, the main problem where there was too much written data was because I was writing the raw data to the file. Changing it to following fixed it.

kinectSensor.ColorImage.EncodeJpeg().Write("Image", store, largeMessages:true);

I believe the second problem ("Access to path XXX is denied") was probably because the mixed of 32 bit (Application) and 64 bit (Psi Studio). The problem went away after I changed my application to 64 bit. There seemed to be a problem with accessing stores from 32 bit application live with PsiStudio.