System.OutOfMemoryException: at System.Buffers.TlsOverPerCoreLockedStacksArrayPool`1.Rent

[stephenschwan]

Unfortunately I was unable to get a dump of the process before the app service was restarted, but we have never encountered this before. No rhyme or reason - there was no memory pressure on the servers (web app - s3). I have attached the stack trace - though i don't think it will provide much.

Though the bigger question is, if this happens again, what should I trace or collect such that we have enough info.

thanks,
stephen
outofmemstacktrace.txt

[RussKeldorph]

@benaadams Is your comment on dotnet/corefx#27185 implying you have a way to consistently reproduce this?

[benaadams]

No, a way to not use the ArrayPool for HttpResponseStreamWriter which it where the call stack is throwing

System.OutOfMemoryException:
   at System.Buffers.TlsOverPerCoreLockedStacksArrayPool`1.Rent
   at Microsoft.AspNetCore.WebUtilities.HttpResponseStreamWriter..ctor
   at Microsoft.AspNetCore.Mvc.Internal.MemoryPoolHttpResponseStreamWriterFactory.CreateWriter
   at Microsoft.AspNetCore.Mvc.Formatters.JsonOutputFormatter+<WriteResponseBodyAsync>d__11.MoveNext

More specifically outlined in the use case of this api request https://github.com/dotnet/corefx/issues/27268

The HttpResponseStreamWriter holds onto the arrays for lifetime of writing/request so long running/blocked requests would cause increased ArrayPool usage; however I'm not sure why it would OOM though as the numbers returned to the pool are capped; so it shouldn't cause a greater issue than allocating fresh.

Related issue of the ArrayPool consuming available memory https://github.com/dotnet/corefx/issues/25841

[RussKeldorph]

Ok thanks for the clarification.

[RussKeldorph]

@jkotas Does this appear actionable? If so, could you help triage it? If not, please close.

[jkotas]

Related to or duplicate of https://github.com/dotnet/corefx/issues/25841 and https://github.com/dotnet/coreclr/issues/7747 .

[Maoni0]

yes, all these should be dups of https://github.com/dotnet/corefx/issues/25841. GC just provides an API to indicate what memory situation we are in; it's really up to the pool implementation to decide on trimming.