Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High Value Of Private Bytes Metric #5654

Open
luckerby opened this issue Feb 17, 2020 · 3 comments
Open

High Value Of Private Bytes Metric #5654

luckerby opened this issue Feb 17, 2020 · 3 comments
Assignees

Comments

@luckerby
Copy link

@luckerby luckerby commented Feb 17, 2020

I'm using some simple C# code on .NET Core to look at how Azure Functions perform. The use case is quite simple: add 10 million int values to an ArrayList as part of an Azure Function, and use a time trigger set to 1 minute to invoke the function. (I know the ArrayList is obsolete and not to be used anymore, but I'm just employing it to get a significant chunk of space allocated on the heap)

Therefore the C# code is this:

using System;
using System.Collections;

public static void Run(TimerInfo myTimer, ILogger log)
{
    ArrayList numbers = new ArrayList();
    Random random = new Random(1);
    int noNumbers = 10000000;
    for(int i=0;i<noNumbers;i++) {
        numbers.Add(random.Next(10));
    }

    log.LogInformation($"Created an ArrayList of {numbers.Count} elements");
    log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
}

For an ArrayList, each boxed int occupies 12 bytes, due to the type object pointer (aka method table address – 4 bytes), the value stored (an int, which takes 4 bytes on either x86/x64) and the sync block index (4 bytes). All these scattered boxed ints total 114.45 MB (12 bytes/object x 10 mil objects). On top of this we have the internal arrays themselves that get allocated in turn, and which take 128 MB (16 bytes + 32 bytes + 64 bytes + … + 32 MB + 64 MB). The total comes up to 242.45 MB. The value can be confirmed by running a similar test method against BenchmarkDotNet.

There's no much else being allocated on the heap aside the value summed above.

Running VMMap against this code (compiled for .NET Framework) shows the private bytes as close to the expected value (I'm assuming one of the GCs gets a chance to free up some of the allocated data up until 242 MB):
VMMap_ArrayList_10mil_elements_clean

But looking at the performance of the Azure Function, either Function App's "private bytes" or the App Insights' "process private bytes" have considerable higher values (which actually come closer to the overall virtual memory usage as seen in VMMap before):
AzurePortal_ArrayList_10mil_elements_everyMin_PrivateBytes

As the private bytes go into the cost of the Azure Functions, it's of interest to understand why the value observed is much higher than one would expect it to be. I've expanded on the context of the issue here.

@luckerby

This comment has been minimized.

Copy link
Author

@luckerby luckerby commented Feb 17, 2020

There are a few more aspects I've been reviewing:

  • The "bitness" setting for the Azure FunctionApp that contains the ArrayList Azure function. I was assuming in my original post that the platform target is x86, but haven't really checked. The only such configuration seems to point to x86 code:
    AzurePortal_ArrayListFunctionApp_PlatformBitnessSetting
  • My original VMMap output was based on code targeting .NET Framework, yet the Azure Function that I've been using was using .NET Core. Below is the private bytes data in VMMap for an identical ArrayList code running against .NET Core 3.0, targeting x86. The moment when the code finishes running the code (and as such all the memory allocations) is selected in the timeline, just as in my original post. The private bytes is the metric presumably used (link) in the Azure Function cost calculation
    VMMap_ArrayList_10mil_elements_DotNetCore_PrivateBytes
  • Although not the metric we should be interested in - according to the above at least- , the bytes allocated over the lifetime of the function are seen below. The value neither matches the expected memory usage for neither x86 code(~240MB), nor x64 (~500MB). This would suggest x86 code running, but with something extra on top (whose overhead is not negligible), which impacts the private bytes significantly. However, the value observed does come close to the "private bytes" metric measured in the Azure portal for the Azure Function (2nd printscreen in my original post)
    AzurePortal_ArrayList_10mil_elements_AllocatedBytesInCode
    Contrast this to the bytes allocated overall for a Console app running against .NET Core (3.0), using the same AppDomain.CurrentDomain.MonitoringTotalAllocatedMemorySize property as above:
    Console_ArrayList_10mil_elements_DotNetCore_AllocatedBytesInCode
  • How often is the private bytes metric measured. Assuming this happens rarely, there will probably be just one measurement once the function completes. Assuming the GC recclaims the unused data before the code completes the final memory allocations, for the code sample used the reported private bytes should be around 180 MB(the ArrayList is implemented by allocating object[] arrays whose length double as the number of elements outgrow the current one; for the last such internal array, its length will be 16,777,216 and each object reference is 4 bytes (32-bit platform), thus consuming 67,108,864 bytes (exactly 64 MB); the boxed ints themselves will take 10 mil x 12 bytes = 120,000,000 bytes (~117 MB), yielding a total of ~181 MB, matching quite well the private bytes highlighted in the VMMap output above). If on the contrary, the private bytes values are sampled often, then the "ramping up" in virtual memory usage should be captured as well, and the average metric value should be significantly lower than the ~181 MB above, in order to keep things fair
@luckerby

This comment has been minimized.

Copy link
Author

@luckerby luckerby commented Feb 19, 2020

I've also looked at the assemblies loaded in the AppDomain by the .NET Core local app and the Azure Function respectively, using AppDomain.CurrentDomain.GetAssemblies().

For the .NET Core local app there’s a minimum number of assemblies loaded (code output is here) , just as expected, but for the Azure Function one there’s literally a ton (output is here). Judging by the fact that the code one types in the Azure Function actually makes it as an assembly in itself most likely (f-ArrayList_Function__-1846423553 in the second listing), there’s probably some other piece of code that executes, loads the mentioned assembly and somehow ends up taking up memory in parallel.

@soninaren

This comment has been minimized.

Copy link
Contributor

@soninaren soninaren commented Feb 20, 2020

cc: @fabiocav , @alrod .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.