Opening IMetaDataEmit causes 40-50% additional memory usage

[discostu105]

I am (finally) raising this issue to point out a known problem, that we've had for a long time and that is repeatedly causing us problems with customers with our Profiler (Dynatrace .NET Code Module).

Opening IMetaDataEmit causes significant memory overhead. Concretely, it allocates ~50% of the size of a Module in additional memory. AFAIK this comes from the fact that the metadata tables are being copied into new datastructures, which are then writable to be able to emit new metadata.

In fact the same issue has been raised by us already in 2008 (!) on Microsoft connect. Although connect is long offline, I still have the original answer from back then:

Posted by Microsoft on 8/26/2008 at 10:23 AM

Hello, Andreas, thank you for entering this into Connect. As we've discussed, this will be an item for us to consider in future releases, though we will most likely not have time to address this in our next major release. I will therefore be closing this issue for now, but the ID you have (361070) is still valid for tracking this. As you continue to receive feedback from your own customers on this, we'd like to hear specific complaints they have.

For completeness, I'm pasting below some of my comments from the email thread we've had on this issue.

Our understanding of the situation is that opening one IMetaDataEmit per module will cause, at worst, a new allocation the size of the existing metadata in that module, in order to make it writeable. Generally, that will translate to about the 40-50% memory usage Karel mentioned below, assuming on average a given module's metadata accounts for about 40-50% of its size. It's important to note those percents are based on the size of the modules, and not the size of the overall memory usage of the app at run-time. So it will depend on the extent of a given application's memory usage, and the contents of its modules' metadata, as to whether this would actually be a significant overhead for that app. I suspect some apps will be affected more by this issue than others.

Further, just increasing memory usage doesn't necessarily translate into a poor experience. For example, if the original metadata just gets paged out of memory and is replaced by the IMetaDataEmit (writeable) copy, then even though committed memory increases, the working set might actually remain constant. I don't know if that's the case with your scenarios, but it's worth measuring.

Our profiler instruments a couple of large assemblies always (mscorlib/System.Private.CoreLib, System, System.Data, ...), others on demand or per customer configuration. Theoretically a customer can configure to instrument methods in almost all assemblies. Thinking of a large application with hundreds of assemblies, with multiple hundreds of Megabytes in size, we're dealing with a huge overhead.

There are customers that run hundreds of .NET Apps (e.g. AppPools) on a single machine. It's often hard to explain the additional (total) memory consumption, just by installing our Profiler. Nowadays, where you typically pay-as-you-go for resources (cloud!), this becomes a real cost factor.

So, my request is: Can we find a way to improve this situation? In my naive mind, it should be possible to avoid copying the whole MD table and allocate only memory for newly emitted metadata entries. Lookup functions should be able to differentiate and look into the right datastructures. I know some MD tables are based on offsets (e.g. Fields and Params), but a mapping function should be able to handle that when resolving metadata. I am hoping for a discussion, so we can find a potential solution to fix this.

[tommcdon]

@davmason

[davmason]

Hi @discostu105, I'm happy to start this conversation but I will have to take some time to get familiar with the impact and what we could do about it.

@noahfalk and @davidwrighton are probably the only two who might have historical context on this (either of you feel free to jump in if you do have some context).

[davidwrighton]

Well, this infrastructure is all written this way as IMetadataEmit was designed to be used for emitting files to disk, and all of the data structures in our current implementation, are designed to be used in this manner. The current design works fairly well for that, and for creating entire new metadata from scratch. For the profiler scenario, the approach chosen is clearly suboptimal.

As you may note; however, the feature of saving metadata to disk was removed from reflection when .NET became .NET Core. What is probably less clear is that there still are cases which use the metadata emission code and serialize the metadata.

1. Reflection.Emit creates an in memory metadata representation. which is based on relying on saving metadata. This metadata is needed by debuggers to be easily processed, and the current format relies on the existing in memory arrangement.
2. ILAsm uses the metadata apis to produce metadata. I have a change that I'd like to take into the master branch that removes the need for ILAsm to use the implementation of metadata compiled into coreclr, but that PR still leaves the codebase unified (and metadata generation logic compiled into ilasm directly)

Any change to the metadata implementation to reduce this sort of excessive memory usage would need to keep these scenarios working, and would thus be somewhat costly to implement. OTOH, if this is truly a significant barrier to use of the profiler api, I can imagine a variety of schemes that could be implemented.

It would be useful to have some numbers for the impacts that you have seen, so that @davmason could produce a good understand of the problem.

[noahfalk]

It would require a bit more research than I did in the past, but there may be an implementation path through here that isn't too costly. From what I recall the main memory overhead occurs when the various memory pools take ownership of the pre-existing data, which they do by copying (see StgPool::TakeOwnershipOfInitMem()). I believe these copies are done partly to simplify the lifetime mechanics and partly to ensure that if the memory was overwritten it doesn't leak back to the source which was supposed to be read-only. In the profiler scenario we could have the RW abstraction maintain a ref count on the original RO version ensuring that the lifetime is maintained, and the profiler updates don't allow existing metadata to be re-written, only new data to be appended so the original memory shouldn't change. The StgPool is already designed as a linked list of memory blocks so having the first block get reused from the RO allocation and later appended blocks come from new allocations should fit into that design.

[EDIT]: Accidentally hit enter too soon.

[k-berger]

I second @discostu105 's concern. The memory overhead can get pretty severe for larger applications. Thinking about the density of apps per core some people run, especially on their old IIS deployments, some servers that are being equipped with state of the art tracing might run into a memory issue.

[jkotas]

The memory overhead is not the only performance problem of IMetaDataEmit. Acquiring IMetaDataEmit wraps all metadata access inside a lock that makes all metadata operations (type loading, JITing, reflection) significantly slower.

I am wondering whether it would be better to have APIs tailored to the operations that the profilers actually want to do. E.g. we can have API to provide a new method IL that is produced via a scheme similar to DynamicMethods (tokens are local), without any metadata editing.

What are the other operations that the profilers do and cannot live without, e.g. do profilers ever add new fields to existing types?

[noahfalk]

I am wondering E.g. we can have API to provide a new method IL that is produced via a scheme similar to DynamicMethods (tokens are local), without any metadata editing.

Its certainly possible and in some scenarios I'd imagine it might be quite advantageous, but it also sounds like a more expensive solution requiring new API design, changes on both the profiler and runtime side and probably dual maintainance with the current solution because of compat requirements. I feel like I'd want to investigate what a more direct removal/replacement of the lock looked like to speed up the existing API before we up'ed the ante move into API redesign territory : )

[discostu105]

What are the other operations that the profilers do and cannot live without, e.g. do profilers ever add new fields to existing types?

So, we currently use a lot of IMetaDataEmit API's such as emitting TypeDefs/Refs/Specs, AssemblyRefs, Custom Attributes, Signatures, etc... to any module (including mscorlib/system.private.corelib).

And yes, we even add new fields & methods to existing types for some features. Could we live without it? Maybe. It would definitively kill some of our features that we would need to drop or re-design. However, since we need this only for some features, we don't need this for every module that we instrument. Having less memory usage for modules where we do not add fields/methods to existing types, would already be a step forward, if that's easier.

An API-Redesign might have it's good reasons, but we currently have a large product built around the current API's. So, if the overhead problem is fixable without (or few) API changes, that would of course be preferred :).

[k-berger]

@jkotas As far as IMetaDataEmit is concerned we rely on

• DefineTypeRefByName
• DefineMemberRef

which injects members being used by rewritten IL code. Redesigning the API sounds like an expensive solution on both sides, although I can imagine that this might be advantageous in the long run.
But I also have to agree with @noahfalk and @discostu105 that from a maintenance point-of-view it would be preferrable to have only little changes on the existing APIs and also not having to implement two ways of doing the same things (current way and a new one based on entirely new APIs).

[mmayr-at]

cc @karelz