VS 2015 CTP 6 compile error

[talebi]

I've already added a comment to this thread:
https://github.com/dotnet/corefx/issues/1001#issuecomment-84494151

Simply cannot compile a project having a reference to Revit API 2013. It says:
Metadata file 'E:\dev\cadtools_revit\Source\Lib\RevitAPI\RevitAPI_x64_2013.dll' could not be opened -- Row ID or heap offset is too large. The same project compiles in previous VS versions.

[ellismg]

This is a better fit for the roslyn repo. I will move this there.

[nguerrera]

@talebi Is that Revit API publicly available?

[nguerrera]

@talebi I found a download here: http://usa.autodesk.com/adsk/servlet/index?siteID=123112&id=2484975, but seems it has only docs and templates and that I need the full Revit product.

[nguerrera]

I'm checking with some folks internally about contacting Autodesk so that we can work with them on this.

[talebi]

@nguerrera Revit api dlls are part of the actual Revit installation so no, it's not part of the api download.

[talebi]

But if you install Revit you can find it alongside Revit.exe (called RevitAPI.dll)

[jbevain]

@nguerrera this is not related to uints :)

The issue is that the assembly is quite large, and you can find a string heap index that uses more than 24 bits (the index of the name in the module table).

Currently PeekReference and StringHandle assume that heap indexes are limited to 24 bits.

[nguerrera]

@jbevain Thanks.

@tmat Ideas? This is actually a pretty fundamental architectural assumption -- that we have enough room in 32 bits to encode references to heap values like metadata tokens.

I'm looking at the bit twiddling around handle types and so forth, and I think I can squeeze more than 24 bits, but the full 32-bit range is just not possible without (a) making Handle and StringHandle wider or (b) allocating some sort of table on the side.

[tmat]

I see. Oops. #US heap indices have to be <= 2^24. The spec is not clear however on the limits for #Blob, #String and #GUID streams. It seems that they are allowed to be indexed by 32-bit index. Not sure if it has to be 29-bit actually, since greater numbers don't fit to compressed integer.

[tmat]

The library is compiled by C++/CLI, which seems to produce a lot of very long identifiers (encoding templates and such).

[tmat]

@nguerrera

After giving it more thought, I think we have 3 options:

1. Keep string, blob and guid handles 32-bit and virtualize handles with indices greater than 2^24.
   Whenever we are about to create a new handle with an index grater than 2^24 we assign it a new virtual handle and add an entry to a map {real index <-> virtual index}. We return a virtual string/blob/guid handle instead (highest bit set). When retrieving a string or byte[] for such handle we look up the map.

2. Disallow conversion to and from Handle.
   This conversion is not that useful anyways. I don't think there is a case in type system metadata where there is a need to return a union of these handles and other handles.
   We have two sub-options:
   a) Keep the handles 32bit, allow index up to 2^29, use 1 bit for virtual flag and 2 bits for kind. 2 bits are good enough since we have only 3 internal representations of string handle and 2 of namespace handles. I think limitation to 2^29 is reasonable since it's the max compressed integer. But one can in theory construct an assembly with one a very large string and one small string that starts at offset 2^30 or so, which we won't be able to represent. I don't think that's a real-world scenario though. Usually there will be many medium sized strings.
   b) Make the handles 40bit, allow index up to 2^32. More memory usage - often 4B more due to alignment.

I propose we go with [2a]. If in future we hit the 2^29 limit for some reason, we could use [1] to deal with larger indices and extend compressed integer encoding to handle greater numbers than 2^29 (Portable PDB use compressed integers to encode heap blobs, regular metadata wouldn't need such extended compressed int). Loosing convertibility to Handle is likely not gonna be an issue.

[1] makes things complicated. [2b] is less memory efficient for no practical benefit.

[jbevain]

@tmat

Heap indexes are never read from compressed integers, my understanding is that they can use the entire 32 bits when the heap is tagged as large.

Agreed with the mitigation, I don't see a semantic reason to allow conversion between a table handle and a string handle. That being said, an assembly somewhere is bound to use more than 29 bits for a heap index and someone is referencing it :)