New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[API Proposal]: CollectionsMarshal method that enumerates dictionary by reference to avoid unnecessary hash calculation. #58333
Comments
Tagging subscribers to this area: @eiriktsarpalis Issue DetailsBackground and motivation
Consider a very common use case of This issue mainly happens to API Proposalnamespace System.Runtime.InteropServices
{
public static class CollectionsMarshal
{
public static DictionaryValueRefEnumerable<TKey, TValue> EnumerateValuesByRef<TKey, TValue>(Dictionary<TKey, TValue> dictionary) => new(dictionary);
public struct DictionaryValueRefEnumerable<TKey, TValue>
{
internal DictionaryValueRefEnumerable(Dictionary<TKey, TValue> dictionary) { ... }
DictionaryValueRefEnumerable<TKey, TValue> GetEnumerable() => this;
public bool MoveNext() { ... }
public ref TValue Current
{
get { ... }
}
}
}
} API Usagestruct Data
{
public int SomeValue;
//Other fields.
}
Dictionary<int, Data> dictionary = new();
//...
foreach (ref var data : CollectionsMarshal.EnumerateValuesByRef(dictionary))
{
if (data.SomeValue != -1)
{
data.SomeValue += 1;
}
} RisksSome design considerations:
public struct ImmutableKeyKeyValuePair<TKey, TValue>
{
public TKey Key { get; }
public TValue Value;
} by converting from the corresponding part of the
|
This would probably benefit from ref fields being a first-class citizen of the runtime & language (see dotnet/csharplang#1147). It's certainly possible to have an implementation without this, but it's likely to have suboptimal performance. public readonly ref struct KeyValuePairByRef<TKey, TValue>
{
private readonly ref Entry _entry;
public ref readonly TKey Key => ref _entry.Key;
public ref TValue Value => ref _entry.Value;
}
public struct TheEnumerator
{
private readonly Entry[] _entries;
private int _index;
public bool MoveNext() { /* checks mimicking current enumerator checks */ }
public KeyValuePairByRef<TKey, TValue> Current => new (ref _entries[_index]); // + whatever other checks need take place
} |
I have another proposal on the use of
EDIT Not quite possible, at least directly given the ref. It would need to change the single linked list to double linked list, which will make huge regression. But it would really be helpful if we could remove while enumerating. I think if the enumerator records more states, it will be possible. For example, it can record 2 pointers, one for scanning for linked list heads (through |
Since it's inside the corelib, maybe |
Perhaps, but we've historically been hesitant to use it for scenarios beyond simply |
I have update the proposal with some benchmark results. Both methods have some improvements in enumerating (the ratio strongly depends on how fast the hash can be calculated). The second method, when used with removing, is also better than the "standard workaround" of creating another list recording keys, because it's not only faster, but also avoids allocation of the list. |
The issue was changed from "allow iterating by references" to "allow removal while iterating", which significantly changes the scope & direction of this work. I think you'll have more success proposing an API which tries to split the two concerns. For example, you could consider a |
|
Yes, but it will need another hash table look up, as I mentioned in the background section. |
I'm saying, the background section states "or Dictionary<,>, modifying the content will invalidates the enumerator, and users are forced to temporarily record keys to remove elsewhere, and remove them after enumeration finishes" which is not correct. This is valid code: var d = new Dictionary<int, int> { { 1, 1 }, { 2, 2 }, { 3, 3 } };
foreach (var pair in d)
{
d.Remove(pair.Key);
} |
I understand. The reason I think it's probably better to mention it here is because if one method can cover both, it will be easier for user to decide which one to use. If there are two proposals, each add one enumeration method, the API will look much more complcated. I think this issue might be an initial attempt that illustrates the problem. If you think it's better having an enumerator first, without allowing the removal, I (or anyone else) can open a separate issue to track it. In any case, I still think both proposals are about avoiding repeated hash calculation and look-up while enumerating, and can be seen as one unified problem and solved together. |
You are right. I will update the wording and probably redo some benchmarks. |
@acaly can you please update the issue to add example scenarios which this could solve? I personally never needed such functionality but YMMV |
@krwq Thanks for the reply. I wrote a very brief description on the scenario in the proposal.
I encountered this problem when I was implementing a physics engine in C#. I stored the active interaction data between entities in a dictionary, with |
I have a similar high performance scenario where the dictionary values (int) all have to be divided by 2 (some kind of decay operation). How about changing dictionarys' Existing code would still be valid, and ref assignments would give a change option. |
Existing source code yes -- but not existing binaries. |
That's true ... but thanks for the quick response! |
Background and motivation
CollectionsMarshal
class has been introduced to expose the collection class's underlying data by reference in order to reduce unnecessary copy of large struct values when accessing only a small part of the value. ForDictionary<TKey, TValue>
, there are methods that manipulate individual key-value pairs. However, these methods mainly requires accessing from know keys. When the user wants to do batch actions to many elements in the dictionary, enumerating keys and then doing a hash table look up would not be the most efficient way.Consider a very common use case of
Dictionary<,>
class: the user wants to use it like an in-memory database table, to store large struct values, each containing the data of one row, and all values are indexed by a primary key, which is itself part of the row data but immutable. When reading and writing data from one row with known primary key, existing methods likeGetValueRefOrNullRef
is sufficient. But if the user wants to read all rows, doing some work to each of them (or to some of them that meets specific condition), it would be better if there is a way to enumerate the dictionary returningref
to each value. This issue mainly happens toDictionary<,>
becauseList<T>
already exposes itsSpan<T>
, which can be used to enumerate by afor (int i = 0; ...
loop to get the references.Another common use pattern of dictionary is to remove specific items while enumerating. For
List<T>
, this can be done by a sequential enumeration.Dictionary<,>
class already allows removing items while enumerating, but to find the removed item, the dictionary will need additional hash table look-ups. This can be avoided if we provide an enumerator that itself supports removing.In summary, using a new enumerator, we should able to achieve:
ref TValue
without additional hash calculation and look-ups.To maximize performance, these two objectives do not necessarily share the same implementation. It's possibly better to address the separately, as mentioned in the discussion below. However, here I will still provide an alternatives that aims at solving both together, because the two objectives can actually be seen as avoiding unnecessary hash calculation and look-ups during enumeration.
API Proposal
Method 1: sequentially enumerate each entry, returning reference to the entry struct.
This will maximize performance, but will not allow removing while enumerating.
Method 2: enumerate each bucket, returning a struct that holds the internal state of enumerating (the current bucket and the last entry index).
This will sacrifice some performance, but it allows a
Remove
method to be added.API Usage
Method 1:
Method 2:
Possible implementations and performances
I have done some preliminary investigation on the implementation and the performance. The conclusion:
Enumerating:
dictionary.Keys
followed byCollectionsMarshal.GetValueRefOrNullRef
.(int, int, int)
keys, 6x for string keys).Deleting while enumerating:
dictionary.Keys
followed byCollectionsMarshal.GetValueRefOrNullRef
. Use aList<TKey>
to record keys to be removed, and after enumeration of the dictionary finishes, enumerate this list to remove them.(int, int, int)
keys, and 2x for string keys).Since I am not an expert in this area, I think there should still be much space for optimization.
Benchmark results
int->int, 10 items
int->int, 1000 items
int->big struct, 10 items
int->big struct, 1000 items
(int, int, int)->int, 10 items
(int, int, int)->int, 1000 items
(int, int, int)->big struct, 10 items
(int, int, int)->big struct, 1000 items
string->int, 10 items
string->int, 1000 items
string->big struct, 10 items
string->big struct, 1000 items
Implementation of Method 1 used in the test
Implementation of Method 2 used in the test
Alternative designs
Linq-like extension method:
As mentioned below, the same effect can be achieved by providing a
RemoveWhere
method that passesTKey
andref TValue
to a delegate:such that the delegate will be able to access the value by reference and specify whether each item should be removed.
Advantages: 1) seems more natrual to C#; 2) the dictionary has chance to do rehashing after finishes enumerating.
Disadvantages: 1) the runtime is currently unable to fully inline delegate calls; 2) if the delegate need to access other variables, creating the closure requires additional allocation at each call.
Risks
Some design considerations:
CollectionsMarshal
methods, but this is by design. User should make sure the use is valid, e.g., by guaranteeing no modification is made to the collection during enumerating.IEnumerable<T>
interface, because the T here is a by-ref type. This should be fine since usually converting to interfaces causes boxing and GC allocation, which is against high-performance requirements.The text was updated successfully, but these errors were encountered: