-
Notifications
You must be signed in to change notification settings - Fork 60
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
next() function #15
Comments
Not yet, I need to do some research into how precisely next works in puc lua first. I don't think |
Sure, thanks! I think it must just walk hash buckets in order, but I haven't actually looked myself yet! |
It has to support deletions as you iterate through (see Lua docs for |
Okay, so as suspected PUC-Lua just walks the hash buckets in order to find a next value. The guarantee about supporting deletions as you iterate through is because a rehash will never occur on deletion, which makes perfect sense. I briefly looked for a Rust HashMap implementation that exposes the internal bucket vector and came up empty. This is probably something I should have been thinking about earlier, tbh. There are a few things we could do:
I guess I'm leaning towards 1 as a solution unless somebody knows of a HashMap with the proper API exposed. What does everyone else think? |
Lua says that deletions and modifications are okay, but insertions are not okay whilst iterating, but it doesn't enforce that at the language level like a Rust implementation would. I think a Rust HashMap would prefer to have things much better defined than that, and so adding functionality to support Lua's rather looser guarantees would not be accepted. (For example, a "give me the next key after this key" call would not have an well-defined meaning for "next key" in the presence of insertions, and that seems much too loose for Rust.) So as you say, option (1) seems like what you'll end up with in order to support Lua's behaviour, unfortunately. It doesn't look straightforward to disrupt the order in option (4) to stop people counting on it. Also, what are you doing about the array part of the table? This is a whole other weird area of Lua. I have code in C which counts on the What gets into the array part is also not terribly sane or well-defined (except by looking at the code or trial and error), e.g. |
Just thinking some more about this. If you don't want to store the array-part of the table separate to the hash, you could just simulate the array-length |
Modification is not really a thing we need to worry about, since nothing that is mutable would change the hash of the value (strings are immutable, you can mutate a table but when used as a key only a table's address is used, same for userdata / closures). It seems totally reasonable, if extremely specific and weird, to ask what the next value after a given value is in the hash buckets. There's no actually concurrent mutation here, in fact in luster both operations would even take place in separate RefCell locks (the inside of Table is stored inside a GcCell after all). It of course may change after an insert due to a rehash, but that's what happens in PUC-Lua too so this is fine? I don't think this really has anything to do with Rust, rather than just being a really odd thing to need from a HashMap implementation. You can even already get the information if you're willing to walk the
I already store the array part of tables separate to the hash part and it works very similarly to what Lua does, and yeah you're right it's not trivial. That actually was one of the first things I did. Edit: I thought a lot about Lua tables at the beginning of the project to match the semantics, especially around sizing the array part vs the hash part, but I should have thought more about iteration at that time as well. Actually I think I did think about it but any answer that involved writing my own HashMap seemed too unreasonable and I guess I was hoping a better solution would magically present itself. |
Yes, agreed. As you say it's "extremely specific and weird", and not really a Rust-like interface, but it's not a safety problem for someone to expose that interface on their HashMap. |
Deletions wouldn't work with an |
That's a good point, I didn't think about that. PUC-Lua doesn't remove keys with nil values from the table until an insert takes place, but since we wouldn't want to mimic that behavior, this would be a problem. That's fine, I thought that was a poor solution anyway. It looks like implementing our own HashMap might be the way to go :/ |
Mostly for fun, and because I thought it'd be easier than understanding other hash maps well enough to modify, I've written a very naive hash map which has the required By naive I mean it's the simplest possible hash map I could think of, so:
I implemented enough to replace |
Ahhhh, you beat me to it, nice work! I'm not an expert at hash map implementations at all, but I was thinking of just porting one that I had written previously in C++ to Rust, after checking The hash map you wrote is not that far off from the one I was going to port, except:
This feels deeply wrong, just copying an existing HashMap implementation, but honestly I think the best way to handle this is probably to do neither of these things and just minimally copy the one from the Rust stdlib... depending on how complex that is. I'm not super sure yet, if a minimal subset of I think maybe looking closely at |
Some comments: But I agree starting from a production |
Oh okay that makes more sense. I still think we should look at how complex the original std::collections::HashMap is and just copy a minimal subset of its implementation, for now. I'm guessing that it's much simpler than hashbrown's one, and it's extremely well tested and is of respectable performance. Later, we can either update it to a faster implementation or beg somebody who maintains an external HashMap for the right APIs. If hashbrown's HashMap is relatively simple we could also copy it instead? I've been wracking my brain trying to figure out a way to implement this that is not really awful and doesn't require maintaining our own HashMap implementation, but I still haven't come up with anything. |
Maybe IndexMap + delay removes until the next insert is the right solution. IndexMap is respectably fast, at least.. I think PUC-Lua plays some garbage collector games with dead indexes though so that they're less bad. Ugh, there are no good solutions here other than maintaining a whole bespoke HashMap :( |
I've had a bit more time to play... This time I've gone for adding a Based on that I've implemented the Lua |
On a related note, how do you feel about including some of the stdlib implemented in Lua? |
Well, in the ridiculous amount of time I paused this project, hashbrown has published a raw table API that solves all of my problems 😆 Now solving this is (relatively) straightforward! |
aaaaand done, by 12fdd6a |
I've started to have a play with
luster
to see how it might fit into another project.I was wondering about adding some of the
table
functions, but notice that the next function isn't implemented yet.Before I start poking around in
Table
, do you have thoughts on an approach? Replacing theFxHashMap
with something like IndexMap might work (thatIndexMap
doesn't quite do everything needed as far as I can tell).Thanks,
Chris
The text was updated successfully, but these errors were encountered: