Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a random access ets:match #7561

Open
lafirest opened this issue Aug 14, 2023 · 3 comments
Open

Add a random access ets:match #7561

lafirest opened this issue Aug 14, 2023 · 3 comments
Assignees
Labels
enhancement team:VM Assigned to OTP team VM

Comments

@lafirest
Copy link

lafirest commented Aug 14, 2023

Is your feature request related to a problem? Please describe.
We are heavy dependence on the order set and the ets:next to find a beginning key and then apply a function both on it and the following, but the est:next won't return an iterator, this means the search always starting from the root, the est:match_xxx can use continuation() as an iterator, but unfortunately it without random access support.

I think it would be better to support iterating with a function and from a random access point.

example(Key) ->
    case ets:next(Table, Key) of
        '$end_of_table' ->
            finished;
        Next ->
            case ets:lookup(Table, Next) of
                [Value] ->
                    AFunction(Value),
                    example(Next);
                _ ->
                    finished
            end
    end.

Describe the solution you'd like

  1. a random access match, like ets:match_xxx(Table, FirstKey, Pattern)
  2. iterate with function, like ets:iterate_with(Table, Key, Function)
@IngelaAndin IngelaAndin added the team:VM Assigned to OTP team VM label Aug 14, 2023
@sverker
Copy link
Contributor

sverker commented Aug 14, 2023

There already is an optimization for next on ordered_set that will save an internal "iterator" for the table. That is, if you do repeated next calls (like your example) then it will not do repeated traversals from the root of the tree, but rather use the internal iterator saved from the previous next call.
For the the optimization to kick in you have to

  • not use table option write_concurrency.
  • not update the table in a way that changes the node layout of the tree. This means all updates except update_element and update_counter that only changes small integers, atoms, pid or ports (immediate terms).
  • only do sequential calls to next with the previous returned key as argument. For example, two processes doing traversal with next on the same table at the same time may destroy the iterator for each other.

@lafirest
Copy link
Author

There already is an optimization for next on ordered_set that will save an internal "iterator" for the table. That is, if you do repeated next calls (like your example) then it will not do repeated traversals from the root of the tree, but rather use the internal iterator saved from the previous next call. For the the optimization to kick in you have to

* not use table option `write_concurrency`.

* not update the table in a way that changes the node layout of the tree. This means all updates except `update_element` and `update_counter` that only changes small integers, atoms, pid or ports (immediate terms).

* only do sequential calls to `next` with the previous returned key as argument. For example, two processes doing traversal with `next` on the same table at the same time may destroy the iterator for each other.

Hi , @sverker
Thanks for your reply. Although you introduced an internal logic for reusing the last iterator for the next call, we can see it is limited and we can't suppose the data set is static when iterating the keys.
I have a skim read of the C code of the order set, I think it is possible to add a new function for this, a simple way is to expose the continuation() structure, so users can construct the continuation themself.

@sverker
Copy link
Contributor

sverker commented Aug 17, 2023

This is similar feature request as #7399.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement team:VM Assigned to OTP team VM
Projects
None yet
Development

No branches or pull requests

3 participants