Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ETS select that can replace or remove while also returning a result #8115

Open
tsloughter opened this issue Feb 14, 2024 · 1 comment
Open
Assignees
Labels
enhancement stalled waiting for input by the Erlang/OTP team team:VM Assigned to OTP team VM
Milestone

Comments

@tsloughter
Copy link
Contributor

Is your feature request related to a problem? Please describe.

Problem is the need to select with removal or replacement like select_delete and select_replace while also returning a result.

There are also times I can know all the keys but since take/2 doesn't take a list of keys I'd have to do a separate take for every key.

Describe the solution you'd like

In metric collection of OpenTelemetry there is need to get the current value of an ETS object while updating or removing the object. The reasons for not wanting to do this in two steps is both performance and at times because it would mean lost writes (like if you want to select an object and reset it to an initial state

For deletion I'd like either a select_take function that returns a list of all the objects removed or, as described on the Erlang forum, an addition to matchspec like {delete}, https://erlangforums.com/t/ets-table-select-take/2758/8

Another nicety would be if take/2 took a list of keys. In Otel support for exemplars it needs to keep a count of the the number of seen metrics since the last collection of the exemplars as well as each seen exemplars. If it has seen more exemplars than it is configured to store it selects a random exemplar to replace. So There are 2 types of keys in that table, the key for how many have been seen and a key that is the other key plus a bucket position, {Key, Pos}.

Since the max number of buckets is known we could generate each {Key, Pos} and if take/2 took a list do, ets:take(ExemplarTab, [Key | BucketKeys]) to get them all removed in one step and (assuming implementation of take/2 is) atomically so no writes are lost.

This could also be done with select_take if it existed, but thought I'd mention the other case where each key is actually known.

Additional context and alternatives

I can give more detail if desired. Right now I've done an alternative of using a generation counter in the key for metrics. This generation is a counter stored in a persistent term. This generation is one for every metric exporter, so almost always just 1, thus just 1 term and not an issue. The generation is incremented on each collection so that any write after collection has started goes to a new key for each metric.

In the case of the exemplars it simply accepts the potential of lost writes:

collect(ExemplarsTab, Key, _State) ->
    Exemplars = ets:match(ExemplarsTab, {{Key, '_'}, '$1'}),

    _ = ets:select_delete(ExemplarsTab, [{{Key, '_'}, [], [true]},
                                         {{{Key, '_'}, '_'}, [], [true]}]),

    Exemplars.
@IngelaAndin IngelaAndin added the team:VM Assigned to OTP team VM label Feb 19, 2024
@jhogberg jhogberg added the stalled waiting for input by the Erlang/OTP team label Feb 19, 2024
@bjorng bjorng added this to the OTP-28.0 milestone Feb 20, 2024
@bjorng
Copy link
Contributor

bjorng commented Feb 20, 2024

After the first release candidate, we generally focus on bug fixes and polishing of features already included or planned for the release. To ensure that Erlang/OTP 27 will be as good as it possibly can be, we need to minimize the time we spend on things not to be included in the release. Therefore, we will not investigate/consider this feature request until after OTP 27 has been released. If we have not came back to it before September, feel free to remind us.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement stalled waiting for input by the Erlang/OTP team team:VM Assigned to OTP team VM
Projects
None yet
Development

No branches or pull requests

4 participants