Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate bulk metadata functions #5596

Open
carl-mastrangelo opened this issue Apr 15, 2019 · 1 comment
Open

Investigate bulk metadata functions #5596

carl-mastrangelo opened this issue Apr 15, 2019 · 1 comment
Milestone

Comments

@carl-mastrangelo
Copy link
Contributor

Metadata provides limited public APIs for interaction, typically requiring a Metadata.Key to do any reads or writes. This is useful to an application, which may know the key name and marshaller ahead of time, but not so much for middle-ware which may need to scan for keys. As an example, a middle-ware that looks for metadata prefixes and builds a custom Key depending on the suffix of the key name has trouble with the existing Metadata API. It requires multiple iterations through the metadata, and up to O(n^2) ops when trying to extract values with duplicate key names.

Metadata is also mutable, and does not want to expose internals, as they may yet change. I am proposing some experimental APIs here to address the bulk metadata functions use case:

Option 1:

// For each value in the MD which matches the key prefix, return the parsed value  
<T> List<T> walk(String keyPrefix, Metadata.BinaryMarshaller<T> marshaller)  {}

Option 2:

<T> void walk(Function<? super String, Optional<Metadata.Key<T>> nameToKeyFn, Consumer<? super Object> valueConsumer)  {}

Option 3:

// For each value in the MD which matches the key prefix, return the parsed value  
void walk(String keyPrefix, Metadata.BinaryMarshaller<?> marshaller, BiConsumer<? super String, ? super Object)  {}

Option 4:

void walk(Function<? super String, Optional<Metadata.BinaryMarshaller<?>> nameToKeyFn, BiConsumer<? super String, ? super Object)  {}

The different options provide the equivalent of a map operation (we might also be able to provide the equivalent of a Java8 stream). Option 1 is the least flexible, but has the simplest API. Option 2 allows the caller to provide their own Key producer based on the key name, but the value consumer doesnt get to know the key name used. Option 3 still uses the same marshaller for each key, but allows the consumer to know the key name. Option 4 is a combination of 2 and 3, allowing both dynamic marshaller construction based on the key name, and allowing the consumer of the the produced values to know the keyname (perhaps for later caching?)

@carl-mastrangelo
Copy link
Contributor Author

cc @tsalomie @ejona86

@ejona86 ejona86 added this to the Unscheduled milestone Apr 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants