New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple tag filters for ad-hoc queries w/HTTP endpoints (catalog, health) #1781

Closed
doertedev opened this Issue Mar 2, 2016 · 14 comments

Comments

Projects
None yet
9 participants
@doertedev

doertedev commented Mar 2, 2016

OHAI.

Am I under the correct assumption, that this means the tag filter for the HTTP API endpoint /v1/catalog/service/<service> only supports one tag filter? This would explain the problems that I'm having right now: I'm writing my services with tags for both the region and environment but filtering for both fails.

https://localhost:8500/v1/catalog/service/zookeeper?tag=region-us-east-1 5 results,
https://localhost:8500/v1/catalog/service/zookeeper?tag=region-us-east-1&tag=env-internal 5 results,
https://localhost:8500/v1/catalog/service/zookeeper?tag=env-internal 1 result,
https://localhost:8500/v1/catalog/service/zookeeper?tag=env-internal&tag=region-us-east-1 1 result.

Second get's ignored if I'm seeing this correctly.

Can we integrate multiple tag filters? Or accept a json array in the URl? Or do we already and I cannot seem to articulate my query properly? Or do you say "prepared query || gtfo"?

Thinking out loud:
When using prepared queries for all those, I'd either have to create a prepared query for all possible combinations of ["environment", "region", "service"] which would right now be 350 prepared queries for us r/n and we just started implementing consul. And another would be to create them session-bound and removing them immediately after sessionclose, which in one swing (one ansible discovery round-trip) would create overhead. I think. Haven't benched prepared queries yet (which is the reason I filed this here).

Thanks in advance,
a tableflipping german. Prost.

@doertedev

This comment has been minimized.

Show comment
Hide comment
@doertedev

doertedev Mar 4, 2016

Just found #294 ... So I guess its "prepared statements" then?

doertedev commented Mar 4, 2016

Just found #294 ... So I guess its "prepared statements" then?

@slackpad

This comment has been minimized.

Show comment
Hide comment
@slackpad

slackpad Mar 10, 2016

Contributor

Hi @doertedev sorry I think the idea of multiple tags for ad-hoc queries over HTTP endpoints like catalog are still worth considering, and got lost in #294.

Contributor

slackpad commented Mar 10, 2016

Hi @doertedev sorry I think the idea of multiple tags for ad-hoc queries over HTTP endpoints like catalog are still worth considering, and got lost in #294.

@slackpad slackpad changed the title from Multiple tag filters to Multiple tag filters for ad-hoc queries w/HTTP endpoints (catalog, health) Mar 10, 2016

@doertedev

This comment has been minimized.

Show comment
Hide comment
@doertedev

doertedev Mar 10, 2016

What do you think about JSON values? (e.g. ?tag=[eu-west-1,production]?)

doertedev commented Mar 10, 2016

What do you think about JSON values? (e.g. ?tag=[eu-west-1,production]?)

@slackpad

This comment has been minimized.

Show comment
Hide comment
@slackpad

slackpad Mar 11, 2016

Contributor

@doertedev I don't think I'd want to put JSON in there, the ?tag=&tag=... kind of syntax seems like the most natural.

Contributor

slackpad commented Mar 11, 2016

@doertedev I don't think I'd want to put JSON in there, the ?tag=&tag=... kind of syntax seems like the most natural.

@doertedev

This comment has been minimized.

Show comment
Hide comment
@doertedev

doertedev Mar 11, 2016

What about negation? That's why I thought about JSON.

doertedev commented Mar 11, 2016

What about negation? That's why I thought about JSON.

@slackpad

This comment has been minimized.

Show comment
Hide comment
@slackpad

slackpad Mar 11, 2016

Contributor

I'd still like to avoid JSON - we could using the prepared query syntax of !tag which would have a minimal amount of URL encoding, or we could do something like ?not-tag=.

Contributor

slackpad commented Mar 11, 2016

I'd still like to avoid JSON - we could using the prepared query syntax of !tag which would have a minimal amount of URL encoding, or we could do something like ?not-tag=.

@doertedev

This comment has been minimized.

Show comment
Hide comment
@doertedev

doertedev Mar 11, 2016

The only thing that really sounds wrong to me is using one URL parameter multiple times. This would maybe break third party software that constructs the URL. Anything else? Sure.

doertedev commented Mar 11, 2016

The only thing that really sounds wrong to me is using one URL parameter multiple times. This would maybe break third party software that constructs the URL. Anything else? Sure.

@leftyftw

This comment has been minimized.

Show comment
Hide comment
@leftyftw

leftyftw Sep 21, 2016

Isn't the "standard" array notation ?tag[]=value1&tag[]=value2&tag[]=value3. I believe that would suffice in this instance. Would love to see this ability.

leftyftw commented Sep 21, 2016

Isn't the "standard" array notation ?tag[]=value1&tag[]=value2&tag[]=value3. I believe that would suffice in this instance. Would love to see this ability.

@senthilkumar-kj

This comment has been minimized.

Show comment
Hide comment
@senthilkumar-kj

senthilkumar-kj Nov 26, 2016

@doertedev please see this comment - #294 (comment) and this stackoverflow answer - http://stackoverflow.com/a/24728298/205585

This is the most common way and for negation we could do "!" like it is done in prepared queries or nt-tag query as @slackpad suggested.

senthilkumar-kj commented Nov 26, 2016

@doertedev please see this comment - #294 (comment) and this stackoverflow answer - http://stackoverflow.com/a/24728298/205585

This is the most common way and for negation we could do "!" like it is done in prepared queries or nt-tag query as @slackpad suggested.

@ffledgling

This comment has been minimized.

Show comment
Hide comment
@ffledgling

ffledgling Jul 6, 2017

Bump. I was looking for this and the docs I found do not explicitly mention that this is not possible at the moment. Would be neat to have this on an ad-hoc basis :)

ffledgling commented Jul 6, 2017

Bump. I was looking for this and the docs I found do not explicitly mention that this is not possible at the moment. Would be neat to have this on an ad-hoc basis :)

@vanderhoofen

This comment has been minimized.

Show comment
Hide comment
@vanderhoofen

vanderhoofen Apr 9, 2018

It's 2018 now and the first thing my devs ask me is "how can I filter on multiple tags?" Prepared Queries create a bad user experience for such a lightweight request

vanderhoofen commented Apr 9, 2018

It's 2018 now and the first thing my devs ask me is "how can I filter on multiple tags?" Prepared Queries create a bad user experience for such a lightweight request

@KrolikROJER

This comment has been minimized.

Show comment
Hide comment
@KrolikROJER

KrolikROJER commented Jul 17, 2018

Any news?

@abhaychaware

This comment has been minimized.

Show comment
Hide comment
@abhaychaware

abhaychaware Jul 30, 2018

Any update/example/documentation/roadmap on this ?

abhaychaware commented Jul 30, 2018

Any update/example/documentation/roadmap on this ?

adilyse added a commit that referenced this issue Sep 25, 2018

Support multiple tags for health and catalog api endpoints
Fixes #1781.

Adds a `ServiceTags` field to the ServiceSpecificRequest to support
multiple tags, updates the filter logic in the catalog store, and
propagates these change through to the health and catalog endpoints.

Note: Leaves `ServiceTag` in the struct, since it is being used as
part of the DNS lookup, which in turn uses the health check.

adilyse added a commit that referenced this issue Sep 25, 2018

Support multiple tags for health and catalog api endpoints
Fixes #1781.

Adds a `ServiceTags` field to the ServiceSpecificRequest to support
multiple tags, updates the filter logic in the catalog store, and
propagates these change through to the health and catalog endpoints.

Note: Leaves `ServiceTag` in the struct, since it is being used as
part of the DNS lookup, which in turn uses the health check.

adilyse added a commit that referenced this issue Sep 25, 2018

Support multiple tags for health and catalog api endpoints
Fixes #1781.

Adds a `ServiceTags` field to the ServiceSpecificRequest to support
multiple tags, updates the filter logic in the catalog store, and
propagates these change through to the health and catalog endpoints.

Note: Leaves `ServiceTag` in the struct, since it is being used as
part of the DNS lookup, which in turn uses the health check.

adilyse added a commit that referenced this issue Sep 26, 2018

Support multiple tags for health and catalog api endpoints
Fixes #1781.

Adds a `ServiceTags` field to the ServiceSpecificRequest to support
multiple tags, updates the filter logic in the catalog store, and
propagates these change through to the health and catalog endpoints.

Note: Leaves `ServiceTag` in the struct, since it is being used as
part of the DNS lookup, which in turn uses the health check.

adilyse added a commit that referenced this issue Sep 26, 2018

Support multiple tags for health and catalog api endpoints
Fixes #1781.

Adds a `ServiceTags` field to the ServiceSpecificRequest to support
multiple tags, updates the filter logic in the catalog store, and
propagates these change through to the health and catalog endpoints.

Note: Leaves `ServiceTag` in the struct, since it is being used as
part of the DNS lookup, which in turn uses the health check.

adilyse added a commit that referenced this issue Sep 26, 2018

Support multiple tags for health and catalog api endpoints
Fixes #1781.

Adds a `ServiceTags` field to the ServiceSpecificRequest to support
multiple tags, updates the filter logic in the catalog store, and
propagates these change through to the health and catalog endpoints.

Note: Leaves `ServiceTag` in the struct, since it is being used as
part of the DNS lookup, which in turn uses the health check.

adilyse added a commit that referenced this issue Oct 10, 2018

Support multiple tags for health and catalog api endpoints
Fixes #1781.

Adds a `ServiceTags` field to the ServiceSpecificRequest to support
multiple tags, updates the filter logic in the catalog store, and
propagates these change through to the health and catalog endpoints.

Note: Leaves `ServiceTag` in the struct, since it is being used as
part of the DNS lookup, which in turn uses the health check.

@banks banks closed this in #4717 Oct 11, 2018

banks added a commit that referenced this issue Oct 11, 2018

Support multiple tags for health and catalog http api endpoints (#4717)
* Support multiple tags for health and catalog api endpoints

Fixes #1781.

Adds a `ServiceTags` field to the ServiceSpecificRequest to support
multiple tags, updates the filter logic in the catalog store, and
propagates these change through to the health and catalog endpoints.

Note: Leaves `ServiceTag` in the struct, since it is being used as
part of the DNS lookup, which in turn uses the health check.

* Update the api package to support multiple tags

Includes additional tests.

* Update new tests to use the `require` library

* Update HealthConnect check after a bad merge
@adilyse

This comment has been minimized.

Show comment
Hide comment
@adilyse

adilyse Oct 12, 2018

Contributor

This is now available in v1.3.0, released yesterday. Thanks for the feedback, which helps us make a better product!

Contributor

adilyse commented Oct 12, 2018

This is now available in v1.3.0, released yesterday. Thanks for the feedback, which helps us make a better product!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment