Skip to content

Loading…

xpath-like selectors on endpoints #243

Closed
sartak opened this Issue · 8 comments

3 participants

@sartak
<sartak> so, given http://api.metacpan.org/module/Text::Xslate
<sartak> one thing that might be nice is pulling out just a single element in that JSON object
<sartak> if, say, http://api.metacpan.org/module/Text::Xslate/distribution would return "Text-Xslate",
<sartak> and http://api.metacpan.org/changes/Text-Xslate/content returns the Changes file
<sartak> I wouldn't need to depend on JSON... :)
<sartak> not a big worry, especially if the biggest reason is to get rid of a dep on JSON, but it'd make my API consumption just a little bit nicer
<rwstauner> sartak: I like that idea
 * rafl too
<rwstauner> care to make a gh-issue on cpan-api?
<sartak> sure
<rafl> how about dpath expressions or some such?
<sartak> yeah, or http://goessner.net/articles/JsonPath/ or anything
<sartak> anyway, yeah, should definitely not be limited to just pulling out strings at the top level of the resource
<sartak> could even just be xpath :)
@sartak
<haarg> given that the data isn't in xmp, xpath sounds like a bad idea
<haarg> you'd have to map it into xml in some way, which is basically the exact problem dpath tries to solve
<sartak> ah, I see
<sartak> xpath has the minor benefit of being widely known, but anything in this space is fine by me
<haarg> xpath/dpath's leading slashes would be rather unfortunate to try to stick into a url path
<sartak> api.metacpan.org/module/Text::Xslate?select=//distribution or whatever
<sartak> or %2F%2Fdistribution (ew)
<haarg> yeah, going that route would make much more sense to me if you wanted the extra flexibility
<sartak> or hell, better to name what syntax you're using so metacpan could add another without friction
<sartak> (better xpath= dpath= jsonpath= than select=)
@monken
MetaCPAN member

https://api.metacpan.org/module/Task::Catalyst?fields=release,author
is that good enough for you? It proxies the fields query parameter to ElasticSearch. That makes it much more efficient since only the requested data is retrieved from ES.

@rwstauner
MetaCPAN member

That would certainly cover simple cases. Nice work!

If we were to add support for something more robust we'd have to try to guard against any sort of excessive evaluation (if any of the expressions allow for havoc like that). I'm speaking generally as I don't know much of what you can do with "${var}path" but i'd hate to bring the server down b/c somebody put some sort of loop or hideous ../find/../find/../find garbage or something.

This less-than-helpful note brought to you Obvious Man™.

I guess it would basically be a matter or reading the docs and determining if things like that were possible (particularly within the maximum possible length of a querystring).

@sartak

It's certainly helpful, thank you! I will adjust my code to use this feature since it's a clear improvement.

However my actual use case is that I wanted to avoid having to parse JSON. :) Ideally pulling out a single field like https://api.metacpan.org/module/Task::Catalyst?fields=distribution (or some different syntax, like ?return=distribution) could just return the bare Task-Catalyst (without quotes too).

@rwstauner
MetaCPAN member

I stumbled upon json-pointer... not sure how well used it is.

@sartak

This feature works on the /module endpoint so I'm switching Pod::Cpandoc to use it (yay)

However it does not work on the /changes endpoint: http://api.metacpan.org/changes/Text-Xslate?fields=content returns everything.

@rwstauner
MetaCPAN member

I think I have this fixed, just need to write some tests.

@rwstauner
MetaCPAN member

oops, never came back to close this.

@rwstauner rwstauner closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.