Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

web plugin: support path queries #3567

Merged
merged 3 commits into from Apr 27, 2020
Merged

web plugin: support path queries #3567

merged 3 commits into from Apr 27, 2020

Conversation

nmeum
Copy link
Contributor

@nmeum nmeum commented Apr 25, 2020

The QueryConverter was added without much further explanation in commit
004e9a8. Unfortunately, the query
converter breaks path queries as flask cannot distinguish between an
URL encoded and a plain '/' character during routing [0]. Additionally,
I don't get why the QueryConverter is needed in the first place as ','
characters can just be used in the URL directly. For example:
/item/query/foo,bar. As conversions of slashes into comma characters
wasn't documented anyhow removing this code shouldn't cause much harm.

Fixes #3566

@sampsyo
Copy link
Member

sampsyo commented Apr 25, 2020

Seems like a good idea overall, but let's do some due diligence to make sure this won't break anything obvious (because it's a breaking change that could cause problems for clients that relied on being able to query with slashes).

Unfortunately, a little searching has already turned up one such example: mopidy-beets is using this endpoint for all its music searches.
https://github.com/mopidy/mopidy-beets/blob/1af80f230da411eb9e203ffec41c1b5adcc2ac1c/mopidy_beets/client.py#L184

Maybe we need to think creatively here about how to support this kind of query without breaking stuff like that? A new path query endpoint, for example, or a separate style that allows arbitrary query strings?

@nmeum
Copy link
Contributor Author

nmeum commented Apr 25, 2020

Maybe we need to think creatively here about how to support this kind of query without breaking stuff like that?

One "creative" quick-fix solution I was considering is using windows path separators, i.e. backslash, in the query URL. On Unix systems the web plugin can convert these back to Unix path separators, i.e. slash, before performing the actual query. Obviously a hacky solution but maybe a solution which introduces less breakage?

@sampsyo
Copy link
Member

sampsyo commented Apr 25, 2020

Yeah, that's a reasonable idea! One other option would be to optionally support an HTTP query string version of the query endpoint. That would let item/query?q=path:/foo/bar work as an alternative way to specify the query string directly. It's also somewhat aesthetically pleasing because it reflects that queries can truly be arbitrary strings…

@nmeum
Copy link
Contributor Author

nmeum commented Apr 26, 2020

Yeah, that's a reasonable idea!

Implemented this idea, I am not familiar with flask and werkzeug so not sure if the implementation is entirely correct as is. If you like the proposed implementation I would also add some documentation for both the / → , and the \ → os.sep behaviour to the web plugin as part of this PR. So let me know what you think.

@nmeum nmeum changed the title web plugin: remove QueryConverter web plugin: support path queries Apr 26, 2020
Copy link
Member

@sampsyo sampsyo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems about right to me! Indeed, would you mind adding that documentation?

beetsplug/web/__init__.py Outdated Show resolved Hide resolved
beetsplug/web/__init__.py Outdated Show resolved Hide resolved
@nmeum
Copy link
Contributor Author

nmeum commented Apr 26, 2020

Adjusted the code for list comprehensions, didn't get around to testing this version yet though. Also added some documentation for the query end point of the web plugin.

nmeum added a commit to nmeum/mmp that referenced this pull request Apr 26, 2020
Without this change the web plugin does not support path queries as
slashes are currently used for joining keywords in the QueryConverter.
Moreover, flask cannot distinguish between an URL encoded and a plain
'/' character during routing [0]. To work around this issue without
introducing a breaking change (i.e. removing the QueryConverter) use the
backslash character for path queries and convert it later on.

Fixes #3566

[0]: pallets/flask#900
@nmeum
Copy link
Contributor Author

nmeum commented Apr 27, 2020

Adjusted the code for list comprehensions, didn't get around to testing this version yet though.

Finally gotten around to testing this again. Found a bug while at it, seems to work now.

@sampsyo sampsyo merged commit 39e6d21 into beetbox:master Apr 27, 2020
sampsyo added a commit that referenced this pull request Apr 27, 2020
sampsyo added a commit that referenced this pull request Apr 27, 2020
@sampsyo
Copy link
Member

sampsyo commented Apr 27, 2020

Looks great; thanks!! Merged with a few tweaks to the wording.

GrahamCobb added a commit to GrahamCobb/beets that referenced this pull request Mar 8, 2021
As discussed in bug beetbox#3867, backslash replacement in query strings is a bit of a
hack but it is useful (see beetbox#3566 and beetbox#3567 for more discussion). However,
it breaks many regular expressions so this patch stops the replacement if the
query term contains '::', indicating it is a regex match.

This commit fixes beetbox#3867.

Signed-off-by: Graham R. Cobb <g+beets@cobb.uk.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

web plugin: Can't do path queries
2 participants