curl no longer accepts a space in a url, but "dict" protocol doesn't allow %20

[dekerser]

So new curl versions don't accept a space, but do support the dict protocol:

$ curl --version | cut -d' ' -f 1-2
curl 7.87.0
Release-Date: 2022-12-21
Protocols: dict
Features: alt-svc
 
$ curl 'DICT://dict.org/SHOW DB'
curl: (3) URL using bad/illegal format or missing URL

But the DICT protocol requires the space, and doesn't allow a "%20"

$ curl 'DICT://dict.org/SHOW%20DB'
220 dict.dict.org dictd 1.12.1/rf on Linux 4.19.0-10-amd64 <auth.mime> <163438761.16210.1673796575@dict.dict.org>
250 ok
500 unknown command
221 bye [d/m/c = 0/0/0; 0.000r 0.000u 0.000s]

So this works for older versions:

$ curl --version | cut -d' ' -f 1-2
curl 7.74.0
Release-Date: 2020-12-09
Protocols: dict
Features: alt-svc

$ /bin/curl "dict://dict.org/SHOW DB" 2>/dev/null | head -n 5
220 dict.dict.org dictd 1.12.1/rf on Linux 4.19.0-10-amd64 <auth.mime> <163441100.19811.1673797845@dict.dict.org>
250 ok
110 166 databases present
gcide "The Collaborative International Dictionary of English v.0.48"
wn "WordNet (r) 3.0 (2006)"

Shouldn't there be an exception for the DICT protocol?

[dfandrich]

curl deals in URLs and URLs don't allow spaces, so if they're allowed, spaces must be % encoded. However, RFC2229 doesn't define a URL format for sending the SHOW DB command. If it worked in curl in the past, it was a happy accident.

[dekerser]

Don't agree here, and I was afraid for this answer:

• Curl implements the "dict" protocol.
• So dict is a separate protocol, the RFC says "SHOW DB", so this is what it should be no encoding.
• You can easily skip the space test if the protocol is "dict"

[dfandrich]

What part don't you agree with? The RFC's lack of specification of SHOW DB in URL format? Or curl's strict adherence to the RFC? It sounds like you're asking that curl support an undocumented URL specification, which is fine, but you need to be clear what you're asking. The curl project doesn't generally create its own URL formats, although it's been forced to in a few cases in the past. How does other software expose this other functionality? Is there a de facto extended URL format that other dict programs agree on in handling this feature?

[dekerser]

What I meant was that the dict protocol is supported by curl.
The "SHOW DB" command is, and many others commands are client side commands according to the RFC.
Indeed command approach seems more like a sendmail command exchange, and is called a "command pipeline".
So by enforcing the "space" as "%20" curl no longer allows a large part of the dict commands. So this means curl no longer supports the full dict protocol. You should mention this in your documentation and show in the --version output that dict protocol is only partial. Or you can make an exception and allow the subset of commands in RFC2229.

Mind you, there is a undocumented feature which allows the commands (which I didn't know before). Which is to replace "<space>" with ":" e.g.:

 $ curl dict://dict.org/SHOW:DB 2>/dev/null | head -n 5
 220 dict.dict.org dictd 1.12.1/rf on Linux 4.19.0-10-amd64 <auth.mime> <163757157.16019.1673978319@dict.dict.org>
 250 ok
 110 166 databases present
 gcide "The Collaborative International Dictionary of English v.0.48"
 wn "WordNet (r) 3.0 (2006)" 

For me this workaround is good enough. But for 'curl' I think you should choose between the above two options.
If you want to ignore this issue (third option), at least the workaround is documented 🙂. If you choose for the third option closing this issue is fine by me.

[dfandrich]

And I'm saying that the fact that it used to work was likely an accident. It's probably worth supporting other commands, but some thought needs to be given to how. Allowing -X (or possibly -Q) would be a cleaner way of supporting other commands that's more in the spirit of how curl operates.

[rsbeckerca]

Perhaps it might make sense to have curl have an option to translate %20 to a in the URL. So something along the lines of:

curl --space-url "dict://dict.org/SHOW%20DB"

would result in a URL "dict://dict.org/SHOW DB" internally, assuming the internal URL parse won't have an issue with that.

[dfandrich]

A new option really isn't necessary. If we decide to preserve the old behaviour, then curl can just decode the URL path itself before sending it to the server. It's just that this isn't a valid dict URL so all bets are off in how it's handled.

[rsbeckerca]

A new option really isn't necessary. If we decide to preserve the old behaviour, then curl can just decode the URL path itself before sending it to the server. It's just that this isn't a valid dict URL so all bets are off in how it's handled.

I guess it's too big a deal to ask all of the Dict server project to fix their URLs to conform to modern URL rules. :(

[bagder]

The fact that curl doesn't URL decode the path when using it for DICT looks like a bug to me.

[dfandrich]

I was thinking about this and, from a curl perspective doing this with curl -X 'show db' dict://dict.org/ is probably the cleanest. -X is there to modify the request that would normally be sent which is basically what's desired here. But, are you saying that people are handing out dict urls in the wild like dict://dict.org/show db?

[rsbeckerca]

I was thinking about this and, from a curl perspective doing this with curl -X 'show db' dict://dict.org/ is probably the cleanest. -X is there to modify the request that would normally be sent which is basically what's desired here. But, are you saying that people are handing out dict urls in the wild like dict://dict.org/show db?

I personally wonder how many software products could handle the embedded space cleanly in a URL. Probably nothing in the Office suite or browsers.

[bagder]

From my view it is not actually a question specifically about space or %20, but about treating URL encoded paths correctly in general for DICT:

As described above, this command line and URL work: curl 'DICT://dict.org/SHOW:DB'

As per how URLs works, we should then be able to also pass this with parts of the path URL-encoded, like if we encode the colon as %3a: curl 'DICT://dict.org/SHOW%3aDB'. This does not work. Replace the S with %53:

curl 'DICT://dict.org/%53HOW:DB' - does not work either. I say this is a bug.