HyperlinkedRelatedField exception on string with %20 instead of space

[tomkoy]

Use serializers.HyperlinkedRelatedField where defined lookup_field link to the field string causes an exception 'Invalid hyperlink - Object does not exist.' when the value of that field contains characters that are encoded in the url. eg space -> %20

Missing unquote when try get the value from the url

[xordoquy]

Looks legit as https://github.com/tomchristie/django-rest-framework/blob/master/rest_framework/generics.py#L96 doesn't unquote the data from url.

[tomchristie]

I'm rather surprised - I'd have fully expected that percent encoding is handled for you by Django's URL routing. A failing test case here might be a good first pass.

[rooterkyberian]

@xordoquy what you linked seems unrelated, as the error mentioned here is most likely raised from:

django-rest-framework/rest_framework/relations.py

Line 345 in 4dd71d6

self.fail('does_not_exist')

The view with custom lookup field doesn't seem to cause any trouble by itself.

So why does resolve it work in view, but not in related field?
'%20' in path gets unquoted quite early on, already in WSGI server (or django test client). This is seems not exactly well defined part of WSGI, so some servers may not unquote this, but django builtin server does, and so is uwsgi and few others. Anyway, this is just explanation why it does seem to work when working with path from taken request object.

In related field we deal with user provided data, which is not parsed by any magical WSGI implementation. So I think we should call uri_to_iri (just like django test server) on urlparse.urlparse(data).path.

[hexvolt]

@tomchristie @rooterkyberian @xordoquy it seems to be broken again since the Django's uri_to_iri() function has changed in Django2.0+

See what they've changed: django/django@03281d8#diff-d3fe562051dcbab80f60f067fa6a5838R180

I.e. now, %20 stays in the URL as opposed to how it worked before. Therefore, HyperlinkedRelatedField doesn't work with hyperlinks containing spaces%20 and returns "Object does not exist".

And the unit test in DRF passes because here #5179 it doesn't cover the %20 case. Thus, HyperlinkedRelatedField appear to be not fully compatible with Django 2.0+

[rpkilby]

Thanks @hexvolt.

[auvipy]

@rpkilby if you don't mind, can you include this on 3.11.x milestone? the milestone tag seems wrong