Fix: REST API "download=False" error

[eimrek]

My proposal for #5574

for context, see the discussion in the issue.

[ltalirz]

Thanks for the fix @eimrek !

Would you mind adding a test that failed before your fix and now passes?

[eimrek]

Ok, I figured out the source of this problem:

The JSONEncoder imported here

aiida-core/aiida/restapi/common/utils.py

Line 15 in cb82327

from flask.json import JSONEncoder

in flask 1.x was based on simplejson, that inherently converts bytes objects into strings. In flask 2.0.0, it was redesigned based on the default python json. See here: https://flask.palletsprojects.com/en/2.0.x/changes/ and here pallets/flask#3555

As a result, in Aiida 2.0 REST (where flask 1 was updated to flask 2), bytes objects are not supported any more.

I implemented a conversion of a bytes array into str in the CustomJSONEncoder. In principle there are other way of doing this as well (e.g. by checking isinstance(o, bytes)). I can change it, if some other implementation is preferable.

Similar issue is probably also the source of #5575 but this PR doesn't fix that. i will investigate that in a separate PR.

I'll try to add a test as well.

[giovannipizzi]

Thanks!

[unkcpz]

This is related to #5573
If you look at

aiida-core/aiida/restapi/resources.py

Line 459 in cb82327

if query_type == 'download' and download not in ['false', 'False', False] and results:

, my suggestion change there can also address this issue.

[unkcpz]

I think it makes sense to combine #5573 to this PR.

[eimrek]

Thanks @unkcpz. I disagree, as the source issues are different. #5573 fixes a bug that has been there forever, and affects how boolean query parameters are parsed. The issue of this PR is in the encoding of the binary string to JSON, that was introduced by the aiida 2.0 upgrade. Although the issue comes out in case of download=False, that might seem similar, it probably also affects other parts of the restapi.

I think it makes sense to isolate the code changes that fix specific issues, instead of merging everything together. the current PR fixes the linked issue (and the PR of #5573 does not affect this).

PR #5573 fixes a separate issue, namely, that lowercase "false" in the query parameters is interpreted as True, but could be bypassed by specifying an uppercase "False" instead. I suspect this is the reason is it was not fixed for over 3 years.

[unkcpz]

@eimrek thanks, I see. I misunderstood it, my bad. here you explicitly use download=False which does not have the issue of #5573.

[unkcpz]

@eimrek sorry for the back and forth. One more minor request, others are all good. Could you then rebase and make a detailed commit message? Thanks.

[unkcpz]

I think an elif for bytes type makes more sense.

[eimrek]

This discussion here seems to prefer the duck-typing version: https://stackoverflow.com/questions/34869889/what-is-the-proper-way-to-determine-if-an-object-is-a-bytes-like-object-in-pytho