Search Inside (i.e. full-text search of Archive books) fails with Unicode

[bfalling]

https://openlibrary.org/search/inside?q=Антон+Чехов

generates the following error:

/opt/openlibrary/deploys/openlibrary/93ddc9e/openlibrary/templates/search/inside.html: error in processing template: UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-4: ordinal not in range(128) (falling back to default template)

[LeadSongDog]

Very odd.
https://openlibrary.org/search?q=%D0%90%D0%BD%D1%82%D0%BE%D0%BD+%D0%A7%D0%B5%D1%85%D0%BE%D0%B2 worked but
https://openlibrary.org/search/inside?q=%D0%90%D0%BD%D1%82%D0%BE%D0%BD+%D0%A7%D0%B5%D1%85%D0%BE%D0%B2 did not

[mekarpeles]

I can look into this eventually, cc'ing @gdamdam only because it's relevant to his full text search work (no action required from him)

[hornc]

unfortunately I merely stopped the error, not made the search work :(
https://dev.openlibrary.org/search/inside?q=%D0%90%D0%BD%D1%82%D0%BE%D0%BD+%D0%A7%D0%B5%D1%85%D0%BE%D0%B2

returns a ridiculous number of results. I think the unicode characters are being dropped now. Needs further investigation.

[hornc]

I think a bug-fixed urlencode is needed like is implemented here: https://github.com/internetarchive/openlibrary/blob/master/openlibrary/utils/solr.py#L12

[tfmorris]

This works for me.

@bfalling are you happy to close it?