Authentication does not work inside `opds.py`

[timholl]

Describe the bug/problem
Currently, I am unable to download any books to my mobile reader via OPDS catalog.

I found that via the @download_required annotation on opds.py line 386, a function download_required in web.py is called, which returns a 403 - Forbidden if current_user.role_download() is not true.
As far as I can see, for OPDS current_user is always Guest (because authentication is done via HTTP Basic Authentication, if any).
So even for non-anonymous OPDS I have to give the Guest user the download permission.

Furthermore, as far as I can see, the Guest user is created on demand (create_anonymous_user in ub.py), but without the download permission, so download via OPDS is effectively disabled by default.

1. Is this intended? If yes, is it documented how to enable it?
2. The Guest user is hidden in the user list, so the best solution I came up with is guessing the Guest users ID in /admin/user/X. Is there a better way to edit the Guest user or is this the intended one?

To Reproduce
Steps to reproduce the behavior:

1. Try to download via OPDS from any device.

Logfile
Add content of calibre-web.log file or the relevant error, try to reproduce your problem with "debug" log-level to get more output.

None

Expected behavior
A clear and concise description of what you expected to happen.

Download should be possible or it should be documented that the Guest user requires download permission.

Screenshots
If applicable, add screenshots to help explain your problem.

None

Environment (please complete the following information):

• OS: Debian 10
• Python version: Python 3.7.3<
• Calibre-Web version: currentmaster (e16c0caebb93244d3ad1d2482553cc84d762c089)
• Docker container: None
• Special Hardware: LXC
• Browser: [e.g. Chrome 83.0.4103.97, Safari 13.3.7, Firefox 68.0.1 ESR]

Additional context
Add any other context about the problem here. [e.g. access via reverse proxy, database background sync, special database location]

[OzzieIsaacs]

I found that via the @download_required annotation on opds.py line 386, a function download_required in web.py is called, which returns a 403 - Forbidden if current_user.role_download() is not true.
-> Correct

As far as I can see, for OPDS current_user is always Guest (because authentication is done via HTTP Basic Authentication, if any).
-> No

The HTTP Basic Authentication also loads the user (if it's send during the download request). For some reason some of the OPDS Feed reader apps "forget" to send the credentials on a download request (or even on a get cover request). You can try it like I did with a computer browser and you will see it works (as far as I know, and my tests are telling me).

Furthermore, as far as I can see, the Guest user is created on demand (create_anonymous_user in ub.py), but without the download permission, so download via OPDS is effectively disabled by default.

It's not really created on demand, this is one of the dataase migration routines if you start with an old database (>2years), or with a fresh installation, then the user is once created, with default no download rights, that's true.

The Guest user is hidden in the user list, so the best solution I came up with is guessing the Guest users ID in /admin/user/X. Is there a better way to edit the Guest user or is this the intended one?
-> This is a bug, if the guest user account is accessible without deactivated anonymous browser. It should also make no differences which settings the anoymous user has if it's deactivated, it should be always deny access.
So the intended way would be activate anonymous browsing in admin -> basic configuration -> feature configuration -> anonymous browsing.

Is this intended? If yes, is it documented how to enable it?
Yes in the wiki https://github.com/janeczku/calibre-web/wiki/Configuration#feature-configuration (as usual, you can only find it if you know where to search for it and what you are searching for)

Some working/nonworking opds readers are also listed in the wiki: https://github.com/janeczku/calibre-web/wiki/FAQ#which-opds-readers-work-with-calibre-web

[OzzieIsaacs]

I double checked it, even if the guest user has download rights, as long as anonymous browsing is disabled the access to the download function is denied.
Lucky me.

[timholl]

Thank you for your answer.

I just tried to reproduce this, removed the download permission from the Guest user and opened the OPDS in my browser:

The Basic Auth credentials are correct, though.

If I patch download_required in web.py to dump the current user just before returning 403 ...

diff --git a/cps/web.py b/cps/web.py
index 8abae0a9..eb54a119 100644
--- a/cps/web.py
+++ b/cps/web.py
@@ -251,6 +251,7 @@ def download_required(f):
     def inner(*args, **kwargs):
         if current_user.role_download():
             return f(*args, **kwargs)
+        log.debug("Current user: " + str(current_user))
         abort(403)

... I get that current_user is Guest

$ tail -f calibre-web.log
[2020-10-13 20:15:10,751]  INFO {cps:99} Starting Calibre Web...
[2020-10-13 20:15:10,878]  INFO {cps.server:168} Starting Tornado server on :443
[2020-10-13 20:15:16,893] DEBUG {cps.web:254} Current user: <User 'Guest'>

If I log in via the web interface and then re-execute the above request, it works (it uses the valid session cookie from the login).
So, maybe you had valid session cookies while trying to reproduce my issue? (Just a guess)

Maybe I do not understand the way authentication is done by OPDS-Clients, but from my understanding a valid session cookie should not be required, since authentication is done by HTTP Basic Auth.

I use the "Kybook 3" reader, which is on the compatibility list.

[timholl]

Furthermore, I also do not see private shelves in OPDS (while I can see public ones).
This is also a hint that the current user is not determined currectly.

[timholl]

As I found out, the load_user_from_request function should be responsible for loading the correct user from requests with HTTP Basic Authentication. Therefore it registers itself to the login manager via @lm.request_loader.

But: It never gets called.

Also the load_user function never gets called for every request to /opds (it gets called for requests to web.py).
So the intended functionality of flask_login does not work inside opds.py.

[OzzieIsaacs]

Let me check

[OzzieIsaacs]

I can't reproduce it:

Left upper corner, my automatic test using requests library of python (no browser). Right upper corner showing the download request from left side with nothing more than basic-auth header.
Bottom right corner access rights from app.db ( admin rights =1, download right = 2, anonymous user =32)
left bottom corner log file showing logged in user is admin

[OzzieIsaacs]

Also for me the function get's called:

You are right, a valid session is not required as you can see in the wireshark trace in the last post, the basic auth header is parsed in the load_user_from_request function.
which version of flask-login / flask are you using?

Also the testruns on windows show no problem in this part of the code with this environment:

[OzzieIsaacs]

This is the linux environment for the tests (with python requests as browser in this part of the test, to not get influcences by stored cookies):

The code for the tests is located in: https://github.com/OzzieIsaacs/calibre-web-test/blob/master/test/test_opds_feed.py (but is not so easy to get the environment up running)

[timholl]

Hmm, that's odd. I have the exact same library versions, use the current git master (version without any changes), removed all caches etc., but still that issue occurs.
Apparently the cause is somewhere out of calibre-web, maybe I have messed up something during the setup.
As I am the only one having that problem, I close this issue for now.

Thank you for your time and debugging efforts.