Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Insecure HTTPS->HTTP 302 redirects in OPDS catalogs fail but not made clear why #7007

Closed
johnbeard opened this issue Dec 17, 2020 · 12 comments · Fixed by #7032
Closed

Insecure HTTPS->HTTP 302 redirects in OPDS catalogs fail but not made clear why #7007

johnbeard opened this issue Dec 17, 2020 · 12 comments · Fixed by #7032
Milestone
2021.01

Comments

@johnbeard
Copy link
Contributor

johnbeard commented Dec 17, 2020 •
edited
Loading

  • KOReader version: 2020.11.60 (nightly)
  • Device: Emulator & Clara HD

Issue

Some OPDS catalogs point to URLs which do a 302 Redirect. For example Project Gutenburg links to

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="http://www.gutenberg.org/cache/epub/46/pg46.epub">here</a>.</p>
<hr>
<address>Apache Server at www.gutenberg.org Port 80</address>
</body></html>

Steps to reproduce

  • Open Project Gutenburg OPDS catalog
  • Try to download any book
  • It will fail and a 302 redirect HTML file saved in its place (as above)

Similar for the ManyBooks catalog which is itself behind a 302: https://manybooks.net/opds/index.php → http://srv.manybooks.net/opds/index.php

And for books from the Flibuter catalog: https://www.flibusta.is/b/226848/epub → http://static.flibusta.is:443/converter/get/convert?url=http%3A%2F%2Fflibusta.is%2Fb%2F7412a0b774fdf8c0998b13e1ebeac686%2Fdownload&out=epub&md5=7412a0b774fdf8c0998b13e1ebeac686
@Frenzie
Copy link
Member

Frenzie commented Dec 17, 2020

The error handling toward the user could be improved slightly, but not allowing downgrading security is not a bug.

@Frenzie
Copy link
Member

Frenzie commented Dec 17, 2020 •
edited
Loading

I.e., this describes a request that should not be allowed (and isn't, which is why you're seeing this).

https://www.gutenberg.org/ebooks/46.epub.noimages
but this redirects to:
http://www.gutenberg.org/cache/epub/46/pg46.epub
and the file that gets saved is:

@johnbeard
Copy link
Contributor Author

johnbeard commented Dec 17, 2020 •
edited
Loading

Oh, so it's the HTTP -> HTTPS downgrade that's triggering this? I didn't notice the protocol was different.

@johnbeard
Copy link
Contributor Author

johnbeard commented Dec 17, 2020 •
edited
Loading

Annoyingly, for PG at least, HTTPS does work for the target: https://www.gutenberg.org/cache/epub/46/pg46.epub does exist and work, so their OPDS generator is busted.

But the ManyBooks cert doesn't cover srv.manybooks.net 🙄

@johnbeard
Copy link
Contributor Author

C.f.: gutenbergtools/gutenbergsite#10

@Galunid
Copy link
Member

Galunid commented Dec 18, 2020

#6942 had follow redirect implemented, it was reverted because of https -> http downgrade, so once guttenberg fixes it it's pretty much ready to go (guards to prevent https -> http downgrade must be implemented though).

@Frenzie
Copy link
Member

Frenzie commented Dec 18, 2020

@Galunid What you did in there was an ugly hack to get around the restriction. Redirects will be followed completely automatically because of #6334

@johnbeard johnbeard changed the title HTTP 302 redirects in OPDS catalogs not followed Insecure HTTPS->HTTP 302 redirects in OPDS catalogs fail but not clear why Dec 22, 2020
@johnbeard johnbeard changed the title Insecure HTTPS->HTTP 302 redirects in OPDS catalogs fail but not clear why Insecure HTTPS->HTTP 302 redirects in OPDS catalogs fail but not made clear why Dec 22, 2020
@johnbeard
Copy link
Contributor Author

Project Gutenburg appears to have fixed it at their end. ManyBooks and FIlbusta still show this.

So I guess this issue is now about popping a sensible message for the user like "Insecure HTTPS->HTTP downgrade blocked: go and moan at the OPDS catalog publisher for doing it wrong"?

@Frenzie
Copy link
Member

Frenzie commented Dec 22, 2020

ManyBooks works just fine for me. Apparently someone changed it to HTTPS, but my program has retained the old URL: http://manybooks.net/opds/index.php.

@Frenzie
Copy link
Member

Frenzie commented Dec 22, 2020

Same for Filibusta. Just change those URLs back.

@Frenzie
Copy link
Member

Frenzie commented Dec 22, 2020

Problem introduced by 960b2ae

Frenzie added a commit to Frenzie/koreader that referenced this issue Dec 22, 2020
@Frenzie
Revert ManyBooks and Flibusta to HTTP
f3826c5 
Cf. <koreader#7007>.
@Frenzie
Copy link
Member

Frenzie commented Dec 22, 2020

Anyway, I'll have a look at the insecure downgrade thing too.

@Frenzie Frenzie mentioned this issue Dec 22, 2020
Merged
Frenzie added a commit to Frenzie/koreader that referenced this issue Dec 22, 2020
@Frenzie
OPDS: Inform and warn about downgrade attack
46dd71c 
Fixes <koreader#7007>.
@Frenzie Frenzie mentioned this issue Dec 22, 2020
Merged
@Frenzie Frenzie added this to the 2021.01 milestone Dec 22, 2020
Frenzie added a commit that referenced this issue Dec 22, 2020
@Frenzie
Revert ManyBooks and Flibusta to HTTP (#7031)
d355e64 
Cf. <#7007>.
Frenzie added a commit that referenced this issue Dec 24, 2020
@Frenzie
OPDS: Inform and warn about downgrade attack (#7032)
ad08ce9 
Fixes <#7007>.
@Frenzie Frenzie mentioned this issue Mar 21, 2021
Merged
@SteadyQuad SteadyQuad mentioned this issue Mar 21, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
2021.01
Development

Successfully merging a pull request may close this issue.

OPDS: Inform and warn about downgrade attack Frenzie/koreader
3 participants
@Frenzie @johnbeard @Galunid