Regression: basic authentication in URL does not work

[dottedmag]

% r2e list  | grep netch
23: [*] netch (https://dottedmag:<redacted password>@netch.dreamwidth.org/data/atom?auth=digest -> dottedmag@dottedmag.net)
% r2e run --no-send
processing error: expected bytes-like object, not str: netch (https://dottedmag:<redacted password>@netch.dreamwidth.org/data/atom?auth=digest -> dottedmag@dottedmag.net)
% r2e --version
rss2email 3.9
%

It used to work in an ancient Python 2 version of rss2email.

[Ekleog]

Are there non-ascii characters in the <redacted password>? That sounds like something that could cause this issue, and be due to the 2->3 switch.

[dottedmag]

Nope, purely uppercase and lowercase ASCII Latin and digits.

[Profpatsch]

That’s a classic issue; we are probably using a bytes-based function and passing a unicode string.

[lkmorlan]

URLs of this format used to work:
https://user:pass@example.com/path

Now I get "processing error: nonnumeric port: 'pass'". It is not correctly parsing the URL and interprets the password to be the port.

[Ekleog]

With what version of feedparser is this new error? Also, do you remember with what versions of rss2email/feedparser the “previously working” behavior was?

[lkmorlan]

I think it was a 2.x version of rss2email. I copied that comment from my 2017 report in the wking repository.

[Ekleog]

Hmm ok, thank you :) just, do you know whether this is happening with feedparser 5 or 6 ?

[lkmorlan]

I am seeing this problem in the rss2email package in Debian which depends on python3-feedparser 5.2.

[auouymous]

http://test:password@foo.com/bar.php can be used to test this since nothing is sent to the server.

[ERROR] processing error: a bytes-like object is required, not 'str': test (http://test:password@foo.com/bar.php -> rss2email)

The problem is entirely in feedparser. Encoding user_passwd (in feedparser) fixes the initial error and then throws the same error in return opener.open(request). That code is different in 6.0 so it might have been fixed.

-    auth = base64.standard_b64encode(user_passwd).strip()
+    auth = base64.standard_b64encode(user_passwd.encode()).strip()

[Ekleog]

Hmm ok, so I guess let's block this on #73 and see whether it still happens after that is fixed, and then reevaluate whether it's an upstream issue or a local one :)

[auouymous]

Feedparser 6.0.1 has the same problem, kurtmckee/feedparser#234 fixes it.

[auouymous]

This is fixed with feedparser 6.0.5. After #188 is merged the required version in setup.py and pyproject.toml should be bumped.

[Ekleog]

Should hopefully be fixed now :) Thank you all for your help!