Request with basic auth does not work when passwords include umlaut

[maros]

• Mojolicious version: 9.35.
• Perl version: 5.38.0
• Operating system: debian 12.2

Steps to reproduce the behavior

Basic auth does not work when passwords include umlaut.

use v5.38;
use Mojo::UserAgent;
use Test::Most;
use utf8;

my $ua = Mojo::UserAgent->new;
my $userinfo = 'testing:testöööng';

binmode *STDOUT,':utf8';
binmode *STDERR,':utf8';

foreach my $url (
    'https://testing:testöööng@outpost.geizhals.at/maro/testing',
    'https://testing:test%C3%B6%C3%B6%C3%B6ng@outpost.geizhals.at/maro/testing',
    'https://testing:test%F6%F6%F6ng@outpost.geizhals.at/maro/testing',  # not sure if this should work
    ) {
    my $tx = $ua->get($url);
    is($tx->req->url->userinfo,$userinfo,'Userinfo parsed ok');
    is($tx->res->code,200,'Code 200');
}

Same url works fine when requested via curl

curl  'https://testing:test%C3%B6%C3%B6%C3%B6ng@outpost.geizhals.at/maro/testing'
hallo

Expected behavior

Request should succeed

Actual behavior

401 error

[fskale]

You should really should study the cookbook:
https://docs.mojolicious.org/Mojolicious/Guides/Cookbook
and the browse to "[Basic authentication]" ;-)
Create e new Mojo::URL Object and provide your authentication data there:
(use the env var MOJO_CLIENT_DEBUG and set it to 1 to get additional debug output).
E.g: (Minimal Script)

use Mojo::UserAgent;
use Mojo::URL;
use Test::Most;

my $ua = Mojo::UserAgent->new;
my $url = Mojo::URL->new('https://outpost.geizhals.at/maro/testing');
my $userinfo = $url->userinfo('testing:testöööng');

my $tx = $ua->get($url);
is($tx->req->url->userinfo,$userinfo,'Userinfo parsed ok');
is($tx->res->code,200,'Code 200');

[maros]

i extended the testcase to also include explicitely created Mojo::URLs. But it also results in a 401

use v5.38;
use Mojo::UserAgent;
use Test::Most;
use utf8;

my $ua = Mojo::UserAgent->new;
my $USERINFO = 'testing:testöööng';
my $BASEURL = 'outpost.geizhals.at/maro/testing';

my $mojo_url = Mojo::URL->new("https://$BASEURL");
$mojo_url->userinfo($USERINFO);

foreach my $url (
    "https://${USERINFO}\@${BASEURL}",
    "https://testing:test%C3%B6%C3%B6%C3%B6ng\@${BASEURL}",
    "https://testing:test%F6%F6%F6ng\@${BASEURL}",
    $mojo_url
    ) {
    my $tx = $ua->get($url);
    is($tx->req->url->userinfo,$USERINFO,'Userinfo parsed ok');
    is($tx->res->code,200,'Code 200');
}

The request authorization header is the same for all four requests.

[kraih]

If any of this is a bug depends entirely on the specs. Please provide references to specific spec sections we are currently violating. A random internet service is not a valid test case.

[maros]

According to RFC-7617 the encoding of the Authoritzation is undefined for backward-compatibility reasons and as long as it remains compatible with US-ASCII. The RFC suggests that it can be latin-1 or utf-8. Most server implementations and browsers nowadays seem to prefer the later.

It is possible to include a charset parameter in the Authorization header, which only accepts utf8 as the only valid value. In this case

The user's name is "test", and the password is the string "123"
followed by the Unicode character U+00A3 (POUND SIGN). Using the
character encoding scheme UTF-8, the user-pass becomes:
't' 'e' 's' 't' ':' '1' '2' '3' pound
74 65 73 74 3A 31 32 33 C2 A3
Encoding this octet sequence in Base64 ([RFC4648], Section 4) yields:
dGVzdDoxMjPCow==

The Authoritzation header from my tests contains 'dGVzdGluZzp0ZXN09vb2bmc=' which is latin-1 and not utf-8 encoded. The webserver i was testing against is a nginx/1.18.0 with no special configuration

    server {
        listen *:443 ssl;
        ssl_certificate /etc/ssl/LE/outpost.geizhals.at.fullchain.pem;
        ssl_certificate_key /etc/ssl/LE/outpost.geizhals.at.privkey.pem;
        ssl_trusted_certificate /etc/ssl/LE/outpost.geizhals.at.fullchain.pem;
        root /var/www/html;
        allow all;

        location /maro {
           auth_basic           "Maros' Reich";
           auth_basic_user_file /etc/nginx/htpasswd;
        }
    }

If you don't want to change Mojo useragent behaviour to use utf8 as default encoding, i suggest to honour the charset parameter in https://metacpan.org/pod/Mojo::UserAgent::Transactor#tx if it equals to utf-8