HTTP Authentication doesn't work for google artifact registry python index

[ArshanKhanifar]

I've created a google cloud artifact repo, and I've created a service account & granted the following permissions to it:

# Service account creation and permissions (this is typically only done once)
# create the service account
deployer-service-account:
	gcloud iam service-accounts create $(sa_name) --display-name="Pypi publisher"

grant-permissions:
	gcloud artifacts repositories add-iam-policy-binding $(artifact_repo) \
		--location=$(artifact_location) \
		--project=$(gcp_project) \
		--member=serviceAccount:$(sa_name)@$(gcp_project).iam.gserviceaccount.com \
		--role=roles/artifactregistry.writer --rle=roles/artifactregistry.reader

I'm able to get the artifact settings like so:

# show the pypi registry settings (useful for modifying the ~/.pypirc file)
show-artifact-settings:
	gcloud artifacts print-settings python \
	    --project=$(gcp_project) \
	    --repository=$(artifact_repo) \
	    --location=$(artifact_location)

This gave me an output like this:

# Insert the following snippet into your pip.conf

[global]
extra-index-url = https://_json_key_base64:<somesecret>=@<somerepo>.pkg.dev/<gcp-project>/<registry>/simple/

Now with this, I'm able to install my package from pip.

pip install my-private-package

Setting UV_EXTRA_INDEX_URL or with the --extra-index-url command-line argument, I'm not able to install the package.

root:shared/ (main✗) # make uv-install                        [18:15:36]
uv pip install -vv --index-url https://_json_key_base64:<sercret>=@<location>-python.pkg.dev/<gcp-project>/<gcp-artifact>/simple/ --extra-index-url https://pypi.org/simple my_private_package 2>&1 > error.log
 uv_requirements::specification::from_source source=my_private_package
    0.000187s DEBUG uv_interpreter::python_environment Found a virtualenv named .venv at: /home/shared/.venv
    0.000535s DEBUG uv_interpreter::interpreter Cached interpreter info for Python 3.11.9, skipping probing: .venv/bin/python
    0.000546s DEBUG uv::commands::pip_install Using Python 3.11.9 environment at .venv/bin/python
 uv_client::linehaul::linehaul
    0.001797s DEBUG uv_client::base_client Using registry request timeout of 300s
 uv_client::flat_index::from_entries
 uv_resolver::resolver::solve
      0.002260s   0ms DEBUG uv_resolver::resolver Solving with target Python version 3.11.9
   uv_resolver::resolver::choose_version package=root
   uv_resolver::resolver::get_dependencies package=root, version=0a0.dev0
        0.002310s   0ms DEBUG uv_resolver::resolver Adding direct dependency: my-private-package*
   uv_resolver::resolver::choose_version package=my-private-package
     uv_resolver::resolver::package_wait package_name=my-private-package
 uv_resolver::resolver::process_request request=Versions my-private-package
   uv_client::registry_client::simple_api package=my-private-package
     uv_client::cached_client::get_cacheable
       uv_client::cached_client::read_and_parse_cache file=/root/.cache/uv/simple-v6/pypi/my-private-package.rkyv
 uv_resolver::resolver::process_request request=Prefetch my-private-package *
 uv_client::cached_client::from_path_sync path="/root/.cache/uv/simple-v6/pypi/my-private-package.rkyv"
          0.002601s   0ms DEBUG uv_client::cached_client No cache entry for: https://pypi.org/simple/my-private-package/
       uv_client::cached_client::fresh_request url="https://pypi.org/simple/my-private-package/"
            0.002625s   0ms DEBUG uv_auth::middleware No credentials found for: https://pypi.org/simple/my-private-package/
     uv_client::cached_client::get_cacheable
       uv_client::cached_client::read_and_parse_cache file=/root/.cache/uv/simple-v6/<somehash>/my-private-package.rkyv
 uv_client::cached_client::from_path_sync path="/root/.cache/uv/simple-v6/<somehash>/my-private-package.rkyv"
          0.081441s   0ms DEBUG uv_client::cached_client Found stale response for: https://<location>-python.pkg.dev/<gcp-project>/<gcp-artifact>/simple/my-private-package/
          0.081456s   0ms DEBUG uv_client::cached_client Sending revalidation request for: https://<location>-python.pkg.dev/<gcp-project>/<gcp-artifact>/simple/my-private-package/
       uv_client::cached_client::revalidation_request url="https://<location>-python.pkg.dev/<gcp-project>/<gcp-artifact>/simple/my-private-package/"
            0.081469s   0ms DEBUG uv_auth::middleware Request already has an authorization header: https://<location>-python.pkg.dev/<gcp-project>/<gcp-artifact>/simple/my-private-package/
          0.389440s 308ms DEBUG uv_client::cached_client Found modified response for: https://<location>-python.pkg.dev/<gcp-project>/<gcp-artifact>/simple/my-private-package/
       uv_client::cached_client::new_cache file=/root/.cache/uv/simple-v6/<somehash>/my-private-package.rkyv
       uv_client::registry_client::parse_simple_api package=my-private-package
         uv_client::html::parse url=https://<location>-python.pkg.dev/<gcp-project>/<gcp-artifact>/simple/my-private-package/
   uv_resolver::version_map::from_metadata
   uv_distribution::distribution_database::get_or_build_wheel_metadata dist=my-private-package==0.1.0.post1
     uv_client::cached_client::get_serde
       uv_client::cached_client::get_cacheable
         uv_client::cached_client::read_and_parse_cache file=/root/.cache/uv/built-wheels-v2/index/<somehash>/my-private-package/0.1.0.post1/manifest.msgpack
        0.392549s 390ms DEBUG uv_resolver::resolver Searching for a compatible version of my-private-package (*)
        0.392590s 390ms DEBUG uv_resolver::resolver Selecting: my-private-package==0.1.0.post1 (my_private_package-0.1.0.post1.tar.gz)
 uv_client::cached_client::from_path_sync path="/root/.cache/uv/built-wheels-v2/index/<somehash>/my-private-package/0.1.0.post1/manifest.msgpack"
   uv_resolver::resolver::get_dependencies package=my-private-package, version=0.1.0.post1
     uv_resolver::resolver::distributions_wait package_id=my-private-package-0.1.0.post1
            0.392790s   0ms DEBUG uv_client::cached_client No cache entry for: https://<location>-python.pkg.dev/<gcp-project>/<gcp-artifact>/my-private-package/my_private_package-0.1.0.post1.tar.gz#sha256=<package-sha256>
         uv_client::cached_client::fresh_request url="https://<location>-python.pkg.dev/<gcp-project>/<gcp-artifact>/my-private-package/my_private_package-0.1.0.post1.tar.gz#sha256=<package-sha256>"
              0.392893s   0ms DEBUG uv_auth::middleware Adding authentication to already-seen URL: https://<location>-python.pkg.dev/<gcp-project>/<gcp-artifact>/my-private-package/my_private_package-0.1.0.post1.tar.gz#sha256=<package-sha256>
error: Failed to download and build: my-private-package==0.1.0.post1
  Caused by: HTTP status client error (401 Unauthorized) for url (https://<location>-python.pkg.dev/<gcp-project>/<gcp-artifact>/my-private-package/my_private_package-0.1.0.post1.tar.gz#sha256=<package-sha256>)
make: *** [Makefile:52: uv-install] Error 2

[ArshanKhanifar]

My UV version:

root:shared/ (main✗) # uv --version                           [18:23:23]
uv 0.1.28

[charliermarsh]

I will try to reproduce this.

[zanieb]

Hm the logs indicate that we are propagating authentication. Perhaps we are parsing it incorrectly. Could be related to the _ in the username or the = at the end of the password?

[zanieb]

Is there an @ in the secret?

[ArshanKhanifar]

The secret ends with a = but doesn't have an @ or = anywhere else

[ArshanKhanifar]

Looks like google cloud uses _json_key_base64 by default.

I wanted to see if I can use a different kind of secret, but doesn't seem like they export the key in any other format.

[charliermarsh]

Okay, I went through the setup and unfortunately it's working for me without issue? I'll note that my base64 key does not end in an equals sign. Is it possible that you've generated it incorrectly?

[samypr100]

If it's base64 encoding, it could very likely end with = or == per RFC4648 and any character from it's alphabet is also fair game (e.g. +, / and the URL safe variants of those -, _ respectively)

[avelychko12]

Have exactly the same issue, but I have == in the end of the key.
It stopped working in 0.1.19

As far as I can understand, you are not decoding password here

uv/crates/uv-auth/src/store.rs

Line 110 in 7bcca28

password: url.password().map(str::to_string),

so later you will try to base64 into %3D%3D instead of ==
but pip is doing unquote for password so %3D%3D will be converted to == https://github.com/pypa/pip/blob/06d21db4ff1ab69665c22a88718a4ea9757ca293/src/pip/_internal/utils/misc.py#L499

[zanieb]

@avelychko12 thanks for the investigation! I put up #2947 if anyone wants to give it a try while I try to write a decent test case for this.

[ArshanKhanifar]

@charliermarsh thanks for following up with this, so my workaround currently for this is to just use a different kind of token.

Basically after activating the service account:

activate-service-account:
	gcloud auth activate-service-account --key-file=$(keyfile_name)

I get the auth token via:

gcloud auth print-access-token

Then I use that with _token set as username:

# Gets the service account's credentials & sets them
get_index_url:
	$(eval token := $(shell gcloud auth print-access-token))
	$(eval index_url := "https://_token:$(token)@$(artifact_location)-python.pkg.dev/$(gcp_project)/$(artifact_repo))/simple/")

my scripts are make scripts so apologies if it seems a bit confusing but all of the above summarized is to get the token via gcloud auth print-access-token then use _token:$ACCESS_TOKEN@url as the pypi index url.

But still with base64 encoded json keys I wasn't able to run this correctly.

[zanieb]

Should be resolved by #2976 and available in the next release.