Add possibility to specify the path of an auth.json

[sippsolutions]

It would be nice to have the option to specify the location of the auth.json.

In our case we have a repository with a ./composer.json that provides tools to build and deploy the application and a src/composer.json that installs the dependencies of the actual software which has its code source in src/.

We need to install modules that require authentication in both ./ and src/ and therefor it would be awesome to have the possibility to use the src/auth.json in the root ./composer.json as well.

Maybe it could be useful for some users to specify multiple paths. We should still respect the user's {home}/auth.json and the auth.json at the same path as the currently evaluated composer.json.

A config example could be:

{
    "config": {
        "auth-files": [
            "src/auth.json",
            "../auth.json",
            "vendor/my/package/auth.json"
        ]
    }
}

[Seldaek]

IMO auth.json should really only ever be in the home dir, as you shouldn't share/commit credentials in the repo.. So I don't see a huge need for this capability.

[sippsolutions]

@Seldaek It's common practice to add an auth.json for Enterprise software (such as Magento, where you have different auth keys for different projects) in the same directory as the composer.json just to identify as authorized to download these packages (which an build and deployment framework needs to be authorized as well), which shouldn't be an issue as those repositories are privately hosted anyways. As it's currently possible to use the auth.json in the same directory as the composer.json, I'd say this feature request is justified - it'd be possible to define other locations as well, not only inside the repository.

[tpetry]

When using docker build using a different auth.json is also very important as the file could be a mounted docker secret which will not be part of the image.

[computator]

Using this with docker is my usecase as well

[interfixnet]

We have an auth.json file in the root of some repos which references CI environment variables to access private repos. When doing local development, we need to use our user's credentials instead. We would like to be able to specify the path to our user's own auth.json file when building locally to bypass the auth.json file in the repo.

[Seldaek]

Do you have a chance to pass env vars easily? Because you could read the user auth.json into COMPOSER_AUTH env var and make that available in docker, then it should override the docker auth.json - https://getcomposer.org/doc/03-cli.md#composer-auth

[DaanBiesterbos]

I would argue that your proposal would be an invitation for bad practices, and possible security issues. Just one recent example I think of... I am currently working for a client that recently migrated from bitbucket to gitlab on premise. If they'd hardcoded auth configs in a couple of thousand projects. This migration would have been so much harder, and arguably not worth it.

In your example it would get worse: "vendor/my/package/auth.json".
This will become impossible to maintain on a larger scale. Let's say your package includes some test suite that requires private packages from a private repository, specifically meant for development packages? (so it won't be possible to deploy them to production by accident). What might happen to our resolved auth.json? Your package suddenly needs to become aware of the context in which it is used, and visa versa. You need to manually maintain a list of files to include. External scripts -/ pipelines / applications / dockerfiles, etc would need to be updated to include the new auth config from your package. Packages provide a means to write reusable code, and to separate concerns. And I think developers would be encouraged to stop creating new packages for the wrong reasons.

If you absolutely have to resolve an auth.json from external locations, you could write a script to merge the files for you, depending on your personal acceptance criteria. I don't think it should be something that should be supported by composer. For several reasons, but most importantly, because it would encourage solutions that won't benefit the PHP community.

Instead, I would recommend to the COMPOSER_AUTH environment variable instead.
https://getcomposer.org/doc/03-cli.md#composer-auth

This will provide all the flexibility you need both in production and during development.

Depending on your circumstances you could COMPOSER_AUTH as a system variable for global use.
Although in practice I rarely do this. To help you out a bit I'll give some examples solutions that work for me.

Set environment variable at runtime:

COMPOSER_AUTH=...  composer install

When you're using docker you could add the variable as a build argument

ARG GITLAB_ACCESS_TOKEN
ARG GITLAB_HOST='gitlab.foo.mycompany.com'
ARG COMPOSER_AUTH='{"gitlab-token":{"${GITLAB_HOST}": "'${GITLAB_ACCESS_TOKEN}'"},"gitlab-domains" :["${GITLAB_HOST}"]}'
ENV COMPOSER_AUTH $COMPOSER_AUTH

Using docker compose and .env files
Usually I create an .env.dist file and add .env to .gitignore.
It is possible to use system variables in your docker compose config. In addition, docker-compose will automatically load a .env file in the same directory. You can use this files to override the variables that we refer to. (look into the docker-compose --env-file option to load other .env files if needed).

ACCESS_TOKEN_FOO=
ACCESS_TOKEN_BAR=

If you run into any problems related to JSON encoding, try using double quotes.
Single quotes work in a Makefile or Dockerfile for example, but you may run into problems when you're using single quotes in .env files or other plain text files that will need to be parsed.

  php:
    image: hello-world/whatever-php:8.2
    environment:
      - COMPOSER_AUTH="{\"gitlab-token\":{\"gitlab.foo.mycompany.com\": \"'${ACCESS_TOKEN_FOO}'\", \"gitlab.bar.mycompany.com\": \"'${ACCESS_TOKEN_BAR}'\"}, \"github-domains\": [\"gitlab.foo.mycompany.com\", \"gitlab.bar.mycompany.com\"]}"

You can load the variables from the .env file into your container at build time, or at runtime when we run the container.
In production you probably would use a secret manager. (you don't want to update X images when you rotate your access token etc, plus devops will probably be able to automate most of it). If you're using pre-build images for every application it makes sense to use build arguments for your access token(s) for example, and keep the ENV COMPOSER_AUTH in a base image. That said, you'll want to use COMPOSER_AUTH in other containers as well.

You might use some composer image to run composer for different PHP versions for example.
(Check https://hub.docker.com/_/composer for more examples.)

export PHP_VERSION=8.2
export COMPOSER_AUTH=...

docker run --rm --interactive --tty \
  --volume $PWD:/app \
  --env COMPOSER_AUTH=$COMPOSER_AUTH \
  --user $(id -u):$(id -g) \
  --entrypoint /usr/local/bin/composer \
  someprivateregistry.helloworld.com/composer:$PHP_VERSION
  install

This is when things would start to become more complex. I personally prefer to use a Makefile to solve this.
In general, I prefer to set the COMPOSER_AUTH variable at runtime. For further reference, I'll just copy a part of a makefile I often use.

ifneq (,)
  $(error This Makefile requires GNU Make. )
endif

DOCKER_UID    = $(shell id -u)
DOCKER_GID    = $(shell id -g)
DOCKER_BIN    ?= docker

PHP_TARGET_VERSION ?= 7.4
PHP_VERSION_PARTS  := $(subst ., ,$(PHP_TARGET_VERSION))
PHP_MAJOR_VERSION  := $(word 1,$(PHP_VERSION_PARTS))
PHP_MINOR_VERSION  := $(word 2,$(PHP_VERSION_PARTS))
PHP_MICRO_VERSION  := $(word 3,$(PHP_VERSION_PARTS))
PHP_SHORT_VERSION  ?= ${PHP_MAJOR_VERSION}.${PHP_MINOR_VERSION}


.PHONY:   help
.DEFAULT_GOAL := help

export COMPOSER_AUTH="{\"github-domains\": [\"gitlab.helloworld.com\"]}"


help: ## Show available targets
	@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n  make \033[36m<target>\033[0m PHP_VERSION<[a-z.-]+> (default: 7.2.34) \n\nTargets:\n"} /^[a-zA-Z_-]+:.*?##/ { printf "  \033[36m%-12s\033[0m %s\n", $$1, $$2 }' $(MAKEFILE_LIST)

shell: ## Open shell
	docker run -it  -v $(PWD):/app  --env COMPOSER_AUTH helloworld/php:1.2.3
	
composer-install: ## Run composer install
	@echo "\033[36mRun composer install\033[0m"
	-@$(DOCKER_BIN) run --rm  -v $(PWD):/opt   -w /opt --user=$(DOCKER_UID):$(DOCKER_GID)  --env COMPOSER_AUTH helloworld/composer:8.1 composer install
	@echo "\033[32m\nDone...\033[0m"

php-cs-fixer: ## Run php-cs-fixer to fix code style issues in a generated PHP projects
	@ echo "Run PHP codestyle fixer and compare to source..."
	@if $(DOCKER_BIN) run --rm \
		-v $(PWD):/data \
		cytopia/php-cs-fixer:2-php$(PHP_SHORT_VERSION) \
		fix .; then \
		echo "OK"; \
	else \
		echo "Failed! Execute PHP code style fixer to fix this error."; \
		exit 1; \
	fi

For local development, you may want to add this code to your makefile. This will prompt the user to enter an accesstoken when needed. Don't forget to add .local.env to .gitignore if you do this.

ifneq (,)
  $(error This Makefile requires GNU Make. )
endif

ENV_FILE=.local.env
ifeq ($(wildcard $ENV_FILE),)
$(shell bash -c 'touch ${ENV_FILE}')
endif
-include $(ENV_FILE)
export $(shell [ ! -n "$(ENV_FILE)" ] || cat $(ENV_FILE) | grep -v \
    --regexp '^('$$(env | sed 's/=.*//'g | tr '\n' '|')')\=')
    
 # Prompt for token if "GITLAB_ACCESS_TOKEN" is undefined
ifndef GITLAB_ACCESS_TOKEN
# Alternatively, provide the variable upon execution. Usage:  GITLAB_ACCESS_TOKEN=... make <target>)
GITLAB_ACCESS_TOKEN := $(shell bash -c 'read -p "Gitlab access token: " new; echo $$new')
# Add tokens to .env so we don't need to enter them next time
$(shell bash -c 'echo "GITLAB_ACCESS_TOKEN=${GITLAB_ACCESS_TOKEN}" >> ${ENV_FILE}')
endif

COMPOSER_AUTH = '{"gitlab-token":{"gitlab.helloworld.com": "'${GITLAB_ACCESS_TOKEN'"}, "gitlab-domains":["gitlab.helloworld.com"]}'


composer-install: ## Run composer install
	@echo "\033[36mRun composer install\033[0m"
	-@$(DOCKER_BIN) run --rm  -v $(PWD):/opt   -w /opt --user=$(DOCKER_UID):$(DOCKER_GID)  --env COMPOSER_AUTH helloworld/composer:1.2.3 composer install
	@echo "\033[32m\nDone...\033[0m"

[sippsolutions]

@DaanBiesterbos I wouldn't say a configurable auth file location would be an invitation for bad practices when currently you can perfectly use an auth file that lies in the same directory as the composer.json. Why would the location matter for security reasons? I'd say it's vice versa - an auth file that does not lie within the project's directory/repository would be better practice.

The vendor/my/package/auth.json was indeed a bad example, but it wouldn't be a use case for me anyway - just wanted to provide some destinations to fill my example.

If you think of my use case where we have multiple clients with each having it's own (privately hosted, non-public) repository with his project in it. To be able to fetch the files from Magento (Adobe Commerce, which is a widely used eCommerce system) you have to provide per-project credentials (as they are linked to the customers' license and bought modules).
Think of a developer's system where he has to install and maintain 10+ projects in that specific setup, while each of the customers having their own pair of credentials for the same composer repo - the only way we could think of (and did for the last years, and according to my experience in the last 10 years every agency dealing with Magento does so) was committing the composer auth.json. While we need the credentials for developer related stuff such as Magento's SemVer check etc. which we integrated in project's root ./composer.json and also the mentioned project itself in ./src/composer.json we're now bound to symlinking ./auth.json to ./src/auth.json. (And not only the developer needs those credentials, but also the deployment server)

Imho it wouldn't make things worse if the auth.json's location would be configurable as it's all better than having it in the same directory/repository as the composer.json.
The COMPOSER_AUTH variable is a good hint. Might think about that after my vacation. But as a first thought that would make things more complicated as the developer would need to pass that variable before being able to execute any composer update, require, etc. ...

[DaanBiesterbos]

@DaanBiesterbos I wouldn't say a configurable auth file location would be an invitation for bad practices when currently you can perfectly use an auth file that lies in the same directory as the composer.json. Why would the location matter for security reasons? I'd say it's vice versa - an auth file that does not lie within the project's directory/repository would be better practice.

The vendor/my/package/auth.json was indeed a bad example, but it wouldn't be a use case for me anyway - just wanted to provide some destinations to fill my example.

If you think of my use case where we have multiple clients with each having it's own (privately hosted, non-public) repository with his project in it. To be able to fetch the files from Magento (Adobe Commerce, which is a widely used eCommerce system) you have to provide per-project credentials (as they are linked to the customers' license). Think of a developer's system where he has to install and maintain 10+ projects in that specific setup, while each of the customers having their own pair of credentials for the same composer repo - the only way we could think of (and did for the last years, and according to my experience in the last 10 years every agency dealing with Magento does so) was committing the composer auth.json. While we need the credentials for developer related stuff such as Magento's SemVer check etc. which we integrated in project's root ./composer.json and also the mentioned project itself in ./src/composer.json we're now bound to symlinking ./auth.json to ./src/auth.json. (And not only the developer needs those credentials, but also the deployment server.)

So imho it wouldn't make things worse if the auth.json's location would be configurable as it's all better than having it in the same directory as the composer.json. The COMPOSER_AUTH variable is a good hint. Might think about that after my vacation.

Hmm as far as security is concerned, I would say that this feature would increase the risk of human error.
You can put access tokens to your auth.json for example.

The bigger problem with features like this is that it's easy to misuse them, so people probably will. And in my personal experience, anything that can be abused, will be abused. By hardcoding and merging auth.json files we'd end up with dependencies that make it harder to port and reuse our code. Which on itself is a bad thing, considering what composer is supposed to do. I am not familiar your use case. So I won't say that this feature would be problematic in your implementation perse.

I've seen pipelines where the auth.json file was generated based on configurable secrets in some cloud bucket... That sounds a bit similar. Of course, that won't work if those clients do share the same container. (which would be something worth considering, if this is not the case). In general environment variables are quite useful to solve problems of this kind.
Of perhaps you can add .client1.env, .client2.env etc plus additional .dist files to commit to the repository.
And populate the environment variables when you build or run the container.

docker run --rm --interactive --tty --env-file=.client1.env image:1.0 composer-install

Or something like that... Depending on your set up you might be able to load some file with secrets from a bucket. Then you don't need the auth files anywhere. Every client would have a config file either physical or in the cloud.

[sippsolutions]

We're using valet.sh instead of Docker, so we're developing multiple projects on one single physical system without virtualization.

When I try to think of a way we would deal with this if it'd be possible to define the auth location, I would see it more like an "authentication method fallback".
So I would probably configure something like this:

{
    "config": {
        "auth-files": [
            "/var/deploy/.composer/auth.json",
            "~/.composer/auth/customer-1.json",
            "~/.composer/auth.json"
        ]
    }
}

This composer.json would work on the deployment server (auth file 1), if this file is not present (development machine) it would use the customer specific auth file on the developer's system (file 2) or the global composer auth.json as a fallback.

We could then automatically provide the auth.json on the deployment system (maybe via a cloud credential system as you mentioned) and do the same on the developer's machine, but as they need to have multiple customer's credentials, we could provide customer-1.json, customer-2.json etc.

If we're not on the server (file 1) and the user has not yet provisioned their customer's auth files (file 2) the global auth file location will still be used. All this without having to copy the file in the project folder or store it within the same repository.

[DaanBiesterbos]

In general this would be exactly what environment variables aim to solve. I am not familiar with valet.sh.
If using environment variables proves to be very difficult, I guess you could consider to create a composer plugin.
If you listen to the "init" event you can load some config files, environment variables, or whatever. You should be able to set COMPOSER_AUTH at runtime.

I did not test this code, but something like this should work.

$default = [];
if (!empty($_SERVER['COMPOSER_HOME']) && file_exists($_SERVER['COMPOSER_HOME'] . '/auth.json')) {
	$default = json_decode(file_get_contents($_SERVER['COMPOSER_HOME'] . '/auth.json'));
}

$auth = [];
$initial = getenv('COMPOSER_AUTH');
if (!empty($initial)) {
	$auth = json_decode($initial, true);
}

$result = array_merge_recursive(
	$default,
	$auth,
	['gitlab-domains' => ['foo.helloworld.com', 'bar.helloworld.com'], 'gitlab-token' => ['foo.helloworld.com' => 'qwerty', 'bar.helloworld.com' => 'pass123']],
	['github-domains' => ['foo.bar.com', 'bar.bar.com'], 'github-token' => ['foo.bar.com' => 'qwerty', 'bar.bar.com'  => 'pass123']],
);

$_SERVER['COMPOSER_AUTH'] = $_ENV['COMPOSER_AUTH'] = json_encode($result);
putenv('COMPOSER_AUTH=' . $_SERVER['COMPOSER_AUTH']);

https://getcomposer.org/doc/articles/scripts.md

[DaanBiesterbos]

And for what it's worth, you'd still rely on COMPOSER_AUTH. So even though you'd do something like this, the code itself remains as is. If you would disable the plugin and set COMPOSER_AUTH instead, nothing changes.