Pass value of environment variables to ddev pull in command line call

[jcnventura]

Is there an existing issue for this?

• I have searched the existing issues

Is your feature request related to a problem? Please describe

The ddev pull command is extremely useful, but is constrained by the fact that you can't pass any custom value to the environment variables.

As such, we need to create and maintain a set of provider scripts that change in only 1 line in order to be able to handle different values of these variables.

Describe the solution you'd like

I'd like to be able to call ddev pull with something like ddev pull --var1=value1, and to have the value of var1 for that provider file be replaced with value1.

The defined value of var1 in the environment_variables of the provider f could be used as fallback.

Describe alternatives you've considered

None.

Additional context

In our case, we already have drush aliases for working with drush for sql-sync and rsync, and we would like to take advantage of these when calling ddev pull so that we don't have to reinvent the wheel. In the case of the file sync command, we have it simply written as:

environment_variables:
  drush_alias: test

files_import_command:
  command: |
    # set -x   # You can enable bash debugging output by uncommenting
    set -eu -o pipefail
    ls /var/www/html/.ddev >/dev/null # This just refreshes stale NFS if possible
    drush rsync @${drush_alias}:%files @self:%files -y
  service: web

At the moment, we have multiple copies of this provider file, just differing on the value of the drush_alias value. We'd like to be able to call ddev pull --drush_alias=stage to ddev pull from the stage environment instead.

[rfay]

You'll note that the platform.yaml already works this way, there's no need to do any editing of the file, and this is the future of all the provided integrations.

So if you put variables into a .env, or a config.yaml, or a global_config.yaml, or a config.*.yaml you already have this behavior. I'm not a fan of people having to provide them on the command line though, because it's not repeatable and nobody could remember how to do it.

[jcnventura]

I don't get it. How can I tell my colleagues to use ddev pull to pull from the @stage alias using the above code without asking them to first edit a file, if not via a command-line parameter?

If they have to edit a file, they might as well be editing the provider file anyway.

[rfay]

So first, with your existing situation, I know that you check in the .ddev folder right? So with no changes at all, you'd copy the example file to yaml file, fix it, then check it in, and your colleagues would never edit anything. For example, if it were pantheon, you'd copy the pantheon.yaml.example to pantheon.yaml, edit what you needed, and check it in.

But platform.yaml is the next level where you don't have to edit anything, you just need to put the project and environment in as environment variables wherever you want them. For most people, that would be the config.yaml web_environment section, and most teams check in the config.yaml, so no editing needed.

I must be missing something here, I'm pretty sure your team checks in project config like .ddev, so this should be easy.

[rfay]

You don't mention the integration that you're talking about, but my intent is to remove things like https://github.com/drud/ddev/blob/3fab17d7b64de7e63df34373b78e77674afc870c/pkg/ddevapp/dotddev_assets/providers/acquia.yaml.example#L27-L29

and replace them with environment variables, which come from config.yaml, .env, or global_config.yaml.

[Berdir]

All those options require a ddev restart, right and manually editing these config files, correct?

Specifically with platform.sh, I think the question here is how to control the pull enviroment at runtime.

When using platform commands directly, I'm used to them defaulting to the currently active environment, which is something I use quite often when pulling data, so I start a new platform.sh environment, or start working on one where a colleague already did some work and want to do some work there. So I want to pull that database to my local env.

Right now, I do this for a new project:

# set project env variable automatically based on current project, something that it could default to if set.
ddev config --web-environment-add="PLATFORM_PROJECT=`platform project:info id`"
# add env variable for environment, could default to current branch name if not set to mimic platform CLI
ddev config --web-environment-add="PLATFORM_ENVIRONMENT=main"

One problem is that calling --web-environment-add will add a second line, there is no --web-environment-replace, that would already be helpful. But I still need to restart.

So what this issue is IMHO asking (and what I came here to do as well but found this as related) is to be able to do

ddev pull platform --env="PLATFORM_ENVIRONMENT=staging"

The command would then set that env within the container before running the pull command.

Or alternatively for better DX pull-type specific commands like ddev pull platform --environment="staging", but I assume that's harder to do

[rfay]

The way most people deal with this is they have two yaml files. ddev pull prod and ddev pull staging. Create a prod.yaml and a staging.yaml. Super easy.

[Berdir]

Ok, but with platform.sh, it's not just those two but we frequently have feature branches where we develop and test new things and fairly frequently switch between them.

[rfay]

Have you tried using ENVVAR=xxx ddev pull platform ?

[rfay]

A related need here is to be able to specify options for the add-on, for example memcached or elasticsearch versions

[rfay]

Also requested in ddev-platformsh:

• Feature request: choose environment for ddev pull ddev-platformsh#63

[rfay]

Hoping you like this @jcnventura @Berdir - it's now in HEAD and will be in next release. If you use homebrew, super easy to brew install --HEAD --fetch-head ddev

[dbarron]

It looks like this was added to 1.21.4 but I'm on 1.22.5 it does not seem to be working for pantheon. Is it supposed to? I don't get errors, but it doesn't override the values in config.yaml.

.ddev/config.yaml

web_environment:
- PANTHEON_PROJECT=myproject
- PANTHEON_ENV=dev

.ddev/providers/pantheon.yaml

environment_variables:
  project: ${PANTHEON_PROJECT}.${PANTHEON_ENV}

When I use this command it pulls, but it pulls from dev (the default in config.yaml), not test, the value set with --environment.
ddev pull pantheon --environment=PANTHEON_ENV=test

I also tried putting the defaults in .ddev/.env and got the same results.

PANTHEON_PROJECT=myproject
PANTHEON_ENV=dev

If I don't set defaults at all and just pass a value into the pull command I get a bash error
bash: line 1: PANTHEON_ENV: unbound variable

I am restarting after each of these changes.

[stasadev]

Hi @dbarron,

It looks like this was added to 1.21.4 but I'm on 1.22.5 it does not seem to be working for pantheon. Is it supposed to? I don't get errors, but it doesn't override the values in config.yaml.

The Panteon recipe has not yet adapted these changes. You can track the progress of this issue here:

• Rework Pantheon provider implementation to avoid environment_variables and make it so people don't have to edit #4760