feat: implement ddev pull upsun provider integration, fixes #5446

[stasadev]

The Issue

• Add ddev pull upsun support #5446

How This PR Solves The Issue

Adds integration for Upsun.

Manual Testing Instructions

Docs are here https://ddev--5529.org.readthedocs.build/en/5529/users/providers/upsun/

Try ddev pull upsun in Upsun project
Try ddev push upsun

Automated Testing Overview

TestUpsunPull and TestUpsunPush.

Related Issue Link(s)

Release/Deployment Notes

[github-actions]

Download the artifacts for this pull request:

• all-ddev-executables.zip
• ddev-linux-amd64.zip
• ddev-linux-arm64.zip
• ddev-macos-amd64.zip
• ddev-macos-arm64.zip
• ddev-windows-amd64-installer.zip

See Testing a PR

[rfay]

ddev-windows-mutagen isn't supposed to run on PRs... I guess it's because you did this PR on ddev org. And of course it's irrelevant to this PR. But TestUpsunPull/Push worked, yay, https://github.com/ddev/ddev/actions/runs/6829336445/job/18575278444?pr=5529

[rfay]

Looking great, thanks!

In docs, I think this needs a mention in the top-level providers markdown file, which is hard to find, that's a different problem.

I also think we need a blog post about providers in general and the two new ones specifically.

Right now, trying to manually test with ddev push upsun I'm stuck on

Uploading database...
Wrote database dump from project 'upsun-d10' database 'db' to file /Users/rfay/workspace/upsun-d10/.ddev/.downloads/db.sql.gz in gzip format.

The server responded:
Hello Randy Fay (UUID: 949f3e03-0c56-479e-8131-3a0eecc6d412), you successfully authenticated, but could not connect to service o2wd4lih6xwpc-main-bvxea6i--ddev-d10 (reason: service doesn't exist or you do not have access to it)

[rfay]

Multiple databases does not yet work, because upsun relationships has a different structure than platform relationships.

To configure multiple databases in upsun you need to have them configured in the .upsun/config.yaml:

services:
  mariadb:
    type: mariadb:10.6
    configuration:
      schemas:
        - main
        - junk
      endpoints:
        mysql:
          default_schema: main
          privileges:
            main: admin
            junk: admin

upsun relationships looks like this:

mariadb:
    -
        host: mariadb.internal
        hostname: 2jins5mtttge2mfrfdcxkxkpte.mariadb.service._.ca-1.platformsh.site
        cluster: 47kohks3azo5c-main-bvxea6i
        service: mariadb
        rel: mysql
        scheme: mysql
        username: mysql
        password: 365d3d4b06cdf5173da0bf86008de9bf
        port: 3306
        epoch: 0
        path: main
        query:
            is_master: true
        fragment: null
        public: false
        host_mapped: false
        type: 'mariadb:10.6'
        ip: 169.254.156.174
        url: 'mysql://mysql:365d3d4b06cdf5173da0bf86008de9bf@mariadb.internal:3306/main'

But platform relationships lists the databases (path):

database:
    -
        username: user
        fragment: null
        ip: 169.254.193.18
        cluster: 5bviezdszcmrg-platform-pull-7tsp6cq
        host: database.internal
        path: main
        query:
            is_master: true
        password: ‘’
        port: 3306
        host_mapped: false
        service: db
        hostname: jzglbn36ugubmwptswhuwv62sm.db.service._.ca-1.platformsh.site
        epoch: 0
        rel: mysql
        scheme: mysql
        type: ‘mariadb:10.4’
        public: false
        url: ‘mysql://user:@database.internal:3306/main’

I guess the difference is the key, database as opposed to mariadb ?

[rfay]

I guess the logic in

ddev/pkg/ddevapp/dotddev_assets/providers/platform.yaml

Lines 54 to 57 in a0e022e

	PLATFORM_RELATIONSHIPS="" platform relationships -y -e "${PLATFORM_ENVIRONMENT}" | yq 'with_entries(select(.[][].type == "mariadb:*" or .[][].type == "*mysql:*" or .[][].type == "postgresql:*")) ' >${db_relationships_file}
	db_relationships=($(yq ' keys | .[] ' ${db_relationships_file}))
	db_names=($(yq '.[][].path' ${db_relationships_file}))
	db_count=${#db_relationships[@]}

is inadequate for upsun.

It's worth a little effort to fix it, or we can just say "only one db for now"

[stasadev]

I will look into it tomorrow, and if I can't figure it out quickly, we can proceed with "only one db for now".

[rfay]

So far I haven't found a good way to list the databases provided. Have questions going in slack. You don't need to fiddle with it until they reply. thanks!

Also added you to project with two databases, https://console.upsun.com/ddev/47kohks3azo5c

[stasadev]

Removed relationships (they do not work and are not needed at the moment) and rebased.

[rfay]

Rebased, but didn't re-push. This will need to be re-pushed before it's merged.

[rfay]

Looks great to me. I manually tested pull with rfay-d10, and branched a new testpush environment and did ddev push upsun --environment=UPSUN_PROJECT=testpush and it worked out fine.

[rfay]

I updated the ddev-webserver image.