v1.17.0: User-defined hosting provider interface, Platform.sh, Acquia, loads of stuff

Installation/Upgrade

See the installation instructions for details, but it's easy:

• macOS Homebrew and Linux Linuxbrew: brew install drud/ddev/ddev or brew upgrade drud/ddev/ddev ). (You may need a brew update for homebrew to find the new release.). This works on Apple Silicon Homebrew as well.
• Windows: Use choco upgrade -y ddev to get this one, or download the ddev_windows_installer below.
• Linux and macOS with the install_ddev.sh script: Download the script, make it executable, and run it: ./install_ddev.sh or curl -LO https://raw.githubusercontent.com/drud/ddev/master/scripts/install_ddev.sh && bash install_ddev.sh
• And anywhere, you can just download the tarball or zipball, untar or unzip it, and place the executable in your path where it belongs.
• Consider ddev delete images after upgrading to free up disk space used by previous docker image versions. This does no harm.
• In the past, a ddev poweroff was required; now ddev itself detects that you have a new version and asks for permission to do a poweroff.

Key changes for v1.17.0

• Composer v2 is now the default composer version. If you need to change it, use ddev config --composer-version=1 && ddev start
• Brand new provider integration system, with user-configurable and extensible techniques, Acquia, Platform.sh, DDEV-Live, Pantheon.io integration. Integration is user-editable, so it's easy to add, improve, share integrations. There are examples provided for rsync and local file sources. You can ddev pull ddev-live, ddev pull platform, etc.
• ddev push is available for those who want it, although it can be dangerous operation, so make sure it fits your workflow. You can push database and/or files to an upstream hosting provider. Implementation examples are provided and tested for Acquia, DDEV-Live, Pantheon.io, and Platform.sh
• Official support for Apple M1 Silicon with Docker Desktop, see docs
• Excellent improvements to ddev snapshot, including ddev snapshot restore --latest, prompted ddev snapshot restore, ddev snapshot --list, ddev snapshot --cleanup, ddev snapshot --all, thanks to @cmuench !
• ddev snapshot restore now shows progress as it goes, and it doesn't time out on huge snapshot restores. You can also use ddev snapshot restore --latest
• Built-in support for Blackfire.io profiling, with a ddev blackfire command to do it. (docs)
• New ddev config --auto option that configures a project with detected defaults
• Web container environment variables can be set in config.yaml or global_config.yaml with the web_environment key, or ddev config global --web-environment=ENV1=someval,ENV2=someotherval
• ddev heidisql command provides a nice database browser on Windows and Windows WSL2 thanks to @andreashager .
• The PHP default for new projects is now PHP 7.4. A few project types (Drupal 6 and Magento 1) override this to use 5.6.
• The MariaDB default for new projects is 10.3.
• New docs theme! Take a look at [https://ddev.readthedocs.io/) ! Thanks @unn !

Smaller changes

• Drush Launcher is no longer used. Drupal 6 and Drupal 7 projects get Drush 8 directly, and Drupal 8 and 9 must have drush site-installed and it will be found in the $PATH. You can however use "drush8", which is installed in the container as /usr/local/bin/drush8. So you can drush8 status for example.
• You can now configure ddev to fail if a hook task fails by using ddev config —fail-on-hook-fail or fail_on_hook_fail: true in the project or global config, thanks to @majamee
• Drupal8/9 no longer has a custom "working_dir", meaning that ddev ssh and ddev exec land in the project root by default, not in the docroot.
• php8.0 extensions redis and memcached are now included
• sudo is removed from ddev-dbserver and ddev-ssh-agent, and is no longer used in startup routines of ddev-webserver. This is important for long-term security posture, but also is a workaround for a bug in gitpod that was preventing gitpod restores in https://github.com/shaal/ddev-gitpod from working.
• Zend assertions are enabled by default, thanks @heddn!
• ddev start now waits for all containers to come up, not just the "main" ones. So if you have third-party services like solr or elasticsearch, it will wait for them to come up and it will error if they die on the way up.
• You can now use gitpod.io to build ddev and its images for contributing.
• Double nginx's http2_max_header_size to 32k in both ddev-router and web container.
• The script test_ddev.sh is now provided. You can quickly test your ddev installation and provide diagnostic output. This is requested in bug reports. curl -sSL -o /tmp/test_ddev.sh https://raw.githubusercontent.com/drud/ddev/master/scripts/test_ddev.sh && bash /tmp/test_ddev.sh

Caveats

• The provider interface has been completely redone. You no longer have to ddev auth pantheon or ddev config pantheon. Now you ddev pull pantheon (or whatever provider). Please see the docs on the new provider interface, and thank you for testing and reporting your results, questions, issues, bugs!
• Since drush launcher has been removed from the web container, on Drupal 8+ you'll want to make sure your project has composer site-installed (ddev composer require drush/drush). Alternately you can use "drush8" inside the web container.
• Because Drupal8+ no longer has a "working_dir" that lands things on the docroot instead of the project root, you may be surprised if you expect if your docroot is different from your project root and you ddev ssh or ddev exec and expect the working directory to be the docroot. It will now be /var/www/html. This will also affect hooks if they expect a specific directory to execute.
• composer 2 is the default composer version now. If you need composer v1 on a project, ddev config --composer-version=1
• If. you had composer_version: "2"in your project config.yaml it's best to remove that line or set composer_version: "", since version 2 is the default now. ddev config --composer-version="". This also avoids conflicts with webimage_extra_packages.
• The command ddev restore-snapshot became ddev snapshot restore. Use ddev snapshot restore -h for full details.

Commits since v1.16.7

ab69043 Docs fixes for v1.17 [skip ci][ci skip] (#2923)
e83bcd2 Specify remote alias locations for acquia integration (#2926)
f7b5344 Remove commented line in platform example [skip ci][ci skip] (#2927)
c44a0f9 Remove github sponsors from docs/index.md [skip ci][ci skip] (#2925)
1806e16 Make sure drush knows about acquia aliases (#2924)
ab27e62 Change testdata for db backup to env var for ddev-live (#2922)
145c955 Update mkcert-in-Firefox docs; Firefox tab naming changed (#2921) [skip ci][ci skip]
01fd373 Use upstream hostname instead of IP for upstream in ddev-router, resolves #2805 (#2913)
e2936a2 Fix ddev-live provider example to use actual filename downloaded as db (#2918)
ad4bb66 Remove developer docs from index [skip ci][ci skip] (#2920)
0494942 Minor docs for v1.17 (#2894) [skip ci][ci skip]
7985f71 Revert "Latest docker (version 20) (#2897) [skip ci][ci skip]" (#2915)
0912a37 Use the real ddev-live latest binary (#2912)
f73dd77 Use text template instead of html template, fixes #2908 (#2909)
6bb5384 Fix provider command documentation (#2907) [skip ci][ci skip]
d0c0790 Make test_ddev.sh not need xq, not quit if not found, fixes #2905 (#2906)
9cd9db4 Revert macos_ddev_nfs_setup script because docker mac m1 has gone back to old networking (#2901)
b797fab Remove "ddev drush launch" from quickstart in cli-usage.md (#2903) [skip ci][ci skip]
88b7ec4 Killing and restarting docker doesn't work on latest docker for mac (#2900)
46108e3 Bump images to v1.17.0-rc1 (#2895)
9e5228a Latest docker (version 20) (#2897) [skip ci][ci skip]
d6d5246 [docs only] Use fully-qualified name for service when configuring in client, fixes #2744 [skip ci][ci skip] (#2892)
588f3d9 Test push only on one test system (#2891)
2c1e7aa Update brew upgrade command (#2888)
0f9a5b4 Fix broken docs sidebar, fixes #2885 [skip ci][ci skip] (#2893)
6a71501 Include architecture in ddev version, fixes #2884 (#2889)
36beb4f test_ddev.sh should not depend on other scripts [skip ci][ci skip] (#2890)
532b450 Implement push command and provide examples, fixes #406 (#2875)
1ce024c Ongoing docs for completion of ddev v1.17.0 (#2873)
bbbee93 Enabling develop mode in xdebug so we get the pretty stack trace outputs (#2882)
7096adc Clarify caveat about snapshot compatibility (#2879)
223d4d7 Fix macos_ddev_nfs_setup.sh - was broken on amd64 (#2876)
868e2ea Convert from packr2 to go:embed for bundling files, fixes #2343 (#2872)
78b0881 Report WSL2 to segment as separate operating system (#2871)
b606da0 Update acquia.md to fix bug referencing wrong token var (#2869) [skip ci][ci skip]
39bbf6d Docs updates for v1.17, including mac M1 update to nfs script, fixes #2792, fixes #2804 (#2868)
7274c32 Use gerardog/gsudo instead of mattn/sudo on Windows sudo.exe (#2864)
1d2cb1d Poweroff if new ddev version, fixes #1786 (#2859)
2f28b15 Add a script to parse docker desktop version, change brew cask install (#2865)
28b28c0 Touch up Windows WSL2 hosts file support and docs, rename DRUD_INTERACTIVE, fixes #2533 (#2867)
0a68078 Update Docker Desktop installation instructions with homebrew - no more brew cask install (#2866)
3883068 Make sure missing USER env doesn't cause panic, fixes #2860 (#2862)
cf65c71 Add test_ddev.sh and update the issue templates [skip ci][ci skip] (#2863)
549852c Store ddev binary artifacts separately for download (#2861)
d2b41f4 [tests only] Wait a little longer on healthcheck startup (#2855)
ba6d030 Wait for all containers to come up on ddev start, fixes #2660 (#2848)
8b2e149 Fix snapshot restore progress broken by docker-compose bug (#2854)
6388092 Updates local docs theme to match live with material theme (#2840)
027e807 Reorganize test scripts, use single generate_artifacts, tests only (#2853)
41e955b Drupal 7: drushrc.php coding standards, fixes #2849 (#2850)
6df6876 Use the correct variable name for Pantheon token (#2847)
c74cdb3 [DOCS] Rename AdditionalSettings.php to AdditionalConfiguration.php (#2852) [skip ci][ci skip]
c91110e Don't let app.Exec() run against non-running container, fixes #1858 (#2830)
5c58c58 Bump NSIS to 3.06.1 in CircleCI test build (#2659)
80c076e Enable zend assertions by default (#2833)
2e3d2a3 Add circleci-based arm64 testing (#2845)
6129763 Improve acquia provider config example, add guardrails for other providers (#2842)
8bf1cb2 move gitpod docker-up to a "secondary" terminal (#2841) [skip ci][ci skip]
c70f298 Use composer version 2 as default (#2835)
8cf313e Be way less aggressive with removing CREATE DATABASE or USE statements in ddev import-db, fixes #2787 (#2831)
21a98d8 Reintroduce magerun and magerun2 (#2829)
37e83c0 Fix acquia provider integration broken by upstream (Acquia) bug (#2837)
b2fb743 Add gitpod support to ddev repository [skip ci][ci skip] (#2838)
7e5acd6 PhpMyAdmin test in TestDdevFullSiteSetup started failing because of upstream image change (#2836)
1053f05 Adds mime type for compressed javascript and css with a .gzip extension to fix TYPO3 compression problem (#2783)
52e4d9d [tests only] Add linux types for buildkite-agent (#2828)
78d2e2e Strip _ce from docker version if added by some linux distros, fixes #2815 (#2823)
737f5cd Fixes a xdebug config reference, which has changed since DDEV updated… (#2827) [skip ci][ci skip]
5f9dba0 Begin buildkite testing with mac M1 and WSL2, fixes #2288 (#2818)
3c12369 Minor fix to circleci container test (#2825)
0a022be Experiment with removing chown from ddev-ssh-agent startup script (#2816)
dc26ba1 [docs][tests] Ongoing docs work and lighten up on markdownlint (#2819)
2351b1f [tests only] git install bats-core since it doesn't seem to be working from package (#2820)
2589713 Use only go 1.16 now that it's released (#2814)
4715231 Bump upstream for ddev-webserver from php 8.0.2, etc. (#2813)
42ffa6b Remove sudo from ddev-dbserver and ddev-ssh-agent, limit use in ddev-webserver, fixes #2607 (#2806)
d68ef2a Process /var/lock/apache2 in ddev-webserver Dockerfile, fixes #2799 (#2809)
8ac90f3 Remove .DS_Store files before restoring snapshot, fixes #2765 (#2810)
d62f87c Refactor ddev-dbserver builds (#2808)
67bff55 Provider refactor followups (#2800)
7165d70 [docs] Update OSX install step order for clarity. (#2793) [skip ci][ci skip]
751405a Ongoing docs updates [skip ci][ci skip] (#2781)
b0ba7ff Acquia hosting integration, fixes #2681, fixes #905 (#2796)
edf85ad Remove special host.docker.internal handling for mac M1 (#2790)
37ac5f0 Large snapshot restore shouldn't return until it's done and should show status (#2747)
3c45945 Provider cleanup and completion (#2788)
0fb72e3 Fix minor formatting issue in Additional Project Hostnames documentation (#2789) [skip ci][ci skip]
6acd654 Re-implement DDEV-Live provider integration (#2784)
67495ed Rework Pantheon integration to use new yaml-based technique (#2782)
3f61198 Adds Drush site:install and uli to D8 and D9 quickstart (#2780)
29078ab Update index.md - link to brew.sh was not correct (#2777) [skip ci][ci skip]
f8a26d2 Platform.sh and generic hosting provider integration, fixes #2384, fixes #2194 (#2763)
78b99e4 Use updated upstream for yarn fix, fixes #2772 (#2775)
41d3f26 Miscellaneous docs fixes [skip ci][ci skip] (#2748)
4a0767a Minor test improvements (tests only) (#2771)
a205053 [docs] Fix typo in step-debugging.md (#2774) [skip ci][ci skip]
5653ef3 Update in-container-configuration.md (#2770)
8c939be Remove magerun from web container as people use composer-installed version (#2768)
5b75fa9 Testcache was wrong for circleci (CircleCI tests only) (#2766)
2fc5203 Use go 1.16-rc1 to build M1 builds (#2764)
0df5360 Remove several circleci jobs as github actions is doing them (#2761)
e68cc34 Improve DDEV configuration validation errors (php_version, mysql_version, etc.) (#2745)
4094078 Turn off buildkit build in ddev start (docker-compose) (#2758)
170cd19 Bring in php8.0 extensions redis and memcached from upstream (#2759)
9db9035 Improve fragile test TestComposeStreams (#2757)
37826a4 MariaDB default didn't get changed to 10.3 for M1 (#2750)
56830ff Remove drush launcher config from web image [skip ci][ci skip] (#2749)
10b71d9 Add Blackfire agent and full support (#2448)
17ee510 Hardened webserver needs to allow /usr/local/bin update, fixes #2725 (#2743)
5427087 Use default_server in nginx configs instead of server_name _, fixes #2729 (#2731)
7a7e4a3 Stop setting working_dir to docroot in Drupal8+, fixes #2534 (#2734)
d73268c Add ddev config --auto, fixes #1096 (#2733)
1d9931f Allow setting web environment in config.yaml, fixes #871 (#2742)
d374ae9 Give an error on duplicate snapshot anme, fixes #2515 (#2739)
51ede53 Improve snapshot restore with --latest, prompting, move to ddev snapshot restore, fixes #1886, fixes #1163 (#2724)
a7cb148 Use GitHub Actions for linux tests, fixes #2604 (#2615)
32db28e Do static checking in github action (#2738)
a66f4e6 Spellcheck docs, fixes #2365 (#2735)
0c2af6b add PHP 8.0 to the supported php versions (#2736) [skip ci][ci skip]
a12c41f Skip TestGetLocalHTTPResponse on Windows (#2732)
547e10f [docs] Update maintained badge on README (#2728)
a1c1f5d Remove duplicate menu entry for shell autocompletion in docs (#2726)
70df9b2 Bump PHP default to php7.4, fixes #2657 (#2716)
dd8369b Miscellaneous docs fixes (#2695) [skip ci][ci skip]
ffd96db [docs] Add "Limitations using ddev composer" for #2547 (#2718) [skip ci][ci skip]
0ef1b0c [docs only] TYPO fixes (#2723) [skip ci]
85e656f Add --list and --cleanup options to snapshot command, for #1163 (#2701)
67f99f5 Stop using drush launcher, use drush8 for drupal6/7, fixes #2514 (#2720)
900d464 Experimental support for Apple Silicon M1 (darwin_arm64), fixes #2338 (#2687)
0688703 Update spf/cobra and dependencies for #2702 (#2715)
d1b68e4 Bump default mariadb version to 10.3, fixes #2232 (#2717)
b6c6f06 Double nginx's http2_max_header_size to 32k, fixes #2697 (#2698)
bc1d660 TestServices has been failing a lot, give solr a little time to come up (#2714)
5491d0c Ignore drushrc.php writing when settings disabled (#2684)
2ff4f0b Create new HeidiSQL command for Windows and WSL2 (#2679)
d86db92 Update cli-usage.md for bedrock (#2699)
5d17f0e Allow configuration to fail when a hook execution fails, fixes #2536 (#2652)
4ab01db Improve 3rd-party Apache SOLR setup in additional-services.md (#2670) [skip ci][ci skip]
e93110b [tests only, circleci] Use Ubuntu 20.04 for builds, bump cache, build makemsis from source (#2713)
95a6f48 auth ssh command had out of date switch in long description (#2711) [skip ci][ci skip]
d8a2e49 Use TYPO3 11 for installations by default (#2707) [skip ci][ci skip]