Skip to content

Commit

Permalink
[TASK] Add composer-mode to our acceptance test matrix
Browse files Browse the repository at this point in the history
All applicable acceptance tests are now also executed
in composer mode. Tests that check classic-mode specific
functions are excluded and are therefore now tagged
as classic-mode test.

The composer mode instance is generated with our
CLI setup tools, allowing to mimic the realworld
case where a instance is creating via our setup tools
instead of from fixtures (which we still use for classic
mode tests, but try to reducde/avoid for composer mode tests).

The existing existing classic-mode test acceptance test
execution will keep running as-is.

Resolves: #103297
Releases: main, 12.4, 11.5
Change-Id: I64973f110931b51ed2ef7ef8f8cc3411834fcf37
Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/83027
Tested-by: Stefan Bürk <stefan@buerk.tech>
Reviewed-by: Christian Kuhn <lolli@schwarzbu.ch>
Tested-by: Christian Kuhn <lolli@schwarzbu.ch>
Reviewed-by: Stefan Bürk <stefan@buerk.tech>
Tested-by: core-ci <typo3@b13.com>
Reviewed-by: Benjamin Franzke <ben@bnf.dev>
Tested-by: Benjamin Franzke <ben@bnf.dev>
  • Loading branch information
bnf committed Mar 19, 2024
1 parent e8b234b commit fa6c3fc
Show file tree
Hide file tree
Showing 29 changed files with 617 additions and 66 deletions.
82 changes: 76 additions & 6 deletions Build/Scripts/runTests.sh
Expand Up @@ -174,6 +174,7 @@ Options:
-s <...>
Specifies the test suite to run
- acceptance: main application acceptance tests
- acceptanceComposer: main application acceptance tests
- acceptanceInstall: installation acceptance tests, only with -d mariadb|postgres|sqlite
- buildCss: execute scss to css builder
- buildJavascript: execute typescript to javascript builder
Expand Down Expand Up @@ -236,7 +237,7 @@ Options:
- pdo_mysql
-d <sqlite|mariadb|mysql|postgres>
Only with -s functional|functionalDeprecated|acceptance|acceptanceInstall
Only with -s functional|functionalDeprecated|acceptance|acceptanceComposer|acceptanceInstall
Specifies on which DBMS tests are performed
- sqlite: (default): use sqlite
- mariadb: use mariadb
Expand Down Expand Up @@ -293,12 +294,12 @@ Options:
Build/Scripts/runTests.sh -s unit -- --filter filterByValueRecursiveCorrectlyFiltersArray
-g
Only with -s acceptance|acceptanceInstall
Only with -s acceptance|acceptanceComposer|acceptanceInstall
Activate selenium grid as local port to watch browser clicking around. Can be surfed using
http://localhost:7900/. A browser tab is opened automatically if xdg-open is installed.
-x
Only with -s functional|functionalDeprecated|unit|unitDeprecated|unitRandom|acceptance|acceptanceInstall
Only with -s functional|functionalDeprecated|unit|unitDeprecated|unitRandom|acceptance|acceptanceComposer|acceptanceInstall
Send information to host instance for test or system under test break points. This is especially
useful if a local PhpStorm instance is listening on default xdebug port 9003. A different port
can be selected with -y
Expand Down Expand Up @@ -566,9 +567,9 @@ fi
# Suite execution
case ${TEST_SUITE} in
acceptance)
CODECEPION_ENV="--env ci"
CODECEPION_ENV="--env ci,classic"
if [ "${ACCEPTANCE_HEADLESS}" -eq 1 ]; then
CODECEPION_ENV="--env ci,headless"
CODECEPION_ENV="--env ci,classic,headless"
fi
if [ "${CHUNKS}" -gt 0 ]; then
${CONTAINER_BIN} run ${CONTAINER_COMMON_PARAMS} --name ac-splitter-${SUFFIX} ${IMAGE_PHP} php -dxdebug.mode=off Build/Scripts/splitAcceptanceTests.php -v ${CHUNKS}
Expand Down Expand Up @@ -630,6 +631,75 @@ case ${TEST_SUITE} in
;;
esac
;;
acceptanceComposer)
rm -rf "${CORE_ROOT}/typo3temp/var/tests/acceptance-composer" "${CORE_ROOT}/typo3temp/var/tests/AcceptanceReports"

PREPAREPARAMS=""
TESTPARAMS=""
case ${DBMS} in
mariadb)
${CONTAINER_BIN} run --rm ${CI_PARAMS} --name mariadb-ac-${SUFFIX} --network ${NETWORK} -d -e MYSQL_ROOT_PASSWORD=acp -e MYSQL_DATABASE=ac_test --tmpfs /var/lib/mysql/:rw,noexec,nosuid ${IMAGE_MARIADB} >/dev/null
waitFor mariadb-ac-${SUFFIX} 3306
PREPAREPARAMS="-e TYPO3_DB_DRIVER=mysqli -e TYPO3_DB_DBNAME=ac_test -e TYPO3_DB_USERNAME=root -e TYPO3_DB_PASSWORD=acp -e TYPO3_DB_HOST=mariadb-ac-${SUFFIX} -e TYPO3_DB_PORT=3306"
TESTPARAMS="-e typo3DatabaseName=ac_test -e typo3DatabaseUsername=root -e typo3DatabasePassword=funcp -e typo3DatabaseHost=mariadb-ac-${SUFFIX}"
;;
mysql)
${CONTAINER_BIN} run --rm ${CI_PARAMS} --name mysql-ac-${SUFFIX} --network ${NETWORK} -d -e MYSQL_ROOT_PASSWORD=acp -e MYSQL_DATABASE=ac_test --tmpfs /var/lib/mysql/:rw,noexec,nosuid ${IMAGE_MYSQL} >/dev/null
waitFor mysql-ac-${SUFFIX} 3306
PREPAREPARAMS="-e TYPO3_DB_DRIVER=mysqli -e TYPO3_DB_DBNAME=ac_test -e TYPO3_DB_USERNAME=root -e TYPO3_DB_PASSWORD=acp -e TYPO3_DB_HOST=mysql-ac-${SUFFIX} -e TYPO3_DB_PORT=3306"
TESTPARAMS="-e typo3DatabaseName=ac_test -e typo3DatabaseUsername=root -e typo3DatabasePassword=funcp -e typo3DatabaseHost=mysql-ac-${SUFFIX}"
;;
postgres)
${CONTAINER_BIN} run --rm ${CI_PARAMS} --name postgres-ac-${SUFFIX} --network ${NETWORK} -d -e POSTGRES_DB=ac_test -e POSTGRES_PASSWORD=acp -e POSTGRES_USER=ac_test --tmpfs /var/lib/postgresql/data:rw,noexec,nosuid ${IMAGE_POSTGRES} >/dev/null
waitFor postgres-ac-${SUFFIX} 5432
PREPAREPARAMS="-e TYPO3_DB_DRIVER=postgres -e TYPO3_DB_DBNAME=ac_test -e TYPO3_DB_USERNAME=ac_test -e TYPO3_DB_PASSWORD=acp -e TYPO3_DB_HOST=postgres-ac-${SUFFIX} -e TYPO3_DB_PORT=5432"
TESTPARAMS="-e typo3DatabaseDriver=pdo_pgsql -e typo3DatabaseName=ac_test -e typo3DatabaseUsername=ac_test -e typo3DatabasePassword=acp -e typo3DatabaseHost=postgres-ac-${SUFFIX}"
;;
sqlite)
PREPAREPARAMS="-e TYPO3_DB_DRIVER=sqlite"
TESTPARAMS="-e typo3DatabaseDriver=pdo_sqlite"
;;
esac

${CONTAINER_BIN} run ${CONTAINER_COMMON_PARAMS} --name acceptance-prepare ${XDEBUG_MODE} -e COMPOSER_CACHE_DIR=${CORE_ROOT}/.cache/composer -e COMPOSER_ROOT_VERSION=${COMPOSER_ROOT_VERSION} -e XDEBUG_CONFIG="${XDEBUG_CONFIG}" ${PREPAREPARAMS} ${IMAGE_PHP} "${CORE_ROOT}/Build/Scripts/setupAcceptanceComposer.sh" "typo3temp/var/tests/acceptance-composer"
SUITE_EXIT_CODE=$?
if [[ ${SUITE_EXIT_CODE} -eq 0 ]]; then
CODECEPION_ENV="--env ci,composer"
if [ "${ACCEPTANCE_HEADLESS}" -eq 1 ]; then
CODECEPION_ENV="--env ci,composer,headless"
fi
if [ "${CHUNKS}" -gt 0 ]; then
${CONTAINER_BIN} run ${CONTAINER_COMMON_PARAMS} --name ac-splitter-${SUFFIX} ${IMAGE_PHP} php -dxdebug.mode=off Build/Scripts/splitAcceptanceTests.php -v ${CHUNKS}
COMMAND=(bin/codecept run Application -d -g AcceptanceTests-Job-${THISCHUNK} -c typo3/sysext/core/Tests/codeception.yml ${EXTRA_TEST_OPTIONS} ${CODECEPION_ENV} "$@" --html reports.html)
else
COMMAND=(bin/codecept run Application -d -c typo3/sysext/core/Tests/codeception.yml ${EXTRA_TEST_OPTIONS} ${CODECEPION_ENV} "$@" --html reports.html)
fi
SELENIUM_GRID=""
if [ "${ACCEPTANCE_HEADLESS}" -eq 0 ]; then
SELENIUM_GRID="-p 7900:7900 -e SE_VNC_NO_PASSWORD=1 -e VNC_NO_PASSWORD=1"
fi
APACHE_OPTIONS="-e APACHE_RUN_USER=#${HOST_UID} -e APACHE_RUN_SERVERNAME=web -e APACHE_RUN_GROUP=#${HOST_PID} -e APACHE_RUN_DOCROOT=${CORE_ROOT}/typo3temp/var/tests/acceptance-composer/public -e PHPFPM_HOST=phpfpm -e PHPFPM_PORT=9000"
${CONTAINER_BIN} run --rm ${CI_PARAMS} -d ${SELENIUM_GRID} --name ac-chrome-${SUFFIX} --network ${NETWORK} --network-alias chrome --tmpfs /dev/shm:rw,nosuid,nodev,noexec ${IMAGE_SELENIUM} >/dev/null
if [ ${CONTAINER_BIN} = "docker" ]; then
${CONTAINER_BIN} run --rm -d --name ac-phpfpm-${SUFFIX} --network ${NETWORK} --network-alias phpfpm --add-host "${CONTAINER_HOST}:host-gateway" ${USERSET} -e PHPFPM_USER=${HOST_UID} -e PHPFPM_GROUP=${HOST_PID} -v ${CORE_ROOT}:${CORE_ROOT} ${IMAGE_PHP} php-fpm ${PHP_FPM_OPTIONS} >/dev/null
${CONTAINER_BIN} run --rm -d --name ac-web-${SUFFIX} --network ${NETWORK} --network-alias web --add-host "${CONTAINER_HOST}:host-gateway" -v ${CORE_ROOT}:${CORE_ROOT} ${APACHE_OPTIONS} ${IMAGE_APACHE} >/dev/null
else
${CONTAINER_BIN} run --rm ${CI_PARAMS} -d --name ac-phpfpm-${SUFFIX} --network ${NETWORK} --network-alias phpfpm ${USERSET} -e PHPFPM_USER=0 -e PHPFPM_GROUP=0 -v ${CORE_ROOT}:${CORE_ROOT} ${IMAGE_PHP} php-fpm -R ${PHP_FPM_OPTIONS} >/dev/null
${CONTAINER_BIN} run --rm ${CI_PARAMS} -d --name ac-web-${SUFFIX} --network ${NETWORK} --network-alias web -v ${CORE_ROOT}:${CORE_ROOT} ${APACHE_OPTIONS} ${IMAGE_APACHE} >/dev/null
fi
waitFor chrome 4444
waitFor chrome 7900
waitFor web 80
if [ "${ACCEPTANCE_HEADLESS}" -eq 0 ] && type "xdg-open" >/dev/null; then
xdg-open http://localhost:7900/?autoconnect=1 >/dev/null
elif [ "${ACCEPTANCE_HEADLESS}" -eq 0 ] && type "open" >/dev/null; then
open http://localhost:7900/?autoconnect=1 >/dev/null
fi

${CONTAINER_BIN} run ${CONTAINER_COMMON_PARAMS} --name ac-${DBMS}-composer ${XDEBUG_MODE} -e XDEBUG_CONFIG="${XDEBUG_CONFIG}" ${TESTPARAMS} ${IMAGE_PHP} "${COMMAND[@]}"
SUITE_EXIT_CODE=$?
fi
;;
acceptanceInstall)
SELENIUM_GRID=""
if [ "${ACCEPTANCE_HEADLESS}" -eq 0 ]; then
Expand Down Expand Up @@ -1009,7 +1079,7 @@ echo "##########################################################################
echo "Result of ${TEST_SUITE}" >&2
echo "Container runtime: ${CONTAINER_BIN}" >&2
echo "PHP: ${PHP_VERSION}" >&2
if [[ ${TEST_SUITE} =~ ^(functional|functionalDeprecated|acceptance|acceptanceInstall)$ ]]; then
if [[ ${TEST_SUITE} =~ ^(functional|functionalDeprecated|acceptance|acceptanceComposer|acceptanceInstall)$ ]]; then
case "${DBMS}" in
mariadb|mysql|postgres)
echo "DBMS: ${DBMS} version ${DBMS_VERSION} driver ${DATABASE_DRIVER}" >&2
Expand Down
50 changes: 50 additions & 0 deletions Build/Scripts/setupAcceptanceComposer.sh
@@ -0,0 +1,50 @@
#!/bin/sh

set -e

cd "$(dirname $(realpath $0))/../../"

PROJECT_PATH=${1:-typo3temp/var/tests/acceptance-composer/}
export TYPO3_DB_DRIVER=${2:-${TYPO3_DB_DRIVER:-sqlite}}
EXTRA_PACKAGES="${3}"

mkdir -p "${PROJECT_PATH}"
ln -snf $(echo "${PROJECT_PATH}" | sed -e 's/[^\/][^\/]*/../g' -e 's/\/$//')/typo3/sysext "${PROJECT_PATH}/typo3-sysext"
ln -snf $(echo "${PROJECT_PATH}" | sed -e 's/[^\/][^\/]*/../g' -e 's/\/$//')/Build/tests/packages "${PROJECT_PATH}/packages"
sed 's/..\/..\/typo3\/sysext/typo3-sysext/' Build/composer/composer.dist.json > "${PROJECT_PATH}/composer.json"

cd "${PROJECT_PATH}"
rm -rf composer.lock config/ public/ var/ vendor/

mkdir -p "config/system/"
cat > "config/system/additional.php" <<\EOF
<?php
$GLOBALS['TYPO3_CONF_VARS']['BE']['debug'] = true;
// "temporary password"
$GLOBALS['TYPO3_CONF_VARS']['BE']['installToolPassword'] = '$argon2i$v=19$m=65536,t=16,p=1$Rk9Edk1UWTd1MUtVY1Nydg$bJJgiAH3NT66LkvcTsnYbQvFS/ePOw/50rYjhxUk8L8';
$GLOBALS['TYPO3_CONF_VARS']['SYS']['displayErrors'] = true;
$GLOBALS['TYPO3_CONF_VARS']['SYS']['devIPmask'] = '*';
$GLOBALS['TYPO3_CONF_VARS']['SYS']['exceptionalErrors'] = E_ALL;
$GLOBALS['TYPO3_CONF_VARS']['SYS']['errorHandlerErrors'] = E_ALL;
$GLOBALS['TYPO3_CONF_VARS']['SYS']['trustedHostsPattern'] = '.*';
$GLOBALS['TYPO3_CONF_VARS']['GFX']['processor'] = 'GraphicsMagick';
$GLOBALS['TYPO3_CONF_VARS']['MAIL']['transport'] = 'mbox';
$GLOBALS['TYPO3_CONF_VARS']['MAIL']['transport_mbox_file'] = \TYPO3\CMS\Core\Core\Environment::getVarPath() . '/log/mail.mbox';
EOF

# `composer require` will implicitly perform an initial `composer install` since there is no composer.lock
composer require --no-progress --no-interaction --dev typo3tests/dataset-import:@dev typo3/testing-framework:dev-main ${EXTRA_PACKAGES}

TYPO3_SERVER_TYPE=apache \
TYPO3_PROJECT_NAME="New TYPO3 site" \
vendor/bin/typo3 setup --force --no-interaction

vendor/bin/typo3 dataset:import vendor/typo3/cms-core/Tests/Acceptance/Fixtures/BackendEnvironment.csv
vendor/bin/typo3 styleguide:generate -c -- all

# Create favicon.ico to suppress potential javascript errors in console
# which are caused by calling a non html in the browser, e.g. seo sitemap xml
ln -snf ../vendor/typo3/cms-backend/Resources/Public/Icons/favicon.ico public/favicon.ico

# @todo: needed for ugly InstallTool tests, that should be replace by a CLI command that properly enables install tool, both in composer and classic mode
mkdir -p var/transient/
6 changes: 4 additions & 2 deletions Build/composer/composer.dist.json
Expand Up @@ -49,7 +49,8 @@
"typo3/cms-sys-note": "13.1.x-dev",
"typo3/cms-t3editor": "13.1.x-dev",
"typo3/cms-tstemplate": "13.1.x-dev",
"typo3/cms-viewpage": "13.1.x-dev"
"typo3/cms-viewpage": "13.1.x-dev",
"typo3/cms-workspaces": "13.1.x-dev"
}
}
},
Expand Down Expand Up @@ -92,6 +93,7 @@
"typo3/cms-sys-note": "@dev",
"typo3/cms-t3editor": "@dev",
"typo3/cms-tstemplate": "@dev",
"typo3/cms-viewpage": "@dev"
"typo3/cms-viewpage": "@dev",
"typo3/cms-workspaces": "@dev"
}
}
2 changes: 2 additions & 0 deletions Build/gitlab-ci.yml
Expand Up @@ -41,6 +41,7 @@ include:
# turns this into a branch 'change-patchset' which executes the pipeline
- local: 'Build/gitlab-ci/pre-merge/acceptance-install.yml'
- local: 'Build/gitlab-ci/pre-merge/acceptance-application.yml'
- local: 'Build/gitlab-ci/pre-merge/acceptance-application-composer.yml'
- local: 'Build/gitlab-ci/pre-merge/integrity.yml'
- local: 'Build/gitlab-ci/pre-merge/functional.yml'
- local: 'Build/gitlab-ci/pre-merge/unit.yml'
Expand All @@ -49,4 +50,5 @@ include:
- local: 'Build/gitlab-ci/nightly/unit.yml'
- local: 'Build/gitlab-ci/nightly/acceptance-install.yml'
- local: 'Build/gitlab-ci/nightly/acceptance-application.yml'
- local: 'Build/gitlab-ci/nightly/acceptance-application-composer.yml'
- local: 'Build/gitlab-ci/nightly/functional.yml'
163 changes: 163 additions & 0 deletions Build/gitlab-ci/nightly/acceptance-application-composer.yml
@@ -0,0 +1,163 @@
acceptance application composer mariadb 10.4 php 8.2 min:
stage: acceptance
tags:
- metal2
needs: []
only:
- schedules
cache:
key: main-composer-min
paths:
- .cache
artifacts:
when: on_failure
paths:
- typo3temp/var/tests/acceptance-composer/var/log
- typo3temp/var/tests/AcceptanceReports
parallel: 8
script:
- Build/Scripts/runTests.sh -s composerInstallMin -p 8.2
- Build/Scripts/runTests.sh -s acceptanceComposer -d mariadb -i 10.4 -p 8.2 -c $CI_NODE_INDEX/$CI_NODE_TOTAL
acceptance application composer mariadb 10.10 php 8.3 max:
stage: acceptance
tags:
- metal2
needs: []
only:
- schedules
cache:
key: main-composer-max
paths:
- .cache
artifacts:
when: on_failure
paths:
- typo3temp/var/tests/acceptance-composer/var/log
- typo3temp/var/tests/AcceptanceReports
parallel: 8
script:
- Build/Scripts/runTests.sh -s composerInstallMax -p 8.3
- Build/Scripts/runTests.sh -s acceptanceComposer -d mariadb -i 10.10 -p 8.3 -c $CI_NODE_INDEX/$CI_NODE_TOTAL

acceptance application composer mysql 8.0 php 8.3 max:
stage: acceptance
tags:
- metal2
needs: []
only:
- schedules
cache:
key: main-composer-max
paths:
- .cache
artifacts:
when: on_failure
paths:
- typo3temp/var/tests/acceptance-composer/var/log
- typo3temp/var/tests/AcceptanceReports
parallel: 8
script:
- Build/Scripts/runTests.sh -s composerInstallMax -p 8.3
- Build/Scripts/runTests.sh -s acceptanceComposer -d mysql -i 8.0 -p 8.3 -c $CI_NODE_INDEX/$CI_NODE_TOTAL
acceptance application composer mysql 8.0 php 8.2 min:
stage: acceptance
tags:
- metal2
needs: []
only:
- schedules
cache:
key: main-composer-min
paths:
- .cache
artifacts:
when: on_failure
paths:
- typo3temp/var/tests/acceptance-composer/var/log
- typo3temp/var/tests/AcceptanceReports
parallel: 8
script:
- Build/Scripts/runTests.sh -s composerInstallMin -p 8.2
- Build/Scripts/runTests.sh -s acceptanceComposer -d mysql -i 8.0 -p 8.2 -c $CI_NODE_INDEX/$CI_NODE_TOTAL

acceptance application composer sqlite php 8.3 max:
stage: acceptance
tags:
- metal2
needs: []
only:
- schedules
cache:
key: main-composer-max
paths:
- .cache
artifacts:
when: on_failure
paths:
- typo3temp/var/tests/acceptance-composer/var/log
- typo3temp/var/tests/AcceptanceReports
parallel: 8
script:
- Build/Scripts/runTests.sh -s composerInstallMax -p 8.3
- Build/Scripts/runTests.sh -s acceptanceComposer -d sqlite -p 8.3 -c $CI_NODE_INDEX/$CI_NODE_TOTAL
acceptance application composer sqlite php 8.2 min:
stage: acceptance
tags:
- metal2
needs: []
only:
- schedules
cache:
key: main-composer-min
paths:
- .cache
artifacts:
when: on_failure
paths:
- typo3temp/var/tests/acceptance-composer/var/log
- typo3temp/var/tests/AcceptanceReports
parallel: 8
script:
- Build/Scripts/runTests.sh -s composerInstallMin -p 8.2
- Build/Scripts/runTests.sh -s acceptanceComposer -d sqlite -p 8.2 -c $CI_NODE_INDEX/$CI_NODE_TOTAL

acceptance application composer postgres 15 php 8.3 max:
stage: acceptance
tags:
- metal2
needs: []
only:
- schedules
cache:
key: main-composer-max
paths:
- .cache
artifacts:
when: on_failure
paths:
- typo3temp/var/tests/acceptance-composer/var/log
- typo3temp/var/tests/AcceptanceReports
parallel: 8
script:
- Build/Scripts/runTests.sh -s composerInstallMax -p 8.3
- Build/Scripts/runTests.sh -s acceptanceComposer -d postgres -i 15 -p 8.3 -c $CI_NODE_INDEX/$CI_NODE_TOTAL
acceptance application composer postgres 10 php 8.2 min:
stage: acceptance
tags:
- metal2
needs: []
only:
- schedules
cache:
key: main-composer-min
paths:
- .cache
artifacts:
when: on_failure
paths:
- typo3temp/var/tests/acceptance-composer/var/log
- typo3temp/var/tests/AcceptanceReports
parallel: 8
script:
- Build/Scripts/runTests.sh -s composerInstallMin -p 8.2
- Build/Scripts/runTests.sh -s acceptanceComposer -d postgres -i 10 -p 8.2 -c $CI_NODE_INDEX/$CI_NODE_TOTAL
17 changes: 17 additions & 0 deletions Build/gitlab-ci/pre-merge/acceptance-application-composer.yml
@@ -0,0 +1,17 @@
acceptance application composer postgres 15 php 8.2 pre-merge:
stage: main
tags:
- metal2
except:
refs:
- schedules
- main
artifacts:
when: on_failure
paths:
- typo3temp/var/tests/acceptance-composer/var/log
- typo3temp/var/tests/AcceptanceReports
parallel: 13
script:
- Build/Scripts/runTests.sh -s composerInstall -p 8.2
- Build/Scripts/runTests.sh -s acceptanceComposer -p 8.2 -d postgres -i 15 -c $CI_NODE_INDEX/$CI_NODE_TOTAL

0 comments on commit fa6c3fc

Please sign in to comment.