Skip to content

Update testing #1080

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 74 commits into from
Jun 18, 2025
Merged
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
839deb3
add upgrade pipeline
michield Dec 29, 2023
1824f09
rename pipeline
michield Dec 29, 2023
40f4ebf
add basic 3.6 DB
michield Dec 29, 2023
a125c7a
update version
michield Dec 29, 2023
54becd4
Add pipeline for upgrade from 3.6 to 3.7 testing (#1009)
michield Dec 30, 2023
cbabebb
update local test setup
michield Jan 1, 2024
72eb575
exclude the behat test tag on CI
michield Jan 1, 2024
9913958
exclude test output
michield Jan 1, 2024
c21d4f4
add composer.json
michield Jan 1, 2024
d411a93
fix deprecations
michield Jan 1, 2024
4863b53
testing updates
michield Jan 1, 2024
0996420
remove old file
michield Jan 1, 2024
e9a2595
add two new feature files
michield Jan 1, 2024
8989c91
ignore .env files
michield Jan 1, 2024
c63c95b
add smtp-sink
michield Jan 2, 2024
8c50ff5
Merge branch 'main' into update-testing
michield Jan 3, 2024
d05bba9
simplify constructor
michield Jan 7, 2024
6b2b184
add plugin tests
michield Jan 7, 2024
b4d0dcc
remove .disabled file
michield Jan 7, 2024
8bc1afe
behat tests update
michield Jan 7, 2024
098f1d1
rename feature file
michield Jan 7, 2024
c3963a4
add more tests
michield Jan 7, 2024
1d5d9a8
tidy up
michield Jan 7, 2024
4b9484b
remove database argument
michield Jan 7, 2024
1528af4
set path for files
michield Jan 7, 2024
505cd26
copy screenshots
michield Jan 7, 2024
5a29b0a
set to manually run the queue
michield Jan 21, 2024
93c2f63
Merge branch 'release-3.7' into update-testing
michield Jan 21, 2024
8925093
add smtp-sink
michield Jan 21, 2024
28c3f1e
set pageroot to avoid warning
michield Jan 21, 2024
95cab8a
set pageroot back. stops everything from working
michield Jan 21, 2024
b33254e
set some timeouts and add step to refresh the page
michield May 25, 2025
4049bca
add refresh the page
michield May 25, 2025
ad87dd2
Merge branch 'release-3.7.0' into update-testing
michield May 27, 2025
bddbc49
add php 8.4
michield May 27, 2025
0f07768
fix parse errors
michield May 27, 2025
55eb57e
mark as WIP
michield May 27, 2025
8eed4fb
create directory for screenshots
michield May 27, 2025
c660625
show progress
michield May 27, 2025
d49a8cf
remove duplicated action
michield May 27, 2025
c653960
add REST API
michield May 27, 2025
1854b11
lint exclude
tatevikg1 May 31, 2025
0a863d3
lint update + split
tatevikg1 May 31, 2025
06d069d
update phplint
michield Jun 7, 2025
f3eb24f
upadte matrix values
michield Jun 7, 2025
9c9151b
increase timeouts
michield Jun 7, 2025
16bb0f0
make 8.0 unsupported and 8.3 standard
michield Jun 7, 2025
b0a7e5b
add clicking save twice
michield Jun 7, 2025
e66bde5
try with firefox to avoid stale element error
michield Jun 7, 2025
785d61f
go back to chrome
michield Jun 7, 2025
29b3a5f
change should see into must see
michield Jun 8, 2025
dc147fc
decrease wait time
michield Jun 8, 2025
94f064a
use debug version to enable VNC
michield Jun 8, 2025
860ab69
set pageroot for CI
michield Jun 8, 2025
72df3a9
show the tab panel with the list created
michield Jun 8, 2025
5af687e
remove timeouts from featurecontext, they are in behat.yml already
michield Jun 9, 2025
2c21fe3
the panel is not there
michield Jun 9, 2025
8298296
mark ready
michield Jun 9, 2025
239cabd
remove javascript tag
michield Jun 9, 2025
24fe0b2
remove timeouts, add chrome to local testing
michield Jun 9, 2025
517bd7f
add chrome to local testing
michield Jun 9, 2025
caaded0
remove javascript tag
michield Jun 9, 2025
7d7cfc0
remove pageroot setting
michield Jun 9, 2025
1cc60b8
upload log as artifact
michield Jun 12, 2025
ba0e62a
mark some failing ones as wip
michield Jun 12, 2025
b76851f
makr another wip
michield Jun 12, 2025
d856ad1
typo
michield Jun 12, 2025
63b8c9e
don't run all the time
michield Jun 12, 2025
63aa7d1
anoteht typo
michield Jun 12, 2025
717629e
makr another one @wip.
michield Jun 12, 2025
81a59da
create volume to map local code into the container
michield Jun 12, 2025
66c4d4c
give example of env var
michield Jun 12, 2025
c1dda24
update README on testing
michield Jun 12, 2025
2a10642
tidy up
michield Jun 18, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/workflows/build-release.yml
Original file line number Diff line number Diff line change
@@ -38,7 +38,7 @@ jobs:
- name: Setup Packages
run: |
cd $GITHUB_WORKSPACE
sudo apt update && sudo apt install -y rsync
sudo apt update && sudo apt install -y rsync postfix

- name: Cache Composer dependencies
uses: actions/cache@v4
@@ -89,6 +89,7 @@ jobs:
mkdir /tmp/screenshots
chmod 777 /tmp/screenshots
mkdir -p build/mails
smtp-sink -c -d "output/mails/%Y%m%d%H/%M." 2500 1024 &
./bin/start-selenium > output/selenium.log 2>&1 &
sleep 5
sudo php -S 0.0.0.0:80 -t public_html > /dev/null 2>&1 &
@@ -119,8 +120,8 @@ jobs:
- name: Run Behat Tests ## problem encountered https://github.com/forceedge01/behat-fail-aid/issues/76
run: |
cd $GITHUB_WORKSPACE/tests
../vendor/bin/behat -n -p chrome --strict --stop-on-failure --tags=@initialise
../vendor/bin/behat -n -p chrome --strict --stop-on-failure --tags="~@initialise && ~@wip"
../vendor/bin/behat -n -fprogress -p chrome --strict --stop-on-failure --tags=@initialise
../vendor/bin/behat -n -fprogress -p chrome --strict --stop-on-failure --tags="~@initialise && ~@wip && ~@behattest"

- name: Upload the output
if: always()
79 changes: 59 additions & 20 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@

name: CI

on: [pull_request]
on: [pull_request, push]

jobs:
test:
@@ -11,10 +11,18 @@ jobs:
strategy:
fail-fast: false
matrix:
php-version: ['7.4', '8.0', '8.1', '8.2']
php-version: ['8.1', '8.2', '8.3']
experimental: [false]
unsupported: [false]
include:
- php-version: '8.3'
- php-version: '7.4'
unsupported: true
experimental: false
- php-version: '8.0'
unsupported: true
experimental: false
- php-version: '8.4'
unsupported: false
experimental: true

steps:
@@ -30,7 +38,7 @@ jobs:
- name: Setup Packages
run: |
sudo apt update
sudo apt install -y rsync
sudo apt -y install rsync postfix

- name: Install dependencies
uses: php-actions/composer@v6
@@ -59,10 +67,8 @@ jobs:
- name: Create Database
run: |
sudo systemctl start mysql.service
sudo mysql -proot -e 'DROP DATABASE IF EXISTS phplistdb'
sudo mysqladmin -proot create phplistdb
sudo mysql -proot -e 'CREATE USER phplist@"%" IDENTIFIED BY "phplist"; GRANT ALL ON phplistdb.* TO phplist@"%"'
continue-on-error: ${{ matrix.experimental }}
sudo mysql -proot -e 'drop database if exists phplistdb; create database phplistdb;'
sudo mysql -proot -e 'create user phplist@"%" identified by "phplist"; grant all on phplistdb.* to phplist@"%"'

- name: Set Bootlist Theme
run: |
@@ -73,29 +79,52 @@ jobs:
rm master.tar.gz
continue-on-error: ${{ matrix.experimental }}

- name: Install the phpList4 based REST API
run: |
cd $GITHUB_WORKSPACE
cd public_html/lists
wget https://github.com/phpList/base-distribution/archive/refs/heads/main.zip
unzip main.zip
rm main.zip
mv base-distribution-main base
cd base
composer update --no-dev
if: ${{ matrix.unsupported == false }}

- name: Start Test Server
run: |
cp -fv tests/ci/behat.yml tests/behat.yml
cp -fv tests/ci/config.php public_html/lists/config/config.php
mkdir -p output/screenshots
mkdir -p build/mails
chmod -R 777 output/screenshots build/mails
mkdir -p /tmp/screenshots/
touch output/screenshots/README.md
mkdir -p output/mails
smtp-sink -c -d "output/mails/%Y%m%d%H/%M." 2500 1024 &
./bin/start-selenium > output/selenium.log 2>&1 &
sleep 15
sudo php -S 0.0.0.0:80 -t public_html > /dev/null 2>&1 &
continue-on-error: ${{ matrix.experimental }}

- name: Check PHP syntax errors
uses: overtrue/phplint@2.4.1
uses: overtrue/phplint@3.0.0
with:
path: ./public_html
options: --exclude=lists/base
if: ${{ matrix.unsupported == false }}
continue-on-error: ${{ matrix.experimental }}

- name: Check PHP syntax errors in API
uses: overtrue/phplint@9.5.0
with:
path: ./public_html/lists/base/vendor/phplist
if: ${{ matrix.unsupported == false }}
continue-on-error: ${{ matrix.experimental }}

- name: Run BDD Tests UI
run: |
cd tests
../vendor/bin/behat -p chrome -f progress --stop-on-failure --tags=@initialise
../vendor/bin/behat -p chrome -f progress --tags="~@initialise && ~@wip"
../vendor/bin/behat -p chrome -f progress --tags="~@initialise && ~@wip && ~@behattest"
continue-on-error: ${{ matrix.experimental }}

- name: Run BDD Tests CLI
@@ -106,9 +135,17 @@ jobs:
export ADMIN_NAME="phpList Administrator"
php public_html/lists/admin/index.php -c public_html/lists/config/config.php -p initialise -f
cd tests
../vendor/bin/behat -p chrome --tags="~@initialise && ~@wip"
../vendor/bin/behat -p chrome -f progress --tags="~@initialise && ~@wip"
continue-on-error: ${{ matrix.experimental }}

- name: copy screenshots
if: always()
run: |
cd $GITHUB_WORKSPACE
images=$(ls -l /tmp/*.png 2>/dev/null | wc -l)
[[ $images -gt 0 ]] && cp -v /tmp/*.png output/screenshots
exit 0;

- name: Handle Experimental Warnings
if: ${{ matrix.experimental && failure() }}
run: |
@@ -118,12 +155,14 @@ jobs:
if: always()
uses: actions/upload-artifact@v4
with:
path: "output"
name: "behat output ${{ matrix.php-version }}"
path: "/tmp/screenshots/"
name: "screenshots ${{ matrix.php-version }}"
retention-days: 3

- name: Display Output on Failure
if: failure() && !matrix.experimental
run: |
find . -type f
cat output/selenium.log
- name: Upload the Selenium Logs
if: always()
uses: actions/upload-artifact@v4
with:
path: "output/selenium.log"
name: "selenium log ${{ matrix.php-version }}"
retention-days: 3
149 changes: 149 additions & 0 deletions .github/workflows/upgrade.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@

name: Upgrade

on:
workflow_dispatch:

jobs:
test:
runs-on: ubuntu-22.04
continue-on-error: ${{ matrix.experimental }}

strategy:
fail-fast: false
matrix:
php-version: ['8.1']
experimental: [false]
steps:
- uses: actions/checkout@v4

- name: Set up PHP ${{ matrix.php-version }}
uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php-version }}
env:
COMPOSER_TOKEN: ${{ secrets.COMPOSER_TOKEN }}

- name: Setup Packages
run: |
cd $GITHUB_WORKSPACE
sudo apt update
sudo apt install -y rsync postfix

# - name: Cache Composer dependencies
# uses: actions/cache@v2
# with:
# path: /tmp/composer-cache
# key: ${{ matrix.php-version }}-${{ hashFiles('**/composer.lock') }}

- name: Install dependencies
uses: php-actions/composer@v6
with:
dev: yes
args: --prefer-dist --no-interaction
php_version: ${{ matrix.php-version }}
php_extensions: xml
version: 2

- name: Report Versions and show ENV vars
run: |
google-chrome --version
lsb_release -a
php -v
chromedriver -v
geckodriver -V
which geckodriver
which chromedriver
firefox -v
vendor/bin/behat -V
mysql --version
mysqladmin --version
export

- name: Create Database
run: |
sudo systemctl start mysql.service
sudo mysql -proot -e 'drop database if exists phplistdb; create database phplistdb;'
sudo mysql -proot -e 'create user phplist@"%" identified by "phplist"; grant all on phplistdb.* to phplist@"%"'
sudo gunzip -c tests/phplist-3.6.sql.gz | sudo mysql -proot phplistdb

- name: Set bootlist theme
run: |
cd $GITHUB_WORKSPACE/public_html/lists/admin/ui/
wget https://github.com/phpList/phplist-ui-bootlist/archive/master.tar.gz
tar -xzf master.tar.gz
mv phplist-ui-bootlist-master phplist-ui-bootlist
rm master.tar.gz

- name: Start Test Server
run: |
cd $GITHUB_WORKSPACE
cp -fv tests/ci/behat.yml tests/behat.yml
cp -fv tests/ci/config.php public_html/lists/config/config.php
mkdir -p output/screenshots
touch output/screenshots/README.md
mkdir -p build/mails
smtp-sink -c -d "output/mails/%Y%m%d%H/%M." 2500 1024 &
./bin/start-selenium > output/selenium.log 2>&1 &
sleep 5
sudo php -S 0.0.0.0:80 -t public_html > /dev/null 2>&1 &

- name: Check PHP syntax errors
uses: overtrue/phplint@2.4.1
with:
path: ./public_html

- name: Upgrade from 3.6 to 3.7
run: |
php public_html/lists/admin/index.php -c $GITHUB_WORKSPACE/public_html/lists/config/config.php -p upgrade

- name: Run BDD Tests UI
run: |
cd $GITHUB_WORKSPACE/tests
../vendor/bin/behat -p chrome -f progress --tags="~@initialise && ~@wip"

- name: Reset Database
run: |
sudo mysql -proot -e 'drop database if exists phplistdb; create database phplistdb;'

- name: Run BDD Tests UI
run: |
cd $GITHUB_WORKSPACE/tests
../vendor/bin/behat -p chrome -f progress --stop-on-failure --tags=@initialise
../vendor/bin/behat -p chrome -f progress --tags="~@initialise && ~@wip"

- name: Run BDD Tests CLI
run: |
cd $GITHUB_WORKSPACE
export ADMIN_EMAIL=admin@phplist.dev
export ADMIN_PASSWORD=Mypassword123+
export ORGANISATION_NAME="phpList"
export ADMIN_NAME="phpList Administrator"
php public_html/lists/admin/index.php -c $GITHUB_WORKSPACE/public_html/lists/config/config.php -p initialise -f
cd $GITHUB_WORKSPACE/tests
../vendor/bin/behat -p chrome -f progress --tags="~@initialise && ~@wip"

- name: copy screenshots
if: always()
run: |
cd $GITHUB_WORKSPACE
images=$(ls -l /tmp/*.png 2>/dev/null | wc -l)
[[ $images -gt 0 ]] && cp -v /tmp/*.png output/screenshots
exit 0;

- name: Upload the screenshots
if: always()
uses: actions/upload-artifact@v4
with:
path: "output"
name: "behat output ${{ matrix.php-version }}"
retention-days: 3

- name: Display output
run: |
cd $GITHUB_WORKSPACE
# find . -type f
# cat output/selenium.log
if: ${{ failure() }}
Comment on lines +9 to +147

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI about 1 month ago

To fix the issue, we will add a permissions block at the root level of the workflow. This block will define the minimal permissions required for the workflow to function. Based on the workflow's steps, it primarily interacts with the repository contents and uploads artifacts. Therefore, we will set contents: read and actions: write as the permissions. This ensures the workflow has only the necessary access.


Suggested changeset 1
.github/workflows/upgrade.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/upgrade.yml b/.github/workflows/upgrade.yml
--- a/.github/workflows/upgrade.yml
+++ b/.github/workflows/upgrade.yml
@@ -3,2 +3,6 @@
 
+permissions:
+  contents: read
+  actions: write
+
 on: 
EOF
@@ -3,2 +3,6 @@

permissions:
contents: read
actions: write

on:
Copilot is powered by AI and may make mistakes. Always verify output.


4 changes: 1 addition & 3 deletions Dockerfile.release
Original file line number Diff line number Diff line change
@@ -3,9 +3,7 @@ FROM debian:bookworm-slim

LABEL maintainer="michiel@phplist.com"

RUN apt-get update && apt-get upgrade -y

RUN apt-get install -y apt-utils \
RUN apt-get update && apt-get upgrade -y && apt-get install -y apt-utils \
apache2 php-mysql \
libapache2-mod-php php-curl php-gd \
cron php-imap php-xml php-zip php-mbstring
Loading
Oops, something went wrong.
Loading
Oops, something went wrong.