-
Notifications
You must be signed in to change notification settings - Fork 685
2.2.0 Test Plan
For both upgrades and fresh installs, here is a list of functionality that requires testing. You can use this for copy/pasting into your QA report.
If you have submitted a QA report already for a 2.2.0 release candidate with successful basic server testing and application acceptance testing sections, then you can skip these sections in subsequent reports, unless otherwise indicated by the Release Manager. This is to ensure that you focus your QA effort on the release-specific changes as well as changes since the previous release candidate.
If you are testing the upgrade scenario, you should create source and journalist accounts before performing the upgrade, and delete at least one journalist account - some test cases cover changes to the behaviour for journalist account deletion. See the test cases for #6225
below for more information.
- Install target:
- Tails version:
- Test Scenario:
- SSH over Tor:
- Release candidate:
- General notes:
- After installing the testinfra dependencies, all tests in
./securedrop-admin verify
are passing:- Install dependencies on Admin Workstation with
cd ~/Persistent/securedrop && ./securedrop-admin setup -t
- Run tests with
./securedrop-admin verify
(this will take a while) - Remove test dependencies:
rm -rf admin/.venv3/ && ./securedrop-admin setup
- Install dependencies on Admin Workstation with
- QA Matrix checks pass
- Can successfully add admin user and login
- I have backed up and successfully restored the app server following the backup documentation
- If doing upgrade testing, make a backup on 2.1.0 and restore this backup on this release candidate
- "Send Test OSSEC Alert" button in the journalist triggers an OSSEC alert and an email is sent
- Can successfully add journalist account with HOTP authentication
- JS warning bar does not appear when using Security Slider high
- JS warning bar does appear when using Security Slider Low
- On generate page, refreshing page produces a new 7-word codename
- On submit page, empty submissions produce flashed message
- On submit page, short message submitted successfully
- On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
- On submit page, file less than 500 MB submitted successfully
- Nonexistent codename cannot log in
- Empty codename cannot log in
- Legitimate codename can log in
- Returning user can view journalist replies - need to log into journalist interface to test
- Can log in with 2FA tokens
- incorrect password cannot log in
- invalid 2fa token cannot log in
- 2fa immediate reuse cannot log in
- Journalist account with HOTP can log in
- Filter by codename works
- Starring and unstarring works
- Click select all selects all submissions
- Selecting all and clicking "Download" works
- You can submit a reply and a flashed message and new row appears
- You cannot submit an empty reply
- Clicking "Delete Source Account" and the source and docs are deleted
- You can click on a document and successfully decrypt using application private key
After updating to this release candidate and running securedrop-admin tailsconfig
- The Updater GUI appears on boot
- Updating to 2.1.0 is successful
-
#6225 Use special "deleted" journalist for associations with deleted users
-
If testing an upgrade scenario:
- before performing the upgrade, create 2 JI users, (
admin1
andadmin2
, say) - on the SI, create a new source with at least one file submission and one message submission
- log on to the JI as
admin1
, download the submissions and reply to the source - verify that the
seen_replies
,seen_files
, andseen_messages
table recordsadmin1
's ID - log out of the JI, log back in as
admin2
, and deleteadmin1
- verify that the
seen_replies
,seen_files
, andseen_messages
table records that referencedadmin1
's id now have null values for the journalist id. - Perform the upgrade to the 2.2.0 release candidate
- verify that a
deleted
user was created in thejournalists
table - verify that the entries in the
seen_*
tables referencing null journalist IDs now referencedeleted
's ID - verify that the Journalist API
/replies/<reply id
endpoint for the reply lists thedeleted
s user's UUID
- before performing the upgrade, create 2 JI users, (
-
If testing an install scenario:
- create JI 2 admin users (
admin1
andadmin2
, say) - verify that only those 2 users exist in the
journalists
table - on the SI, create a new source with at least one file submission and one message submission
- log on to the JI as
admin1
, download the submissions and reply to the source - verify that the
seen_replies
,seen_files
, andseen_messages
table recordsadmin1
's ID - log out of the JI, log back in as
admin2
, and deleteadmin1
- verify that a
deleted
user was created in thejournalists
table, andadmin1
is no longer present - verify that the entries in the
seen_*
tables referencingadmin1
's ID now referencedeleted
's ID - verify that the Journalist API
/replies/<reply id
endpoint for the reply lists thedeleted
s user's UUID
- create JI 2 admin users (
-
-
#6222 Enforce 160-bit HOTP secret length and verify OTP secret length on login.
- If testing an upgrade scenario (existing 80-bits):
- log into the JI with an existing HOTP-enabled user and verify that the OTP code is successfully ... verified
- log out, updated the user's otp secret with a valid secret shorter than 16 chars (with, eg.
UPDATE journalists SET otp_secret='JHCOGO7' WHERE username='<your username>';
, and verify that login now fails with an error likeLogin failed. Your 2FA details are invalid...
- log in as an admin user and reset HOTP for the user above
- Confirm that the user can log in
- Confirm that their otp_secret value is now 32 chars long
- For both fresh installs and upgrades:
- log in to the JI as an admin user, go to the Admin section and select Add User
- verify that if is using a Yubikey [HOTP] is checked but no HOTP secret is specified, adding a user fails with a form validation message like
The "opt_secret" field is required...
- verify that if is using a Yubikey [HOTP] is checked but a HOTP secret shorter than 40 chars is specified, adding a user fails with a form validation message like
HOTP secrets are 40 chars long...
- verify that if is using a Yubikey [HOTP] is checked but a non-hexadecimal 40-char HOTP secret is specified , adding a user fails with a flash message like
Invalid HOTP secret format:...
- verify that if is using a Yubikey [HOTP] is checked and a hexadecimal 40-char HOTP secret is specified , adding a user succeeds and the secret can be used to generate a valid token on the next page.
- verify that if is using a Yubikey [HOTP] is checked but no HOTP secret is specified, adding a user fails with a form validation message like
- log in to the JI as an admin user, go to the Admin section and select Add User
- If testing an upgrade scenario (existing 80-bits):
-
#6195 Remove "Refresh codename" feature
- Verify that the codename refresh button is no longer displayed on
/generate
- Verify that reloading the page generates a new codename, and that that new codename is displayed in the codename reminder after clicking through to
/lookup
- Verify that the codename refresh button is no longer displayed on
-
#6130 Display codename reminder only on first login
- Create a new sourc on the SI, submit a file, and verify that the codename reminder is displayed
- Log out, then log in again as the same source. Verify that the codename reminder is not displayed.
- Submit a new message as the source, and verify that the codename remonder is not displayed when the page reloads.
-
#6187 Adds SI and JI header updates
-
On app, verify that
/etc/apache2/sites-enabled/source.conf
contains the Header directives:Header onsuccess unset X-Frame-Options Header always set X-Frame-Options "DENY" Header onsuccess unset Referrer-Policy Header always set Referrer-Policy "same-origin" Header onsuccess unset Cross-Origin-Opener-Policy Header always set Cross-Origin-Opener-Policy "same-origin" Header onsuccess unset Cross-Origin-Embedder-Policy Header always set Cross-Origin-Embedder-Policy "same-origin" Header onsuccess unset Cross-Origin-Resource-Policy Header always set Cross-Origin-Resource-Policy "same-site" # Set a strict CSP; "default-src 'self'" prevents 3rd party subresources from # loading and prevents inline script from executing. Header onsuccess unset Content-Security-Policy Header always set Content-Security-Policy "default-src 'none'; script-src 'self'; style-src 'self'; img-src 'self'; font-src 'self'; frame-ancestors 'none';" Header onsuccess unset X-Content-Type-Options Header always set X-Content-Type-Options "nosniff"
-
On app, verify that
/etc/apache2/sites-enabled/journalist.conf
contains the Header directives:Header onsuccess unset X-Frame-Options Header always set X-Frame-Options "DENY" Header onsuccess unset Referrer-Policy Header always set Referrer-Policy "no-referrer" Header onsuccess unset Cross-Origin-Opener-Policy Header always set Cross-Origin-Opener-Policy "same-origin" Header onsuccess unset Cross-Origin-Embedder-Policy Header always set Cross-Origin-Embedder-Policy "same-origin" Header onsuccess unset Cross-Origin-Resource-Policy Header always set Cross-Origin-Resource-Policy "same-site" Header onsuccess unset X-Content-Type-Options Header always set X-Content-Type-Options "nosniff" Header onsuccess unset Content-Security-Policy Header always set Content-Security-Policy "default-src 'none'; script-src 'self'; style-src 'self'; img-src 'self'; font-src 'self'; frame-ancestors 'none';"
-
-
#6242 Updated expected grsec kernel version to 5.15.18
- On app and mon, verify that
uname -r
returns5.15.18-grsec-securedrop
- On app and mon, verify that
-
#6227 Updated Tor version to 0.4.6.9
- On app and mon, verify that
tor --version
indicates that the Tor version is0.4.6.9
- On app and mon, verify that
-
#6174 Disable caching of GPG passphrases
- On app, verify that the file
/var/lib/securedrop/keys/gpg-agent.conf
contains the lines:allow-loopback-pinentry default-cache-ttl 0
- On app, verify that the file
- Install or upgrade occurs without error
- Source interface is available and version string indicates it is 2.2.0
- A message can be successfully submitted
- The updater GUI appears on boot
- The update successfully occurs to 2.2.0
- After reboot, updater GUI no longer appears