Workstation Acceptance Tests

SecureDrop Workstation test scenarios

Some scenarios require a supported printer. We're tracking compatible printers available to team members here.

Also see https://drive.google.com/drive/u/0/folders/1lyk2V15e7amI9OxvgUK_YhYIUVNDWJo0 for different types of file submissions we'll want to test in the client and also for printing (mostly pulled from https://file-examples.com/).

Qubes scenarios

Some of these may be a bit time-consuming, so we typically have a subset of devs test these specific scenarios. In these scenarios, we try to validate the behavior of different components of the system. There's a separate section with detail on Client functionality specifically.

Verify tor connection to Journalist API

• Create VM for accessing JI via Tor Browser: qvm-create --template whonix-ws-16 --property netvm=sd-whonix --label orange sd-research. Open Tor Browser in that VM and confirm you can log in to the Journalist Interface. This confirms that sd-whonix is configured correctly (but does not use sd-proxy).
• Change the netvm to sys-whonix and confirm you can load the public Source Interface, but not the Journalist Interface. (N.B. you must leave the netvm set to sys-whonix, otherwise make clean and sdw-admin --uninstall will fail.)

Verify default DispVM settings

• Open a shell in a non-SDW VM, e.g. sd-dev. Download a PDF file and open it via: qvm-open-in-dvm <pdf_file>. Confirm it opens in a DispVM, and that the DispVM is based on sd-viewer.
• Open a shell in sd-app and find an already downloaded submission in ~/.securedrop_client/data/. Run xdg-open <file_path> and confirm it opens in a DispVM, and that the DispVM is based on sd-viewer.

RPC Policies

• Open a shell in a non-SDW VM, e.g. sd-dev. Run: QUBES_GPG_DOMAIN=sd-gpg qubes-gpg-client -k. Confirm that the request is denied, i.e. you do NOT see pubkey info for the SecureDrop Submission Key.
• Try to copy/paste from the Client to a non-SDW VM, e.g. sd-dev. Confirm you cannot.
• Add the clipboard tags to sd-dev as described in the docs, and repeat the copy/paste procedure. Confirm it works.

Logging

• Logs are sent to sd-log VM
• No warnings are displayed about logging denials (e.g. sd-log to sd-log; see https://github.com/freedomofpress/securedrop-workstation/issues/755)

GUI updater

• Reboot the workstation after installing SDW. Confirm that the prelaunch updater window appears automatically after logging, prompting for an update.
• Proceed with GUI updater, confirm it runs without errors.

Client scenarios

Some client tests do not require being run in a SecureDrop Workstation environment. The list of scenarios below is a complete list for full acceptance testing, which should be done in the supported target Qubes environment.

Scenario: Online mode

Prerequisites:

• server is available and contains source test data
• access to sd-gpg keyring has not been previously granted
• ~/.securedrop_client/data in sd-app is empty, and ~/.securedrop_client/svs.sqlite does not exist (do not delete the entire ~/.securedrop_client directory)
• the sd-devices VM is not running (shut down manually if necessary)
• a supported printer is available, but not attached.
• all VMs are up-to-date
• test instance contains several sources, including some with files & some with HTML characters in messages

Login

• when SecureDrop desktop icon is double-clicked, preflight updater is displayed
• After preflight updater runs, when user clicks Continue, login dialog is displayed
• In login dialog:
  • show/hide password functionality works
  • incorrect password cannot log in
  • 2FA token reuse cannot log in after password failure
  • invalid 2FA token cannot log in
  • valid credentials and 2FA can log in

Sources

• after valid login:
  • the login dialog closes
  • source data is downloaded and source list is populated
  • user is prompted for GPG key access
  • submissions and replies are decrypted
  • the source list is displayed but no sources are selected by default
  • the conversation view is not populated
• when a source is selected in source list:
  • conversation view is populated with source conversation
  • a source message containing HTML is displayed as unformatted text
  • source submissions have an active Download button
  • source submission compressed file size is displayed accurately
• when the upper right 3-dot button is clicked:
  • a menu is displayed with a delete source account option
  • when delete source account is selected:
    • the source is deleted from the source list and the conversation view is blanked
    • the source is deleted from the server and not restored on next sync
    • source submissions and messages are removed from the client's data directory
• when a source is starred in source list, and the client is closed and reopened in Online mode:
  • the source is still starred in the source list

Replies

• when a source is selected in the source list:
  • the reply panel is available for use and there is no message asking the user to sign in
  • a reply can be added to the conversation
  • a pending reply can be added to the conversation (for development environments, you can use: wget https://gist.githubusercontent.com/creviera/7f19a7d10334359f40dbdbb2354cd13a/raw/a2ef94913a155aa4019b753cf916f844c9cffa3a/pending-reply && git apply pending-reply then send a reply; alternatively, disconnect the network or sd-whonix after sending a reply)
  • a failed reply can be added to the conversation (for development environments, you can use: wget https://gist.githubusercontent.com/creviera/5ba70d50c12b6a6df6f98ed40ad09645/raw/5caef3339ceab1fc997ccb6b9e337bc8828ef12f/failed-reply && git apply failed-reply then send a reply; alternatively, sign out after the previous step to confirm that the reply transitions to "failed" state)
  • a reply containing HTML is displayed as unformatted text
  • a reply with a single string of characters longer than 100 chars is displayed, but truncated (https://github.com/freedomofpress/securedrop-client/issues/815).
  • a reply with a line longer than 100 chars is displayed correctly
  • two replies added immediately after each other are ordered correctly

Submissions

For sample files in different formats, see this GDrive folder.

Preview

• when Download is clicked on a submission:
  • the submission is downloaded and decrypted
  • the Download button is replaced with Print and Export options
  • the submission filename is displayed.
• For a DOC submission:
  • when the submission filename is clicked, a disposable VM (dispVM) is started.
  • after the dispVM starts, the submission is displayed in LibreOffice
  • when LibreOffice is closed, the dispVM shuts down
• For a PDF submission:
  • when the submission filename is clicked, a dispVM is started.
  • after the dispVM starts, the submission is displayed in evince
  • when evince is closed, the dispVM shuts down
• For a JPEG submission:
  • when the submission filename is clicked, a dispVM is started.
  • after the dispVM starts, the submission is displayed in Image Viewer
  • when Image Viewer is closed, the dispVM shuts down

Export

• When Export is first clicked on a submission:
  • the "Preparing to export..." message is displayed
  • the sd-devices VM is started
  • the user is prompted to insert an Export USB
  • On clicking Cancel, the prompt closes and the file is not exported
• When Export is clicked on the submission again:
  • the "Preparing to export..." message is displayed
  • the user is prompted to insert an Export USB
  • When the user inserts an invalid Export USB, attaches it to the sd-devices VM and clicks OK:
    • a message is displayed indicating that the Export USB is invalid and the user is prompted to insert a valid device
• When Export is clicked on the submission again:
  • the "Preparing to export..." message is displayed
  • the user is prompted to insert an Export USB
  • When the user inserts a valid Export USB, attaches it to the sd-devices VM, and clicks OK:
    • the user is prompted for the Export USB's password
  • When the user enters an invalid Export USB password and clicks Submit:
    • a failure message is displayed and the user is prompted to enter the password again
  • When the user enters an valid Export USB password and clicks Submit:
    • the file is saved to the Export USB
• When the user detaches the Export USB and mounts it on another VM or computer:
  • the decrypted submission is available in on the Export USB, in a directory sd-export-<timestamp>/export_data

Print

• When the user clicks Print on a downloaded submission:
  • a "Preparing to print..." message is displayed
  • the sd-devicesVM is started
  • the user is prompted to connect a supported printer
• When the user connects a printer, attaches it to the sd-devices VM, and clicks Continue:
  • a "Printing..." message is displayed
  • the X Printer Panel dialog is displayed with the printer selected
• When the user clicks Print in the X Printer Panel:
  • the submission is printed successflly.

Closing the client

• When the user clicks the main window close button:
  • the client exits.

Scenario: Offline mode without existing data

Prerequisites:

• server is available and contains source test data
• test data includes at least one previously downloaded submission
• test data includes at least one undownloaded submission
• ~/.securedrop_client/data in sd-app is empty, and ~/.securedrop_client/svs.sqlite does not exist (do not delete the entire ~/.securedrop_client directory)
• the sd-devices VM is not running (shut down manually if necessary)
• a supported printer is available, but not attached.

Offline to Online

• When SecureDrop desktop icon is double-clicked, preflight updater is displayed
• After preflight updater runs, when user clicks Continue, login dialog is displayed
• When user clicks Work Offline, login dialog closes and main window opens
• after startup:
  • there is no sync attempt with the server
  • the source list is empty
• When the user clicks the top-left user icon and chooses Sign in:
  • the login dialog is displayed over the main window
• When the user enters valid login details and clicks Log in:
  • the login dialog closes
  • The user icon is updated to reflect the user's details
  • the client is synced with the server and the source list is updated
• When the user selects a source with submissions from the source list:
  • the conversation view is populated with the source conversation
  • the reply panel is active
  • a reply can be sent to the source
  • a submission can be downloaded
  • a downloaded submission can be exported
• When the user clicks the main window close button:
  • the client exits.

Scenario: Offline mode with existing data

Prerequisites:

• server is available and contains source test data
• test data includes at least one previously downloaded submission
• test data includes at least one undownloaded submission
• client data directory has been synced with server in a previous login
• the sd-devices VM is not running (shut down manually if necessary)
• a supported printer is available, but not attached.

Offline to Online

• When SecureDrop desktop icon is double-clicked, preflight updater is displayed
• After preflight updater runs, when user clicks Continue, login dialog is displayed
• When user clicks Work Offline, login dialog closes and main window opens
• after startup:
  • there is no sync attempt with the server
  • the source list is populated with contents of last server sync
• When the user selects a source with submissions from the source list:
  • the conversation view is populated with the source conversation
  • the reply panel is inactive, with a "Sign in" message
  • a previously downloaded submission can be exported
  • a previously downloaded submission can be printed
  • When the user clicks Download on an undownloaded submission, a message is displayed instructing the user to sign in to perform the download
• When the user clicks the top-left user icon and chooses Sign in:
  • the login dialog is displayed over the main window
• When the user enters valid login details and clicks Log in:
  • the login dialog closes
  • The user icon is updated to reflect the user's details
  • source data is synced with the server
• When the user selects a source with submissions from the source list:
  • the conversation view is populated with the source conversation
  • the reply panel is active
  • When the user replies to a source, the reply is added to the source conversation
  • When the user clicks Download on an undownloaded submission, the submission is downloaded and decrypted
  • When the user clicks Export on a submission, the export process can be completed
  • When the user clicks Print on a submission, the print process can be completed
• When the user clicks the main window close button:
  • the client exits.

Scenario: Client and Journalist Interface both in use

Note: this scenario requires access to the Journalist Interface (JI) via Tor Browser. If the scenario is being tested on Qubes, the JI address can be found in sd-whonix in /usr/local/etc/torrc.d/50_user.conf. See https://github.com/freedomofpress/securedrop-workstation/wiki/Developer-Tips#how-to-connect-to-the-journalist-interface-in-qubes for instructions on how to connect to the JI in a VM.

Prerequisites:

• server is available and contains source test data
• client data directory is empty

Login

• when SecureDrop desktop icon is double-clicked, preflight updater is displayed
• After preflight updater runs, when user clicks Continue, login dialog is displayed
• after valid login to client:
  • the login dialog closes
  • source data is downloaded and source list is populated
  • user is prompted for GPG key access
  • submissions and replies are decrypted
  • the source list is displayed but no sources are selected by default
  • the conversation view is not populated
• when the JI address is visited in Tor Browser:
  • JI login page is displayed
• after valid login to JI using same account as for client:
  • sources page is displayed, containing the same sources as the client (order may differ)

Sources, replies, submissions

• when a source is starred in the client:
  • the source is also starred in the JI after a page reload
• when a starred source is unstarred in the JI:
  • the source is also unstarred in the client after next sync.
• when a reply is sent to a source via the client:
• the reply is visible in the JI and can be viewed by the source in the Source Interface
• when a reply is sent to a source via the JI:
  • the reply is visible in the source conversation view after next sync
• when the journalist account used to reply is deleted by an admin in the JI:
  • the next sync is successful
  • the reply is visible in the conversation view
  • the journalist's details are deleted from the client database
• when a reply is deleted by a source:
  • the reply is flagged as having being read in the client
• when an individual file submission is deleted in the JI:
  • the submission is no longer listed in the conversation view
  • the submission files are deleted from the client data directory
• when an individual message is deleted in the JI:
  • the message is no longer listed in the conversation view
  • the messages are deleted from the client database
• when a source is deleted in the JI:
  • the source is no longer listed in the client after next sync
  • files associated with the source are no longer present in the client data directory
• when a source is deleted in the client:
  • the source is no longer listed in the JI after a page reload

Scenario: Large dataset

Prerequisites:

• server is available and contains large source test dataset (256 sources, submission sizes ranging from 1-500MB)
• client data directory is empty

Sources

• after valid login:
  • the login dialog closes
  • all source data is downloaded and source list is populated
  • user can scroll to bottom of source list
  • user is prompted for GPG key access
  • submissions and replies are decrypted
  • the source list is displayed but no sources are selected by default
  • the conversation view is not populated
• when a source is selected in source list:
  • conversation view is populated with source conversation
  • a source message containing HTML is displayed as unformatted text
  • source submissions have an active Download button
  • source submission compressed file size is displayed accurately
• when the upper right 3-dot button is clicked:
  • a menu is displayed with a delete source account option
  • when delete source account is selected:
    • the source is deleted from the source list and the converation view is blanked
    • the source is deleted from the server and not restored on next sync
    • source submissions and messages are removed from the client's data directory
• when a source is starred in source list, and the client is closed and reopened in Online mode:
  • the source is still starred in the source list

Replies

• when a source is selected in the source list:
  • the reply panel is available for use and there is no message asking the user to sign in
  • a reply can be added to the conversations
  • a reply containing HTML is displayed as unformatted text
  • two replies added immediately after each other are ordered correctly

Submissions

Preview

• when Download is clicked on a submission:
  • the submission is downloaded and decrypted
  • the Download button is replaced with Print and Export options
  • the submission filename is displayed.
• For a DOC submission:
  • when the submission filename is clicked, a disposable VM (dispVM) is started.
  • after the dispVM starts, the submission is displayed in LibreOffice
  • when LibreOffice is closed, the dispVM shuts down
• For a PDF submission:
  • when the submission filename is clicked, a dispVM is started.
  • after the dispVM starts, the submission is displayed in evince
  • when evince is closed, the dispVM shuts down
• For a JPEG submission:
  • when the submission filename is clicked, a dispVM is started.
  • after the dispVM starts, the submission is displayed in Image Viewer
  • when evince is closed, the dispVM shuts down

Release-specific test plans

Some of the tests below should be incorporated into main test plan after the release, while others will not need to be re-tested with each release.

SecureDrop Workstation 0.3.0

Moved to https://github.com/freedomofpress/securedrop-workstation/issues/548

SecureDrop Client 0.2.0

Moved to https://github.com/freedomofpress/securedrop-client/pull/1083