Skip to content

Workstation Acceptance Tests

Jump to bottom
rocodes edited this page Apr 17, 2024 · 25 revisions

SecureDrop Workstation test scenarios

Some scenarios require a supported printer. We're tracking compatible printers available to team members here.

Also see https://drive.google.com/drive/u/0/folders/1lyk2V15e7amI9OxvgUK_YhYIUVNDWJo0 for different types of file submissions we'll want to test for printing and display (mostly pulled from https://file-examples.com/).

Internationalization (reference)

At least one tester should check that:

  • Under the LANG=en_XA pseudolocale, all GUI strings like foo are correctly wrapped like [!foo!]. (Any string not so wrapped is missing its gettext() wrapper.)
  • Generate a conversation transcript and export it. Inspect to make sure it displays correctly in the specified locale.

Qubes scenarios

Some of these may be a bit time-consuming, so we typically have a subset of devs test these specific scenarios. In these scenarios, we try to validate the behavior of different components of the system. There's a separate section with detail on Client functionality specifically.

Verify tor connection to Journalist API

  • Create VM for accessing JI via Tor Browser: qvm-create --template whonix-ws-16 --property netvm=sd-whonix --label orange sd-research. Open Tor Browser in that VM and confirm you can log in to the Journalist Interface. This confirms that sd-whonix is configured correctly (but does not use sd-proxy).
  • Change the netvm to sys-whonix and confirm you can load the public Source Interface, but not the Journalist Interface. (N.B. you must leave the netvm set to sys-whonix, otherwise make clean and sdw-admin --uninstall will fail.)

Verify default DispVM settings

  • Open a shell in a non-SDW VM, e.g. sd-dev. Download a PDF file and open it via: qvm-open-in-dvm <pdf_file>. Confirm it opens in a DispVM, and that the DispVM is based on sd-viewer.
  • Open a shell in sd-app and find an already downloaded submission in ~/.securedrop_client/data/. Run xdg-open <file_path> and confirm it opens in a DispVM, and that the DispVM is based on sd-viewer.

RPC Policies

  • Open a shell in a non-SDW VM, e.g. sd-dev. Run: QUBES_GPG_DOMAIN=sd-gpg qubes-gpg-client -k. Confirm that the request is denied, i.e. you do NOT see pubkey info for the SecureDrop Submission Key.
  • Try to copy/paste from the Client to a non-SDW VM, e.g. sd-dev. Confirm you cannot.
  • Add the clipboard tags to sd-dev as described in the docs, and repeat the copy/paste procedure. Confirm it works.
  • /etc/qubes/policy.d/ contains 2 xx-securedrop-workstation.policy files (todo: more testing to ensure default deny in place for SD VMs on new qrexec policies)

Logging

VM wellbeing

  • Inspect individual VMs with sudo journalctl while performing regular system testing and look for relevant errors/warnings (todo: needs a bit clearer instruction for testers)
  • Apparmor is running on VMs (sudo apparmor_status - nothing unconfined)
  • grsec kernel is installed on VMs

GUI updater

  • Reboot the workstation after installing SDW. Confirm that the prelaunch updater window appears automatically after logging, prompting for an update.
  • Proceed with GUI updater, confirm it runs without errors.

Client scenarios

Some client tests do not require being run in a SecureDrop Workstation environment. The list of scenarios below is a complete list for full acceptance testing, which should be done in the supported target Qubes environment.

Scenario: Online mode

Prerequisites:

  • server is available and contains source test data
  • access to sd-gpg keyring has not been previously granted
  • ~/.securedrop_client/data in sd-app is empty, and ~/.securedrop_client/svs.sqlite does not exist (do not delete the entire ~/.securedrop_client directory)
  • the sd-devices VM is not running (shut down manually if necessary)
  • a supported printer is available, but not attached.
  • all VMs are up-to-date
  • test instance contains several sources, including some with files & some with HTML characters in messages

Login

  • when SecureDrop desktop icon is double-clicked, preflight updater is displayed
  • After preflight updater runs, when user clicks Continue, login dialog is displayed
  • In login dialog:
    • show/hide password functionality works
    • incorrect password cannot log in
    • 2FA token reuse cannot log in after password failure
    • invalid 2FA token cannot log in
    • valid credentials and 2FA can log in

Sources

  • after valid login:
    • the login dialog closes
    • source data is downloaded and source list is populated
    • user is prompted for GPG key access
    • submissions and replies are decrypted
    • the source list is displayed but no sources are selected by default
    • the conversation view is not populated
  • when a source is selected in source list:
    • conversation view is populated with source conversation
    • a source message containing HTML is displayed as unformatted text
    • source submissions have an active Download button
    • source submission compressed file size is displayed accurately
  • when the upper right 3-dot button is clicked:
    • a menu is displayed with a delete source account option
    • when delete source account is selected:
      • the source is deleted from the source list and the conversation view is blanked
      • the source is deleted from the server and not restored on next sync
      • source submissions and messages are removed from the client's data directory
  • when a source is starred in source list, and the client is closed and reopened in Online mode:
    • the source is still starred in the source list

Replies

  • when a source is selected in the source list:
    • the reply panel is available for use and there is no message asking the user to sign in
    • a reply can be added to the conversation
    • a pending reply can be added to the conversation (for development environments, you can use: wget https://gist.githubusercontent.com/creviera/7f19a7d10334359f40dbdbb2354cd13a/raw/a2ef94913a155aa4019b753cf916f844c9cffa3a/pending-reply && git apply pending-reply then send a reply; alternatively, disconnect the network or sd-whonix after sending a reply)
    • a failed reply can be added to the conversation (for development environments, you can use: wget https://gist.githubusercontent.com/creviera/5ba70d50c12b6a6df6f98ed40ad09645/raw/5caef3339ceab1fc997ccb6b9e337bc8828ef12f/failed-reply && git apply failed-reply then send a reply; alternatively, sign out after the previous step to confirm that the reply transitions to "failed" state)
    • a reply containing HTML is displayed as unformatted text
    • a reply with a single string of characters longer than 100 chars is displayed, but truncated (https://github.com/freedomofpress/securedrop-client/issues/815).
    • a reply with a line longer than 100 chars is displayed correctly
    • two replies added immediately after each other are ordered correctly

Submissions

For sample files in different formats, see this GDrive folder.

Preview
  • when Download is clicked on a submission:
    • the submission is downloaded and decrypted
    • the Download button is replaced with Print and Export options
    • the submission filename is displayed.
  • For a DOC submission:
    • when the submission filename is clicked, a disposable VM (dispVM) is started.
    • after the dispVM starts, the submission is displayed in LibreOffice
    • when LibreOffice is closed, the dispVM shuts down
  • For a PDF submission:
    • when the submission filename is clicked, a dispVM is started.
    • after the dispVM starts, the submission is displayed in evince
    • when evince is closed, the dispVM shuts down
  • For a JPEG submission:
    • when the submission filename is clicked, a dispVM is started.
    • after the dispVM starts, the submission is displayed in Image Viewer
    • when Image Viewer is closed, the dispVM shuts down
  • For an audio submission:
    • when the submission filename is clicked, a dispVM is started.
    • After the dispVM starts, the submission is played in Audacious
    • Sound is audible
    • when Audacious is closed, the dispVM shuts down
  • For a video submission:
    • when the submission filename is clicked, a dispVM is started.
    • After the dispVM starts, the submission is played in Totem
    • Sound is audible if applicable
    • when Totem is closed, the dispVM shuts down
  • For a compressed (archive) submission:
    • when the submission filename is clicked, a dispVM is started.
    • After the dispVM starts, the submission is opened in FileRoller
    • Individual files can be extracted and previewed
    • when FileRoller is closed, the dispVM shuts down
Export
  • When Export is first clicked on a submission:
    • the "Preparing to export..." message is displayed
    • the sd-devices VM is started
    • the user is prompted to insert an Export USB
    • On clicking Cancel, the prompt closes and the file is not exported
  • When Export is clicked on the submission again:
    • the "Preparing to export..." message is displayed
    • the user is prompted to insert an Export USB
    • When the user inserts an invalid Export USB, attaches it to the sd-devices VM and clicks Next:
      • a message is displayed indicating that the Export USB is invalid and the user is prompted to insert a valid device
  • When Export is clicked on the submission again:
    • the "Preparing to export..." message is displayed
    • the user is prompted to insert an Export USB
    • When the user inserts a valid Export USB, attaches it to the sd-devices VM, and clicks Next:
      • the user is prompted for the Export USB's password if the (LUKS) device is locked
      • the user is not prompted for the USB's password, if the (LUKS or VeraCrypt) device is unlocked
      • the user is not prompted for the USB's password, if the (LUKS or VeraCrypt) device is unlocked and mounted
    • When the user enters an invalid Export USB password and clicks Next:
      • a failure message is displayed and the user is prompted to enter the password again
    • When the user enters an valid Export USB password and clicks Next:
      • the file is saved to the Export USB and a success message is shown
  • When the user detaches the Export USB and mounts it on another VM or computer:
    • the decrypted submission(s) is available in on the Export USB, in a directory sd-export-<timestamp>/export_data
Print
  • When the user clicks Print on a downloaded submission:
    • a "Preparing to print..." message is displayed
    • the sd-devices VM is started
    • the user is prompted to connect a supported printer
  • When the user connects a printer, attaches it to the sd-devices VM, and clicks Continue:
    • a "Printing..." message is displayed
    • the X Printer Panel dialog is displayed with the printer selected
  • When the user clicks Print in the X Printer Panel:
    • the submission is printed successflly.
  • A multi-page document can be printed successfully

Closing the client

  • When the user clicks the main window close button:
    • the client exits.

Scenario: Offline mode without existing data

Prerequisites:

  • server is available and contains source test data
  • test data includes at least one previously downloaded submission
  • test data includes at least one undownloaded submission
  • ~/.securedrop_client/data in sd-app is empty, and ~/.securedrop_client/svs.sqlite does not exist (do not delete the entire ~/.securedrop_client directory)
  • the sd-devices VM is not running (shut down manually if necessary)
  • a supported printer is available, but not attached.

Offline to Online

  • When SecureDrop desktop icon is double-clicked, preflight updater is displayed
  • After preflight updater runs, when user clicks Continue, login dialog is displayed
  • When user clicks Work Offline, login dialog closes and main window opens
  • after startup:
    • there is no sync attempt with the server
    • the source list is empty
  • When the user clicks the top-left user icon and chooses Sign in:
    • the login dialog is displayed over the main window
  • When the user enters valid login details and clicks Log in:
    • the login dialog closes
    • The user icon is updated to reflect the user's details
    • the client is synced with the server and the source list is updated
  • When the user selects a source with submissions from the source list:
    • the conversation view is populated with the source conversation
    • the reply panel is active
    • a reply can be sent to the source
    • a submission can be downloaded
    • a downloaded submission can be exported
  • When the user clicks the main window close button:
    • the client exits.

Scenario: Offline mode with existing data

Prerequisites:

  • server is available and contains source test data
  • test data includes at least one previously downloaded submission
  • test data includes at least one undownloaded submission
  • client data directory has been synced with server in a previous login
  • the sd-devices VM is not running (shut down manually if necessary)
  • a supported printer is available, but not attached.

Offline to Online

  • When SecureDrop desktop icon is double-clicked, preflight updater is displayed
  • After preflight updater runs, when user clicks Continue, login dialog is displayed
  • When user clicks Work Offline, login dialog closes and main window opens
  • after startup:
    • there is no sync attempt with the server
    • the source list is populated with contents of last server sync
  • When the user selects a source with submissions from the source list:
    • the conversation view is populated with the source conversation
    • the reply panel is inactive, with a "Sign in" message
    • a previously downloaded submission can be exported
    • a previously downloaded submission can be printed
    • When the user clicks Download on an undownloaded submission, a message is displayed instructing the user to sign in to perform the download
  • When the user clicks the top-left user icon and chooses Sign in:
    • the login dialog is displayed over the main window
  • When the user enters valid login details and clicks Log in:
    • the login dialog closes
    • The user icon is updated to reflect the user's details
    • source data is synced with the server
  • When the user selects a source with submissions from the source list:
    • the conversation view is populated with the source conversation
    • the reply panel is active
    • When the user replies to a source, the reply is added to the source conversation
    • When the user clicks Download on an undownloaded submission, the submission is downloaded and decrypted
    • When the user clicks Export on a submission, the export process can be completed
    • When the user clicks Print on a submission, the print process can be completed
  • When the user clicks the main window close button:
    • the client exits.

Scenario: Client and Journalist Interface both in use

Note: this scenario requires access to the Journalist Interface (JI) via Tor Browser. If the scenario is being tested on Qubes, the JI address can be found in sd-whonix in /usr/local/etc/torrc.d/50_user.conf. See https://github.com/freedomofpress/securedrop-workstation/wiki/Developer-Tips#how-to-connect-to-the-journalist-interface-in-qubes for instructions on how to connect to the JI in a VM.

Prerequisites:

  • server is available and contains source test data
  • client data directory is empty

Login

  • when SecureDrop desktop icon is double-clicked, preflight updater is displayed
  • After preflight updater runs, when user clicks Continue, login dialog is displayed
  • after valid login to client:
    • the login dialog closes
    • source data is downloaded and source list is populated
    • user is prompted for GPG key access
    • submissions and replies are decrypted
    • the source list is displayed but no sources are selected by default
    • the conversation view is not populated
  • when the JI address is visited in Tor Browser:
    • JI login page is displayed
  • after valid login to JI using same account as for client:
    • sources page is displayed, containing the same sources as the client (order may differ)

Sources, replies, submissions

  • when a source is starred in the client:
    • the source is also starred in the JI after a page reload
  • when a starred source is unstarred in the JI:
    • the source is also unstarred in the client after next sync.
  • when a reply is sent to a source via the client:
  • the reply is visible in the JI and can be viewed by the source in the Source Interface
  • when a reply is sent to a source via the JI:
    • the reply is visible in the source conversation view after next sync
  • when the journalist account used to reply is deleted by an admin in the JI:
    • the next sync is successful
    • the reply is visible in the conversation view
    • the journalist's details are deleted from the client database
  • when a reply is deleted by a source:
  • when an individual file submission is deleted in the JI:
    • the submission is no longer listed in the conversation view
    • the submission files are deleted from the client data directory
  • when an individual message is deleted in the JI:
    • the message is no longer listed in the conversation view
    • the messages are deleted from the client database
  • when a source is deleted in the JI:
    • the source is no longer listed in the client after next sync
    • files associated with the source are no longer present in the client data directory
  • when a source is deleted in the client:
    • the source is no longer listed in the JI after a page reload

Scenario: Large dataset

Prerequisites:

  • server is available and contains large source test dataset (256 sources, submission sizes ranging from 1-500MB)
  • client data directory is empty

Sources

  • after valid login:
    • the login dialog closes
    • all source data is downloaded and source list is populated
    • user can scroll to bottom of source list
    • user is prompted for GPG key access
    • submissions and replies are decrypted
    • the source list is displayed but no sources are selected by default
    • the conversation view is not populated
  • when a source is selected in source list:
    • conversation view is populated with source conversation
    • a source message containing HTML is displayed as unformatted text
    • source submissions have an active Download button
    • source submission compressed file size is displayed accurately
  • when the upper right 3-dot button is clicked:
    • a menu is displayed with a delete source account option
    • when delete source account is selected:
      • the source is deleted from the source list and the converation view is blanked
      • the source is deleted from the server and not restored on next sync
      • source submissions and messages are removed from the client's data directory
  • when a source is starred in source list, and the client is closed and reopened in Online mode:
    • the source is still starred in the source list

Replies

  • when a source is selected in the source list:
    • the reply panel is available for use and there is no message asking the user to sign in
    • a reply can be added to the conversations
    • a reply containing HTML is displayed as unformatted text
    • two replies added immediately after each other are ordered correctly

Submissions

Preview
  • when Download is clicked on a submission:
    • the submission is downloaded and decrypted
    • the Download button is replaced with Print and Export options
    • the submission filename is displayed.
  • For a DOC submission:
    • when the submission filename is clicked, a disposable VM (dispVM) is started.
    • after the dispVM starts, the submission is displayed in LibreOffice
    • when LibreOffice is closed, the dispVM shuts down
  • For a PDF submission:
    • when the submission filename is clicked, a dispVM is started.
    • after the dispVM starts, the submission is displayed in evince
    • when evince is closed, the dispVM shuts down
  • For a JPEG submission:
    • when the submission filename is clicked, a dispVM is started.
    • after the dispVM starts, the submission is displayed in Image Viewer
    • when evince is closed, the dispVM shuts down

Release-specific test plans

Some of the tests below should be incorporated into main test plan after the release, while others will not need to be re-tested with each release.

SecureDrop Workstation 0.3.0

Moved to https://github.com/freedomofpress/securedrop-workstation/issues/548

SecureDrop Client 0.2.0

Moved to https://github.com/freedomofpress/securedrop-client/pull/1083

Clone this wiki locally