Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates pip dependencies: Ansible, Pyyaml, Jinja2, Molecule, Werkzeug #4865

Merged
merged 10 commits into from Sep 27, 2019

Conversation

@emkll
Copy link
Contributor

emkll commented Sep 25, 2019

Status

Ready for review

Description of Changes

Fixes #4692
Fixes #4343
Fixes #4693

Testing

  • CI passes
  • Hashes are valid
  • All instances of Jinja2, Ansible, Pyyaml and Werkzeug are updated and should not have any CVEs associated to them
  • Diff reviews were done for all production (admin and app code requirements)
  • The addition to the GitHub template for PRs (Diff review in checklist) makes sense
  • Production install (on VMs or hardware) is successful with the new version of Ansible

Deployment

  1. ./securedrop-admin update will update admin requirements
  2. securedrop-app-code requirements will be bundled as part of the virtualenv shipped by the securedrop-app-code debian package to production instances
  3. All other changes are dev-only

Checklist

If you made changes to securedrop-admin:

  • Linting and tests (make -C admin test) pass in the admin development container

If you made changes to the system configuration:

If you made non-trivial code changes:

  • I have written a test plan and validated it for this PR

If you added or updated a code dependency:

emkll added 8 commits Sep 20, 2019
Due to CVE-2019-14806:

Pallets Werkzeug before 0.15.3, when used with Docker, has insufficient debugger PIN randomness because Docker containers share the same machine id.
Addresses CVE-2019-10156. Given that SecureDrop would not process untrusted variables in templates, this vulnerability did not impact the system.

Also updates SecureDrop installer's ansible version check
This is required to upgrade pyyaml to 5.1 (resolving CVE-2017-18342) and jinja2 (resolving CVE-2019-10906)
sudo pip installing (even in the virtualenv) would cause permission issues for the molecule tmp files at runtime
Resolves CVE-2017-18342
Also updates docs with correct python3 requirements path
@emkll emkll changed the title Updates pip dependencies Updates pip dependencies: Ansible, Pyyaml, Jinja2, Molecule, Werkzeug Sep 25, 2019
Resolves CVE-2019-10206
@emkll emkll force-pushed the pip-updates-sprint-38 branch from a31292d to aacaed7 Sep 25, 2019
@emkll emkll marked this pull request as ready for review Sep 25, 2019
@emkll emkll added this to Ready for review in SecureDrop Team Board Sep 25, 2019
During review of #4865, I misinterpreted the new checkboxes as a
sequential list, rather than an either/or choice. Adding a small
clarification to prevent future misunderstandings. =)
@conorsch

This comment has been minimized.

Copy link
Contributor

conorsch commented Sep 27, 2019

CI is passing. Confirmed diff review was performed according to https://github.com/freedomofpress/securedrop/wiki/Dependency-Update-Policies. Visual review looks good, with one small nit: the additions to the PR template weren't immediately clear to me, so I just pushed up a tiny clarification. Letting CI cook again on that, will proceed with clean install in prod VMs before approving.

Copy link
Contributor

conorsch left a comment

Clean install on prod VMs went smoothly. Functional interaction with Source Interface showed no problems. As part of testing in virtualized Tails, I ran:

  • ./securedrop-admin setup
  • ./securedrop-admin sdconfig
  • ./securedrop-admin install
  • ./securedrop-admin tailsconfig

No surprises. Thanks for tackling the upgrades and diff review, @emkll!

@conorsch conorsch merged commit 94e4a9d into develop Sep 27, 2019
9 checks passed
9 checks passed
LGTM analysis: JavaScript No code changes detected
Details
LGTM analysis: Python No new or fixed alerts
Details
ci/circleci: admin-tests Your tests passed on CircleCI!
Details
ci/circleci: app-tests Your tests passed on CircleCI!
Details
ci/circleci: lint Your tests passed on CircleCI!
Details
ci/circleci: staging-test-with-rebase Your tests passed on CircleCI!
Details
ci/circleci: static-analysis-and-no-known-cves Your tests passed on CircleCI!
Details
ci/circleci: translation-tests Your tests passed on CircleCI!
Details
ci/circleci: updater-gui-tests Your tests passed on CircleCI!
Details
SecureDrop Team Board automation moved this from Ready for review to Done Sep 27, 2019
@conorsch conorsch mentioned this pull request Sep 27, 2019
2 of 13 tasks complete
@emkll emkll deleted the pip-updates-sprint-38 branch Oct 8, 2019
kaiiyer added a commit to kaiiyer/securedrop that referenced this pull request Oct 20, 2019
* Adds support for v3 Onion to SSH inventory script

Preserves the v2 Onion lookup logic, but prefers v3 Onions if those are
found on the Admin Workstation. Ensures that Admins connecting to the
servers after migrating to v3 Onion URLs are using them for SSH.

* Tweak icon alignment, spacing; fix mobile regressions

* Add footer.html to Apache whitelist

* update mkvirtualenv command to reflect update to python3

* Invalidate Session When Admin Resets Journalist Password

* Improve deletion of submissions

Replace srm with shred, which is faster, reducing the chance that
deletion of a submission will be interrupted.

Update rq and redis requirements, to eliminate long-standing bugs.

Add manage.py tasks for detecting and correcting submissions that have
been disconnected from their files on disk, and vice versa. Update
manage.py to explicitly run with the production virtualenv. Also
specify the virtualenv in WSGI scripts and the run-test script. In the
dev/test Docker container, install requirements in a virtualenv at the
same path as production.

Add a supervisor script for requeuing interrupted rq jobs. If the app
server is rebooted while an rq job is running, that job has already
been deleted from the queue and rq will not automatically resume it on
reboot, but it does have a record of it in the queue's started job
registry. This script checks that registry for jobs that aren't
already queued or being run, and requeues them.

* Fix CI lint requirements installation

* Fix mobile regression in footer layout, rename class

Resolves freedomofpress#4701

* Use final footer wording

Resolves freedomofpress#4700

* Adjust tooltip sizing/weight to reduce likelihood of abbreviation

Resolves freedomofpress#4716

* Use config.SECUREDROP_ROOT for worker path in test_worker.py

* Delete Submissions with Sources that were deleted

This is accomplished by an alembic migration that removes any orphaned
submissions from the database.

* store: add method to find a file without its filesystem_id

if a source has been deleted, we no longer have its filesystem_id
in the database. as such, we will need to find the file, being careful
to guard against potential duplicates due to the journalist_designations
not necessarily being unique (this is a very rare case)

* alembic: add replies to the migration and enqueue jobs to cleanup the files

* create new exception class and handle in migration for resiliency

two situations _could_ potentially arise trying to think of the
edge cases here:

- Admin has been deleting files manually by digging around for
files that consume a lot of disk in the securedrop data directory
- A very rare situation can occur where there is a colliding
journalist_designation

* alembic: we should also delete objects that have invalid source_ids

invalid objects being those which do not have a matching source_id
in the sources table

* alembic: ensure row is deleted if the file is gone

easiest to do with a custom exception, but for files that were
manually deleted by a curious admin who e.g. removed large files that
were not properly deleted due to this bug, we should also remove
the corresponding row in the database.

* alembic: migration contains raw SQL, add `# nosec`

bandit flags this because the table name is passed in as a variable.
It's not user controlled so this does not introduce a security problem
(function is in the migration only). I could suppress the
alert by removing the function but in the spirit of not
making this alembic migration even more repetitive I'm adding `# nosec`.

* alembic: guard against config.py not existing yet

one of the quirks of SecureDrop is that config.py might
not yet exist until the app Ansible role has ran

this means for fresh installs, the database must be created
via alembic upgrade head without any errors raising, else
the securedrop-app-code package install will fail

* deletion cleanup: srm -> secure_delete, new session_nonce

* Update documentation for translators

* deletion cleanup: worker.py now imports config.py

so it won't import on existing installs

* deletion cleanup: mv two imports that import worker.py into try/except

* Fixes freedomofpress#4708, adds sdconfig prompt for v2/v3 services

Now it asks if the admin wants to enable v2 or v3 services.
It will not let you to disable v3 if you already disabled v2.

* Adds test cases and related change for v2/v3 user input

We now have two more integration tests to verify that
we can have "enabled v2 and disabled v3" and both versions
enabled at the same time.

* Updates based on feedback of strings

* Exposes Onion URLs (v2 & v3) to webapp

We write plaintext files to /var/lib/securedrop so that the application
can provide user-facing messaging about upcoming v2 -> v3 Onion URL
transitions. For example, if a v3 Source Onion URL exists, but a source
is browsing via the v2 URL, we can recommend the v3 URL instead.

* Documents running subset of testinfra tests

Since testinfra is based on pytest, the pytest env var for passing
additional options works just fine. We can use the `-k` option to
perform a substring search on test names, and run only those tests
matching the selector.

* Renames admin _config var for clarity

There's already a `.config` attribute in the sdconfig flow, so let's
rename `._config` -> `._config_in_progress` to make the intended use
case explicit to future maintainers. The contents of the
_config_in_progress dict are consulted during validation steps as part
of `sdconfig`.

* Fixes freedomofpress#4677 warns admin if v3 and https both are enabled

* Use dev-shell for make translate

* Add Weblate translation status widgets

Weblate provides widgets showing translation progress. Add a badge to
our README, and larger widgets to the Translators section of our
Contributing doc.

* admin: add warning message for v3 onions and HTTPS

* Update Tor Debian package version to 0.4.1.5-1

* Bump Tor Browser in Dockerfile to 8.5.4

Fixes 404 on Tor Browser Download for dev env

* Incorporate source string feedback from KwadroNaut

* Update the path to rqworker in dev-deps supervisor config

The supervisor configuration in the dev container was still pointing
to /usr/local/bin/rqworker, which has been moved to
/opt/venvs/securedrop-app-code/bin. Correcting the path fixes
submission hashing and deletion.

* Correct cleanup of disconnected submission files

In securedrop.management.submissions.find_disconnected_fs_submissions,
check Replies as well as Submissions for a valid association to any
file found in the store, before reporting it as disconnected.

* Add install-time check for JSON keyfile if client_auth files present on Admin Workstation

* Updated to allow for SSH-over-LAN scenarios

* Updating tailsconfig to always run network hook

* Update Python script interpreter directives

For Python scripts run where the /opt/venvs/securedrop-app-code
virtualenv is used, specify that in the interpreter directive line.

Remove the shebang line from crypto_util.py, since it's never run.

Leave unmodified the scripts used on workstations or in packaging VMs.

* Teach update_version.sh to bump the expected sdist tarball version

* ci: bump staging timeout to 35m

* Remove custom_logo.png from app code build logic

rsync filter rules are processed on a first-match basis, which means the exclusion rule for custom_logo.png must appear before the wildcard inclusion of all png files.

* Update release management docs to ensure repository is clean prior to building packages.

* build: fetch sources only during securedrop-app-code build

* Make build-debs more verbose

For securedrop-app-code, give pip the --verbose option.

In build-debs.sh, give molecule the --debug option if the environment
variable SD_DEBUG_BUILD_DEBS is "yes".

* Fixes freedomofpress#4762 adds missing variables for tailsconfig

* docs: fix typos

severs -> servers
that so that -> so that

* Add instructions on updating Tails USBS.

Refer to updates instead of upgrades. Clarify generic Tails manual update process. Add warning to Tails Setup docs on verifying iso image. Additional changes in wording per pr review.

* Default v3_onion_services to off in validate_tails_environment.yml

The lack of this variable has tripped us up in several roles, and
depending on how the validate role is used, adding it to
defaults/main.yml within that role might not fix it, so let's add a
fact in the validate_tails_environment playbook defaulting it to off.

* Internal links on Tails backup process

* Fix typo naming Tails instead of Ubuntu

* update torbrowser version

* Update screenshots in docs for 1.0.0

Also update source guide to reflect button label changes.

Cropped the Tor download dialogs to not show the background, so they
have a longer shelf life.

Resolves freedomofpress#4721

* Update Tor restart dialog screenshot

* Remove v3 onion secrets when transitioning to ssh over local

- v2 aths secret file pattern does not match the one for v3
- Use async task for reboot for more immediate feedback
- Instruct admin to run tailsconfig to update the ssh config on tails workstation

* updated documentation to include references to v3 services

* Expand .gitgnore for venv

The wilcard did not match `.venv/` folder used for python2 virtualenv in the securedrop-admin context

* Rework Ansible restore role to detect Onion service version mismatch

Make the restore playbook extract the Tor configuration from the
backup, retrieve the app server's Tor configuration, and compare
them. If the service versions differ, abort the restore with an
explanation.

* Instruct localization manager to squash translation commits

* Overhaul Transfer Device and export recommendations

Resolves freedomofpress#4620
Resolves freedomofpress#4646
Resolves freedomofpress#4434
Resolves freedomofpress#4670

In addition to introducing the Transfer and Export Device,
this commit clearly breaks out optional hardware into its
own checklist, more strongly recommends purchase of a
printer, and strengthens recommendations for malware
mitigation.

It removes some outdated recommendations and a reference to
storing the journalist's GPG passphrase in KeePassX
(not mentioned anywhere else).

It removes instructions for journalists to set up individual
GPG keys, as they are unlikely to be followed, and the more
critical recommendation is to ensure tha the Export Device
is encrypted.

It updates the overview diagram and data flow diagram
consistent with the above changes. This update also makes
the diagrams more consistent with terminology and current
practices. It removes OnionShare from the data flow
diagram, as it is not currently mentioned elsewhere in the
docs.

* Rename files for consistency with new organization/content

* Improve loading behavior of language selector in Tor

Specifying a minimum width ensures the widget is always rendered
as it should be.

Fixes freedomofpress#4810

* added setup command to pull in dependencies for v3 key generation

* Update Tails desktop icons, updater banner, updater icon

These were still using the old logo. Note that resources_rc.py
needed to be regenerated for these updates to be reflected in the
updater UI.

* updates based on PR review

* Fix journalist test_delete_one

JournalistNavigationStepsMixin's _journalist_delete_one is more
reliable with an ActionChains move/click incantation, but switching to
that in the functional test safe_click* methods breaks a bunch of
other tests, so I'm just rolling with the Selenium voodoo.

* Update logo in README

`docs/images/logo.png` is used for no other purpose, so deleting.

* Add documentation of submission cleanup

* Update builder hash

Builder image needed security updates

* l10n: updated Arabic (ar)

contributors:
  Thalia Rahme

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* l10n: updated Catalan (ca)

contributors:
  Benet (BennyBeat) R. i Camps
  John Smith

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* l10n: updated German (de_DE)

contributors:
  Ettore Atalan
  John Hensley
  Robin Schubert

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* l10n: updated Greek (el)

contributors:
  Dimitris Maroulidis

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* l10n: updated Spanish (es_ES)

contributors:
  Adolfo Jayme-Barrientos
  carlos
  erinm

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* l10n: updated French (fr_FR)

contributors:
  AO

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* l10n: updated Hindi (hi)

contributors:
  AbhayKaushik
  Chandan Kumar (raukadah)
  Drashti

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* l10n: updated Icelandic (is)

contributors:
  Oktavia
  Sveinn í Felli

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* l10n: updated Italian (it_IT)

contributors:
  Beatrice Martini
  Claudio Arseni
  Giandomenico Lombardi

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* l10n: updated Norwegian (nb_NO)

contributors:
  John Hensley
  Øyvind Bye Skille

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* l10n: updated Dutch (nl)

contributors:
  Pander
  Thom
  kwadronaut

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* l10n: updated Portuguese, Brasil (pt_BR)

contributors:
  communiaa

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* l10n: updated Romanian (ro)

contributors:
  mickael e

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* l10n: updated Russian (ru)

contributors:
  Adham Kurbanov
  erinm

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* l10n: updated Swedish (sv)

contributors:
  Jonas Franzén

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* l10n: updated Turkish (tr)

contributors:
  Kaya Zeren
  tekrei

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* l10n: updated Chinese, Traditional (zh_Hant)

contributors:
  Chi-Hsun Tsai

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 75d090c303ed9827cb5fd0eb647f322b56d5d707

* Adds ./ to manage.py invocations in admin docs

Same as we do elsewhere, in codeblocks, let's explicitly prefix the
"manage.py" executable with "./", since Admins will need to type that
invocation exactly in order for the subcommand to work.

* Add 0.14.0 to 1.0.0 upgrade guide, remove oldest one, DRY up docs a bit

* Update builder hash

Builder needed security updates

* Clearer hardware prioritization, updated threat model

The hardware recommendations more clearly prioritize choices we
can support and are familiar with (VeraCrypt, USB w/ write switch)
over ones we cannot currently support well (hardware-encrypted
drives, write blockers).

The threat model has been updated consistent with the data flow
updates; note that the clarified data flow means that the Transfer
Device never stores _decrypted_ submissions.

The use of write blockers for the Transfer Device has been
clarified, as well.

* added StartupNotify=true to desktop shortcuts

* Add flaky to translation-test

* SecureDrop 1.0.0~rc1

(cherry picked from commit 4b4e557)

* SecureDrop 1.0.0-rc2

(cherry picked from commit 09bfa45)

* SecureDrop 1.0.0-rc3

(cherry picked from commit 874d48c)

* Updates to 1.0.0~rc4

(cherry picked from commit 392a6e7)

* Fix PR # for orphaned submission entry in changelog

(cherry picked from commit 7291100)

* Release 1.0.0

(cherry picked from commit 162288f)

* Updates to 1.1.0~rc1

* added HTTPS EV cert note

* Stronger warning against using `manage.py reset`

* docs: improvements to RM guide after 1.0.0 release

* Move grsec role before common role

The in securedrop-staging.yml, the grsec role is bring run before the common role.

[1] : https://github.com/freedomofpress/securedrop/blob/9f32d46136861546af556d3f54d92e31cebe2eb0/install_files/ansible-base/securedrop-staging.yml

* Fixes v3 keypair generation in upgrade box prep

During 1.0, we did not test the v3 keypair generation in the upgrade
scenario. Makes sense: we only run that scenario *after* a release is
final. The git-repo-root logic assumed we were in Tails or running
against normal staging VMs. In the vagrant-package logic (which builds
"upgrade" boxes for testing), the proper path structure requires that
filepaths be relative to the secondary git repo, cloned in order to
prepare an old version from the known-good tag.

This var override ensures that the v3 keys are stored inside the
secondary git repo. We could instead update the rev-parse logic to be
relative to the role, but that'd require additional testing in Tails,
and this works just as well.

* Add upgrade boxes for SecureDrop 1.0.0

* updated Tor version in python2 dockerfile

* workaround for Tails detection in securedrop-admin setup

* added check for Tails in /etc/os-release in validate role

* added Tails 4.0 check in inventory-dynamic

* Update upgrade playbook for v3 onion services

Use v2 and v3 onion services

* reordered manage.py args in parser to put common operations first and dev-only operations last

* ci: remove python 2 CI steps/jobs

* dev: remove python 2 support from dev env

recall that we have a separate dev env for the securedrop-admin
CLI tool in admin/

* app: remove python 2 support

* bandit: add exception for input() since it's safe in Python 3

* code owners: @msheiny has been offboarded from team SD 😢

* docs: update mac dev and type annotation instructions for python 3

* securedrop-app-code: remove python 2 reference in comment

(we're using python 3.5)

* Update the app test instructions

It's now just "make test", instead of "make -C securedrop test".

* Bump wekzeug to 0.16.0

Due to CVE-2019-14806:

Pallets Werkzeug before 0.15.3, when used with Docker, has insufficient debugger PIN randomness because Docker containers share the same machine id.

* Bump ansible to 2.6.18

Addresses CVE-2019-10156. Given that SecureDrop would not process untrusted variables in templates, this vulnerability did not impact the system.

Also updates SecureDrop installer's ansible version check

* Bump molecule to 2.22

This is required to upgrade pyyaml to 5.1 (resolving CVE-2017-18342) and jinja2 (resolving CVE-2019-10906)

* Don't sudo pip install in container for linting

sudo pip installing (even in the virtualenv) would cause permission issues for the molecule tmp files at runtime

* Update Pyyaml in admin requirements

Resolves CVE-2017-18342

* Add dependency diff review to pull request template

* Fix tests for new version of testinfra

* Pin hashes for dev requirements

Also updates docs with correct python3 requirements path

* Bump ansible to 2.6.19

Resolves CVE-2019-10206

* Apply mix fixes in install_files/securedrop-app-code/debian/copyright

* Also updates Upstream-Name

These "securedrop-proxy" strings were leftover from packaging logic
updates in freedomofpress#4870. Hat tip to @rmol for catching this one during review.

* Reload extension to refresh desktop icons in Tails 4

* Fix scaling of diagrams in docs

Resolves freedomofpress#4873

* Improve specification of securedrop-app-code dependencies

The securedrop-app-code Debian package didn't fully specify its
dependencies, crucially libpython3.5, which is almost always installed
on SecureDrop servers built according to our instructions, but doesn't
have to be. This fixes that, and makes the securedrop-app-code package
the sole source of truth about its dependencies, eliminating the
out-of-band installation of some of them via the "app" Ansible role.

This also fixes a few dependency specifications in the instructions
for setting up a development environment.

* Clarifications on diff review in PR template

During review of freedomofpress#4865, I misinterpreted the new checkboxes as a
sequential list, rather than an either/or choice. Adding a small
clarification to prevent future misunderstandings. =)

* Moves securedrop-admin to Python3

This will inform the admins to rerun ./securedrop-admin setup
as this will create .venv3 dirctory for Python3 dependencies.

* Works for both Tor4 and Tor3

As Tor4 has Python 3.7.3, we should just mention without any
particular version.

* Python3 compat for Admin tooling

A smattering of the familiar Python 2 -> 3 changes, mostly around string
handling, filepaths to interpreters and virtualenvs, and syntax updates.
Specifically:

  * coerces major release number to int
  * converts iteritems() -> items() in dynamic inventory
  * updates string handling in dynamic inventory
  * ensures securedrop_init runs under python3

Tested under Tails 3.16 (based on Stretch), running Python 3.5. Worked
well, after these fixes landed, although I've observed a handful of the
"weakref" stderr output. The weakref output doesn't break anything, but
it may be alarming to Admins who haven't updated to Tails 4 & Python 3.7
yet.

* added check for admin password in GUI updater

* Update Ansible to 2.7.13

* Resolve Ansible 2.7 deprecation warnings for apt

Several instances of the warning below:

[DEPRECATION WARNING]: Invoking "apt" only once while using a loop via
squash_actions is deprecated. Instead of using a loop to supply multiple items
and specifying `name: "{{ item }}"`, please use `name: '{{ unused_packages }}'`
 and remove the loop. This feature will be removed in version 2.11. Deprecation
 warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

* l10n: updated Czech (cs)

contributors:
  1000101
  Michal Stanke
  darmozrac
  michaela-bot

updated from:
  repo: https://github.com/freedomofpress/securedrop-i18n
  revision: 570f2978a6bef4b542ded00541183a90fc0860a4

* Add Czech to list of supported languages

Also update the translation-test script so it gets the list of
supported locales from i18n_tool.py and will no longer have to be
updated when we change that list.

* Edit README section about documentation

Edits for readability and concision. Also remove last of language
related to RTD migration.

* Add pip, setuptools and supervisor to develop requirements

These requirements were installed via pip in the Dev/Admin Docker containers. We should instead add them to the develop-requirements file and pin hashes.

The `--allow-unsafe` flags should not impact security: it is to allow pip to pin setuptools and pip-tools (see issue freedomofpress#4686).

* Pin hashes for test-requirements.txt

Also adds --allow-unsafe to allow pinning hashes of setuptools in test-requirements.txt

The `--allow-unsafe` flags should not impact security: it is to allow pip to pin setuptools and pip-tools (see issue freedomofpress#4686).

* Bump tor to 0.4.1.6

* docs: update the list of supported languages

* Fix column/button alignment problems on source start page

Certain combinations of language and viewport size could result in the
two columns on the source start page not lining up with each
other. This changes the columns' justify-content property to push the
top and bottom containers apart, and controls the borders and content
of the buttons to align them better.

* Fix link to Tails update guide

The page moved to a new URL without a redirect

* Bump app tests timeout to 15 minutes

(See freedomofpress#4691)

* Reduce the space between index row content and buttons

* Fix securedrop-admin crash when no operation given under Python 3

Under Python 3, running securedrop-admin with no positional argument
results in an ugly error, due to https://bugs.python.org/issue16308.

Under Python 3.7, we could simply add dest and required arguments to
the add_subparsers call, but required isn't available in Python 3.5,
so would break under Tails 3.

* Remove Tails 3.x admin virtualenv when upgrading to Tails 4.x

Fixes freedomofpress#4910

* Add tests for tails 3 admin venv removal in tails 4

* Check that securedrop-admin has a good virtualenv

Adds a "checkenv" argument to admin/bootstrap.py, used in
securedrop-admin to ensure that the admin virtualenv is
functional. Also make sure bootstrap.py runs under Python 3.

* Fix pipenv docs URLs (HTTPS on redirect is broken)

* Restore dropped virtualenv activation in securedrop-admin

* Fix Tails detection in Python 3 + Tails 3.x

The final comparison failed to account for the fact that the
subprocess module returns a byte literal.

Incorrect Tails detection caused torify not to be used, which
caused a network error during the creation of the virtualenv
on Tails 3.x.

Decoding the string first reduces the likelihood of such
coding errors.

Fixes freedomofpress#4925

* Add 1.0.0->1.1.0 upgrade guide

Includes instructions to rebuild the venv after the Tails 4 upgrade,
so securedrop-admin and the graphical updater continue to work with
Python 3.

Removed older upgrade guides along the way per standard procedure.

* Delete newly created virtualenv if creation fails

Fixes freedomofpress#4929: If a virtualenv is created, but a transient network error prevents core dependencies to be installed in this virtualenv (for example, pip3), we should delete the virtualenv to prevent subsequent `./securedrop-admin setup` runs from failing.

* Add "Troubleshooting securedrop-admin" section

* Add test to ensure envsetup deletes venv if exception is raise.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
2 participants
You can’t perform that action at this time.