Merge 0.5.1 changes into master #2888
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
These checks were often missed due to being at the end of a section. Let's rely on the Ansible tasks in post_ubuntu_install_checks.yml and not expect admins to do these additional manual checks on each server.
Consistently use the term "third-party" throughout docs
Instead of duplicating the content which will be harder to maintain.
Containerize development environment
During a security audit of trying to track down dependencies, this was overlooked at first pass and makes the logic trickier to debug. Lets stop using meta dependencies and instead always use top-level dependencies.
move meta dep. from grsecurity role -> playbook
…ttext i18n: s/lazy_gettext/gettext/ for pybabel extract
docs: add warning about MAT being unmaintained
…-started docs: point to securedrop.club contribution guide
…ment-source docs: the user experience goal is more generic than just mobile
cleanup: s/javascript/JavaScript/ig
It is confusing that the custom logo is mentionned twice in the list of elements to prepare before running sdconfig, under two slightly different names (custom header vs custom logo). The details about the custom logo (size) are better included in a dedicated section to keep the list short.
It is not uncommon for network engineers working in a media to have the necessary expertise to translate the firewall recommendations and implement them on the hardware they already have. It is error prone to browse the long description using the pfSense web interface and figure out what needs to be implemented in an abstract way. A paragraph is added, precisely describing the intended rules, for the benefit of skilled network engineers and in the spirit of minimizing errors.
If sources arrive at the generate page, it is because they clicked on the "Submit Document" button in the index page. We remove the "USE EXISTING CODENAME" from the generate page because it means the exact opposite. If sources change their mind, they can go back to the index page. Removing this button is important for another reason: the wording may confuse sources. Some may think "EXISTING CODENAME" refers to the codename displayed in the generate page although it means the exact opposite. And the "NEW CODENAME" could also be interpreted as a way to create a codename that is different from the one displayed in the page (for instance because it is difficult to memorize). But it actually means the opposite. Removing this button is also a way to avoid confusion in the future. The previous wording of the buttons (using CONTINUE) was slightly confusing. And I merged the current wording because it sounded like it was an improvement but it turns out to be worse. This is a hint that the problem is not in the wording of the buttons.
Because they create controversy about the choice of the flags.
The language name is displayed from the locale data, as found in babel, in the native language (i.e français instead of French etc). It is capalized or not, depending on the locale conventions. This does not look pretty when displayed in a list: change to always capitalize.
…curedrop-admin sdconfig`
…logo docs: better installation/update instructions for the custom logo
source: mention gettor@torproject.org as an alternative
docs: abstract firewall rules definition
Fail fast on grsec tasks and make sure any subsequent tasks are run through grsecurity's highly annoying locked down protections.
During a code review, it was discovered that this code is no longer used. SecureDrop packages are now built with molecule/docker.
This code will never get triggered because we have checks earlier in the inventory scripts that will bomb out if the code is missing. We have enough scaffolding and checks in place to make this piece of helper code neglible at this point.
It was introduced manually in the past, for test purposes, and was left there.
Ahmad Gharbeia, Ali Boshanab, ButterflyOfFire, Erin McConnell, Gabriele Kahlout, Jasmine Khalil, Jennifer Helsby, kwadronaut, Ramy Raoof, Scharik Yousif, Thalia Rahme Review: https://weblate.securedrop.club/user/ramyraoof/ (localizationlab) https://weblate.securedrop.club/user/agharbeia/ (localizationlab)
[0.5.1] l10n: sync de_DE es_ES fr_FR nb_NO nl pt_BR zh_Hant tr it_IT ar translations
|
Here were the conflicts manually resolved: |
|
Thanks for reporting the conflicts explicitly, @redshiftzero—that's a lot of conflicts. I'll run through the merge locally in disposable branches and confirm that I see what you see. I'd like to understand why all those control files are conflicting.... |
|
hum, something's not right, I'm curious to understand why |
|
release/0.5 is the same as origin/master, there is that $ git --no-pager log --oneline --cherry-mark --right-only origin/master...origin/release/0.5 *nothing* But there are a few commits from release/0.5 that are not in develop $ git --no-pager log --oneline --cherry-mark --right-only origin/develop...origin/release/0.5 = e4fded23 SecureDrop 0.5 + 37ccc263 Merge pull request #2676 from freedomofpress/wip-dachary-0.5-arabic = 3906f5e4 Tests: Remove 'ar' from test_render_locales + 31ac7411 Fix tests after pulling out Arabic translations = c6cc6f58 l18n: arabic is postponed to the next version = a426c2a9 SecureDrop 0.5-rc6 + 267f50eb Merge pull request #2672 from freedomofpress/wip-i18n = 1799e4c4 l10n: sync ar de_DE es_ES fr_FR nb_NO nl pt_BR translations And a few more from master that are not in develop $ git --no-pager log --oneline --cherry-mark --right-only origin/develop...origin/master + 1c49bd81 Merge pull request #2689 from freedomofpress/release/0.5 = e4fded23 SecureDrop 0.5 + 37ccc263 Merge pull request #2676 from freedomofpress/wip-dachary-0.5-arabic = 3906f5e4 Tests: Remove 'ar' from test_render_locales + 31ac7411 Fix tests after pulling out Arabic translations = c6cc6f58 l18n: arabic is postponed to the next version = a426c2a9 SecureDrop 0.5-rc6 + 267f50eb Merge pull request #2672 from freedomofpress/wip-i18n = 1799e4c4 l10n: sync ar de_DE es_ES fr_FR nb_NO nl pt_BR translations + 365dc82b Merge pull request #2480 from freedomofpress/release/0.4.4 + d3fcd125 Merge pull request #2286 from freedomofpress/release/0.4.3 + 85a6a4da Merge pull request #2109 from freedomofpress/release/0.4.2 + 2ca05ba6 Merge pull request #2064 from freedomofpress/release/0.4.1 |
|
I think the following happened:
For the translations the right way to resolve the problem is to unconditionally use the files from the release/0.5.1 branch because weblate owns these files: there is no risk of split brain. CONFLICT (content): Merge conflict in securedrop/translations/pt_BR/LC_MESSAGES/messages.po CONFLICT (content): Merge conflict in securedrop/translations/nl/LC_MESSAGES/messages.po CONFLICT (content): Merge conflict in securedrop/translations/nb_NO/LC_MESSAGES/messages.po CONFLICT (content): Merge conflict in securedrop/translations/messages.pot CONFLICT (content): Merge conflict in securedrop/translations/fr_FR/LC_MESSAGES/messages.po CONFLICT (content): Merge conflict in securedrop/translations/es_ES/LC_MESSAGES/messages.po CONFLICT (content): Merge conflict in securedrop/translations/de_DE/LC_MESSAGES/messages.po CONFLICT (modify/delete): securedrop/translations/ar/LC_MESSAGES/messages.po deleted in HEAD and modified in release/0.5.1. Version release/0.5.1 of securedrop/translations/ar/LC_MESSAGES/messages.po left in tree. CONFLICT (content): Merge conflict in install_files/ansible-base/roles/tails-config/templates/desktop-source-icon.j2 CONFLICT (content): Merge conflict in install_files/ansible-base/roles/tails-config/templates/desktop-journalist-icon.j2 CONFLICT (modify/delete): install_files/ansible-base/roles/tails-config/templates/ar.po deleted in HEAD and modified in release/0.5.1. Version release/0.5.1 of install_files/ansible-base/roles/tails-config/templates/ar.po left in tree. |
|
The remainder of the divergence between develop and master is this (removed the files owned by weblate for clarity): $ git --no-pager diff --stat origin/develop...origin/master .travis.yml | 3 +- changelog.md | 4 +- docs/conf.py | 4 +- docs/development/i18n.rst | 4 +- docs/install.rst | 1 - docs/set_up_admin_tails.rst | 4 +- install_files/ansible-base/group_vars/all/securedrop | 2 +- install_files/ansible-base/securedrop-configure.yml | 2 +- install_files/securedrop-app-code/DEBIAN/control | 2 +- install_files/securedrop-app-code/usr/share/doc/securedrop-app-code/changelog.Debian | 28 +- install_files/securedrop-keyring/DEBIAN/control | 2 +- install_files/securedrop-ossec-agent/DEBIAN/control | 2 +- install_files/securedrop-ossec-server/DEBIAN/control | 2 +- molecule/builder/tests/vars.yml | 2 +- securedrop/test | 2 +- securedrop/tests/test_journalist.py | 2 +- securedrop/version.py | 2 +- |
|
Although we added Arabic in 0.5.1, we also did not add it to the page-layout tests, reason why there is no conflict and if we had, it would be the same sooooo, no harm done here. diff --git a/.travis.yml b/.travis.yml index 690a19f3..70769e4d 100644 --- a/.travis.yml +++ b/.travis.yml @@ -47,8 +47,7 @@ script: # are time consuming. # * en_US: source strings # * fr_FR: left-to-right translations - # * ar: right-to-left translations - - sh -c "export DISPLAY=:1 ; cd securedrop && PAGE_LAYOUT_LOCALES='en_US,ar,fr_FR' pytest -v tests --page-layout" + - sh -c "export DISPLAY=:1 ; cd securedrop && PAGE_LAYOUT_LOCALES='en_US,fr_FR' pytest -v tests --page-layout" - pip freeze -l - SECUREDROP_TESTINFRA_TARGET_HOST=travis testinfra -v testinfra/development/ after_success: |
|
The following files are version change only such as -securedrop_version: "0.5-rc5" +securedrop_version: "0.5" the diff from release/0.5.1 are unconditional used to resolve the conflict: changelog.md docs/conf.py docs/set_up_admin_tails.rst install_files/ansible-base/group_vars/all/securedrop install_files/securedrop-app-code/DEBIAN/control install_files/securedrop-keyring/DEBIAN/control install_files/securedrop-ossec-agent/DEBIAN/control install_files/securedrop-ossec-server/DEBIAN/control securedrop/version.py |
|
install_files/securedrop-app-code/usr/share/doc/securedrop-app-code/changelog.Debian does not contain useful information or history and the version from release/0.5.1 can prevail. --- a/install_files/securedrop-app-code/usr/share/doc/securedrop-app-code/changelog.Debian +++ b/install_files/securedrop-app-code/usr/share/doc/securedrop-app-code/changelog.Debian @@ -1,32 +1,8 @@ -securedrop-app-code (0.5-rc5) trusty; urgency=medium +securedrop-app-code (0.5) trusty; urgency=medium * See changelog.md - -- SecureDrop Team Sun, 03 Dec 2017 02:21:03 +0000 - -securedrop-app-code (0.5-rc4) trusty; urgency=medium - - * See changelog.md - - -- SecureDrop Team Sat, 02 Dec 2017 07:19:35 +0000 - -securedrop-app-code (0.5-rc3) trusty; urgency=medium - - * See changelog.md - - -- SecureDrop Team Fri, 01 Dec 2017 01:30:56 +0000 - -securedrop-app-code (0.5-rc2) trusty; urgency=medium - - * See changelog.md - - -- SecureDrop Team Wed, 29 Nov 2017 19:59:52 +0000 - -securedrop-app-code (0.5-rc1) trusty; urgency=medium - - * See changelog.md - - -- SecureDrop Team Wed, 29 Nov 2017 00:36:28 +0000 + -- SecureDrop Team Tue, 05 Dec 2017 17:39:00 +0000 securedrop-app-code (0.4.4) trusty; urgency=medium |
|
The remainer of the files are about removing the Arabic translation because it was not ready and for all of them the version from release/0.5.1 should prevail since Arabic was restored. diff --git a/docs/development/i18n.rst b/docs/development/i18n.rst
index a609c7b3..1453fd7f 100644
--- a/docs/development/i18n.rst
+++ b/docs/development/i18n.rst
@@ -152,7 +152,7 @@ to the ``develop`` branch via pull requests for merge on a regular basis.
$ git fetch lab
$ git checkout -b wip-i18n origin/develop
$ git checkout lab/i18n -- securedrop/translations \
- install_files/ansible-base/roles/tails-config/templates/{ar,nl,fr,de_DE,nb_NO,pt_BR,es_ES}.po
+ install_files/ansible-base/roles/tails-config/templates/{nl,fr,de_DE,nb_NO,pt_BR,es_ES}.po
$ git add translations
$ vagrant ssh development
$ cd /vagrant/securedrop ; ./manage.py --verbose translate-desktop --compile
@@ -165,7 +165,7 @@ Verify the translations are not broken:
$ vagrant ssh development
$ cd /vagrant/securedrop
- $ PAGE_LAYOUT_LOCALES='ar,de_DE,es_ES,fr_FR,nb_NO,nl,pt_BR' \
+ $ PAGE_LAYOUT_LOCALES='de_DE,es_ES,fr_FR,nb_NO,nl,pt_BR' \
pytest -v --page-layout tests/pages-layout
Go to https://github.com/freedomofpress/securedrop and propose a pull request.
diff --git a/docs/install.rst b/docs/install.rst
index 3e405f06..21b7baab 100644
--- a/docs/install.rst
+++ b/docs/install.rst
@@ -31,7 +31,6 @@ Localization of the source and journalist interfaces
The source and journalist interface are translated in the following
languages:
-* Arabic (ar)
* German (de_DE)
* Spanish (es_ES)
* French (fr_FR)
diff --git a/install_files/ansible-base/securedrop-configure.yml b/install_files/ansible-base/securedrop-configure.yml
index 535269a4..0c08eb16 100644
--- a/install_files/ansible-base/securedrop-configure.yml
+++ b/install_files/ansible-base/securedrop-configure.yml
@@ -112,7 +112,7 @@
- name: securedrop_supported_locales
# the list is from the securedrop/translations repository
- prompt: Space separated list of additional locales to support (ar de_DE en_US es_ES fr_FR nb_NO nl)
+ prompt: Space separated list of additional locales to support (de_DE en_US es_ES fr_FR nb_NO nl)
default: ""
private: no
diff --git a/securedrop/test b/securedrop/test
index b15e71a5..38d98425 100755
--- a/securedrop/test
+++ b/securedrop/test
@@ -19,7 +19,7 @@ trap cleanup EXIT
mkdir -p "/tmp/test-results/logs"
-export PAGE_LAYOUT_LOCALES="en_US,ar,fr_FR"
+export PAGE_LAYOUT_LOCALES="en_US,fr_FR"
pytest \
--page-layout \
--durations 10 \
diff --git a/securedrop/tests/test_journalist.py b/securedrop/tests/test_journalist.py
index 330cb563..5355766f 100644
--- a/securedrop/tests/test_journalist.py
+++ b/securedrop/tests/test_journalist.py
@@ -1198,7 +1198,7 @@ class TestJournalistApp(TestCase):
try:
if supported:
del config.SUPPORTED_LOCALES
- config.SUPPORTED_LOCALES = ['en_US', 'fr_FR', 'ar']
+ config.SUPPORTED_LOCALES = ['en_US', 'fr_FR']
source, _ = utils.db_helper.init_source()
self._login_user()
|
|
As a conclusion the conflict resolution is actually very simple: we can override any change in master with the latest from release/0.5.1. And I'll make sure release/0.5.1 is proposed to be merged back to develop this time. I did the merge like so: All conflicts automatically resolved by prefering the version from release/0.5.1 with: git checkout -b wip-master origin/master git merge -m 'merge release/0.5.1 into master' -s recursive -X theirs origin/release/0.5.1 Manually restoring deleted Arabic translation with updated version from release/0.5.1 Conflicts: install_files/ansible-base/roles/tails-config/templates/ar.po securedrop/translations/ar/LC_MESSAGES/messages.po And pushed it to https://github.com/freedomofpress/securedrop/tree/wip-dachary-master for comparing with this branch: $ git diff --stat origin/resolve-conflicts-051-into-master origin/wip-dachary-master changelog.md | 16 ++++++------- securedrop/translations/fr_FR/LC_MESSAGES/messages.po | 102 +++++++++++++++++++++++++++++++++++++++--------------------------------------- securedrop/translations/messages.pot | 1 + 3 files changed, 60 insertions(+), 59 deletions(-) These are only whitespace changes but we should not have them otherwise we'll get conflicts. And the whitespace changes in the French translation are actually significant because they are small non breakable space and should not be converted to space. |
ghost
suggested changes
Jan 18, 2018
|
I was able to merge Otherwise I expect we're running the same commands. To be clear, I tried (yes, some of these commands are redundant, just trying to be explicit): |
|
Here is what I have (virtualenv) loic@raise:~/software/securedrop$ git clone https://github.com/freedomofpress/securedrop s Cloning into 's'... remote: Counting objects: 45884, done. remote: Compressing objects: 100% (147/147), done. remote: Total 45884 (delta 75), reused 138 (delta 46), pack-reused 45672 Receiving objects: 100% (45884/45884), 64.99 MiB | 2.71 MiB/s, done. Resolving deltas: 100% (30098/30098), done. (virtualenv) loic@raise:~/software/securedrop$ cd s # .... edit .git/config ... (virtualenv) loic@raise:~/software/securedrop$ git config --get pull.rebase true (virtualenv) loic@raise:~/software/securedrop/s$ git fetch (virtualenv) loic@raise:~/software/securedrop/s$ git checkout master Branch master set up to track remote branch master from origin. Switched to a new branch 'master' (virtualenv) loic@raise:~/software/securedrop/s$ git pull origin master From https://github.com/freedomofpress/securedrop * branch master -> FETCH_HEAD Current branch master is up to date. (virtualenv) loic@raise:~/software/securedrop/s$ git checkout -b try-merging-0.5.1-into-master Switched to a new branch 'try-merging-0.5.1-into-master' (virtualenv) loic@raise:~/software/securedrop/s$ git merge origin/release/0.5.1 Removing testinfra/vars/apptestclient.yml Auto-merging securedrop/version.py CONFLICT (content): Merge conflict in securedrop/version.py Auto-merging securedrop/translations/pt_BR/LC_MESSAGES/messages.po CONFLICT (content): Merge conflict in securedrop/translations/pt_BR/LC_MESSAGES/messages.po Auto-merging securedrop/translations/nl/LC_MESSAGES/messages.po CONFLICT (content): Merge conflict in securedrop/translations/nl/LC_MESSAGES/messages.po Auto-merging securedrop/translations/nb_NO/LC_MESSAGES/messages.po CONFLICT (content): Merge conflict in securedrop/translations/nb_NO/LC_MESSAGES/messages.po Auto-merging securedrop/translations/messages.pot CONFLICT (content): Merge conflict in securedrop/translations/messages.pot Auto-merging securedrop/translations/fr_FR/LC_MESSAGES/messages.po CONFLICT (content): Merge conflict in securedrop/translations/fr_FR/LC_MESSAGES/messages.po Auto-merging securedrop/translations/es_ES/LC_MESSAGES/messages.po CONFLICT (content): Merge conflict in securedrop/translations/es_ES/LC_MESSAGES/messages.po Auto-merging securedrop/translations/de_DE/LC_MESSAGES/messages.po CONFLICT (content): Merge conflict in securedrop/translations/de_DE/LC_MESSAGES/messages.po CONFLICT (modify/delete): securedrop/translations/ar/LC_MESSAGES/messages.po deleted in HEAD and modified in origin/release/0.5.1. Version origin/release/0.5.1 of securedrop/translations/ar/LC_MESSAGES/messages.po left in tree. Auto-merging securedrop/tests/test_journalist.py CONFLICT (content): Merge conflict in securedrop/tests/test_journalist.py Removing securedrop/static/i/flags/zh.png Removing securedrop/static/i/flags/nl.png Removing securedrop/static/i/flags/nb.png Removing securedrop/static/i/flags/fr.png Removing securedrop/static/i/flags/es.png Removing securedrop/static/i/flags/en.png Removing securedrop/static/i/flags/de.png Removing securedrop/static/i/flags/br.png Removing securedrop/static/i/flags/ar.png Removing securedrop/sass/modules/_flag.sass Auto-merging securedrop/sass/modules/_button.sass Auto-merging securedrop/sass/libraries/_normalize.sass Auto-merging securedrop/sass/libraries/_font-awesome-iconography.sass Removing securedrop/requirements/securedrop-requirements.txt Auto-merging securedrop/requirements/securedrop-app-code-requirements.in Auto-merging securedrop/bin/test Auto-merging molecule/builder/tests/vars.yml CONFLICT (content): Merge conflict in molecule/builder/tests/vars.yml Auto-merging molecule/aws/tests/test_tor_interfaces.py Auto-merging install_files/securedrop-ossec-server/DEBIAN/control CONFLICT (content): Merge conflict in install_files/securedrop-ossec-server/DEBIAN/control Auto-merging install_files/securedrop-ossec-agent/DEBIAN/control CONFLICT (content): Merge conflict in install_files/securedrop-ossec-agent/DEBIAN/control Auto-merging install_files/securedrop-keyring/DEBIAN/control CONFLICT (content): Merge conflict in install_files/securedrop-keyring/DEBIAN/control Auto-merging install_files/securedrop-app-code/usr/share/doc/securedrop-app-code/changelog.Debian CONFLICT (content): Merge conflict in install_files/securedrop-app-code/usr/share/doc/securedrop-app-code/changelog.Debian Auto-merging install_files/securedrop-app-code/DEBIAN/control CONFLICT (content): Merge conflict in install_files/securedrop-app-code/DEBIAN/control Auto-merging install_files/ansible-base/securedrop-configure.yml Auto-merging install_files/ansible-base/roles/tails-config/templates/desktop-source-icon.j2 CONFLICT (content): Merge conflict in install_files/ansible-base/roles/tails-config/templates/desktop-source-icon.j2 Auto-merging install_files/ansible-base/roles/tails-config/templates/desktop-journalist-icon.j2 CONFLICT (content): Merge conflict in install_files/ansible-base/roles/tails-config/templates/desktop-journalist-icon.j2 CONFLICT (modify/delete): install_files/ansible-base/roles/tails-config/templates/ar.po deleted in HEAD and modified in origin/release/0.5.1. Version origin/release/0.5.1 of install_files/ansible-base/roles/tails-config/templates/ar.po left in tree. Removing install_files/ansible-base/roles/grsecurity/meta/main.yml Auto-merging install_files/ansible-base/group_vars/all/securedrop CONFLICT (content): Merge conflict in install_files/ansible-base/group_vars/all/securedrop Auto-merging docs/set_up_admin_tails.rst CONFLICT (content): Merge conflict in docs/set_up_admin_tails.rst Auto-merging docs/install.rst Removing docs/google_authenticator.rst Auto-merging docs/development/i18n.rst CONFLICT (content): Merge conflict in docs/development/i18n.rst Auto-merging docs/conf.py CONFLICT (content): Merge conflict in docs/conf.py Auto-merging changelog.md CONFLICT (content): Merge conflict in changelog.md Auto-merging .travis.yml Automatic merge failed; fix conflicts and then commit the result. |
|
Ah, let's use your branch |
False. The testing instructions I claimed above were inaccurate. Specifically, there's no While following your duplicated instructions in a clean repo, I can reproduce the conflicts you describe, @dachary. Apologies for sending you on a wild goose chase. |
ghost
mentioned this pull request
|
Closing in favor of #2889 |
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Status
Ready for review
Description of Changes
Resolves final checkbox in #2813
Testing
Do you notice any issues with how conflicts were resolved?
Deployment
Already in prod
Checklist
If you made changes to the app code:
If you made changes to the system configuration: