Include various client related strings in test data

[kushaldas]

Status

Ready for review

Description of Changes

Fixes freedomofpress/securedrop-client#1080

Adds many different test strings to test securedrop-client text formatting.

Testing

• make dev should have 2 sources
• NUM_SOURCES=10 make dev should create 10 sources
• NUM_SOURCES=ALL make dev shoudl create 43 sources
• Use the securedrop-client against the local server and look many strange looking texts :)

Deployment

Any special considerations for deployment? Consider both:

1. Upgrading existing production instances.
2. New installs.

Checklist

If you made changes to the server application code:

• Linting (make lint) and tests (make test) pass in the development container

If you made changes to securedrop-admin:

• Linting and tests (make -C admin test) pass in the admin development container

If you made changes to the system configuration:

• Configuration tests pass

If you made non-trivial code changes:

• I have written a test plan and validated it for this PR

If you made changes to documentation:

• Doc linting (make docs-lint) passed locally

If you added or updated a code dependency:

Choose one of the following:

• I have performed a diff review and pasted the contents to the packaging wiki
• I would like someone else to do the diff review

[lgtm-com]

This pull request introduces 1 alert when merging fd8a0f2 into ecf6255 - view on LGTM.com

new alerts:

• 1 for Implicit string concatenation in a list

[lgtm-com]

This pull request introduces 1 alert when merging 69c070f into ecf6255 - view on LGTM.com

new alerts:

• 1 for Implicit string concatenation in a list

[lgtm-com]

This pull request introduces 1 alert when merging 211a394 into ecf6255 - view on LGTM.com

new alerts:

• 1 for Implicit string concatenation in a list

[sssoleileraaa]

I like this CLIENT_TEST_DATA env var solution. Nice work!

[sssoleileraaa]

Do you want to add all the different test strings from the wiki?

Up to you. I think since this is not default, it would be great to add all the test strings from https://github.com/freedomofpress/securedrop-client/wiki/Message-Test-Data and also maybe a couple from https://github.com/minimaxir/big-list-of-naughty-strings/blob/master/blns.txt as you see fit.

[kushaldas]

Do you want to add all the different test strings from the wiki?

Up to you. I think since this is not default, it would be great to add all the test strings from https://github.com/freedomofpress/securedrop-client/wiki/Message-Test-Data and also maybe a couple from https://github.com/minimaxir/big-list-of-naughty-strings/blob/master/blns.txt as you see fit.

@creviera I updated with more text form blns.txt, creating at least 20 sources will be nicer to consume all strings properly.

[lgtm-com]

This pull request introduces 1 alert when merging 63f9eeb into 514b9e9 - view on LGTM.com

new alerts:

• 1 for Syntax error

[sssoleileraaa]

this lgtm - linter test fails though:

create-dev-data.py:91:85: E0001: invalid syntax (<unknown>, line 91) (syntax-error)

might want to move the special strings into a txt file?

[kushaldas]

Our future test strings are so good that they are breaking Python linting tools 🐍🕺 🐍

********** Module specialstrings
specialstrings.py:63:85: E0001: invalid syntax (<unknown>, line 63) (syntax-error)
************* Module create-dev-data
create-dev-data.py:18:0: E0001: Cannot import 'specialstrings' due to syntax error 'invalid syntax (<unknown>, line 63)' (syntax-error)

[lgtm-com]

This pull request introduces 1 alert when merging 9a75a52 into 309f17e - view on LGTM.com

new alerts:

• 1 for Implicit string concatenation in a list

[sssoleileraaa]

so you're passing client_test in for source_count? could you instead pass num_sources and make CLIENT_TEST_DATA a boolean that says whether or not special strings should be used?

UPDATE
Let's talk more tomorrow about the design and how we imagine QA will use this feature as well as how we might use this for our screenshot tests. It'll also make it easier for me to review.

[redshiftzero]

Here's a version with Allie's suggested change above, which works nicely for me (feel free not to use this):

diff --git a/securedrop/create-dev-data.py b/securedrop/create-dev-data.py
index e689c6080..de5b8ea85 100755
--- a/securedrop/create-dev-data.py
+++ b/securedrop/create-dev-data.py
@@ -59,18 +59,17 @@ def main(staging=False):

         # Add test sources and submissions
         num_sources = int(os.getenv('NUM_SOURCES', 2))
+        client_test = bool(os.getenv('CLIENT_TEST_DATA', 0))
         for i in range(1, num_sources + 1):
             if i == 1:
                 # For the first source, the journalist who replied will be deleted
                 create_source_and_submissions(
-                    i, num_sources, journalist_who_replied=journalist_tobe_deleted
+                    i, num_sources, journalist_who_replied=journalist_tobe_deleted,
+                    client_test_data=client_test
                 )
                 continue
-            create_source_and_submissions(i, num_sources)
+            create_source_and_submissions(i, num_sources, client_test_data=client_test)

-        client_test = int(os.getenv('CLIENT_TEST_DATA', 0))
-        for i in range(1, client_test):
-            create_source_and_submissions(i, client_test, client_test_data=True)
         # Now let us delete one journalist
         db.session.delete(journalist_tobe_deleted)
         db.session.commit()

[sssoleileraaa]

further thoughts in here: #5224 (review)

[sssoleileraaa]

After my conversation with Ro about how this feature could be used during QA, I think it would make sense to have a follow-up discussion about a) populating production and staging servers with the same test data that we use here in our development environment so it can be used during QA, and b) what our default configuration of the server should be when testing.

Even though the change in this PR only applies to the dev container, I tried to think about how this would be used by QA in the following feedback:

• Like messages, we should be able to populate replies with all the different test data

• The default configuration should be the most broadly useful during test. For me, this means it should be fast so that I can test as I develop (this is less important for QA testers but it doesn't hurt), as well as easily reproducible to faciliate creating STRs when bugs are found.

  So far your change meets this criteria but I think we can take it a step further and make it the default, i.e. running make dev should use all the test data strings. This would mean that you could get rid of CLIENT_TEST_DATA all together and skip converting it into a Boolean. So, by default, you could just populate the client with as many sources as you need in order to show all strings in the test data file, and if NUM_SOURCES is set to more or less than that, you just cycle through the list. It would be convenient to only show one message and one reply by default that each show the same test data string.

• In the future it would be nice to also add to the default configuration special cases, such as:

  • a source with no messages or replies
  • a source with one message
  • a source with one message and file
  • a source with one file
  • a source with one reply

• In the future we could add a scalability-testing envrionment variable for configuring sources with many, many messages and replies as well.

[lgtm-com]

This pull request introduces 1 alert when merging 2fca9dd into 2c65008 - view on LGTM.com

new alerts:

• 1 for Implicit string concatenation in a list

[rocodes]

@kushaldas Tried the changes in 2fca9dd and imo they look good and work well. I'm not going to weigh in on whether make dev should make automatically with this number of sources (adds maybe 10 seconds to make time?), so I'll leave others to weigh in on the right amount of test data to be the default, but I think this looks good.

A few other points that came out of my conversation with @creviera:

• There could be a useful integration between this and qa_loader down the line
• The idea of reproducible data being populated during make dev paves the way nicely for more automated testing/screenshots in future too.

[eloquence]

Personally I like the behavior as it is now; for new developers using the development environment, it doesn't add a huge performance penalty (creation of sources is pretty zippy; it takes 15 seconds to create 43 sources on my older X1), and for more experienced devs, it's easy enough to add NUM_SOURCES. I think in general there's some value in doing more testing with a more highly populated environment by default.

[eloquence]

The one caveat is that all replies are identical right now. We've sometimes seen issues that arise only in messages, or only in replies, such as the recent styling differences reported in freedomofpress/securedrop-client#1082 (comment), or the context menu bug in freedomofpress/securedrop-client#1064 . I don't think this absolutely needs to be resolved in this PR, but it would be IMO preferable for messages and replies to both use the special strings file.

[sssoleileraaa]

Yeah, I agree that we can build on top of this change and that it doesn't need to do all the things. To be clear about what my review comment is requesting for changes:

• Like messages, we should be able to populate replies with all the different test data

This is to catch bugs that only occur in the reply bubbles. Reply bubbles descend from the same class as message bubbles but they introduce new GUI behavior, like adding failure messages and changing the color bar, and so can we might miss a styling bug if we only show test data strings in message bubbles.

Also, this would allow us to see special strings in preview snippets, which would allow us to catch more bugs.

This can be made into a smaller issue that can be done outside of this PR.

• The default configuration should be the most broadly useful during test. For me, this means it should be fast so that I can test as I develop (this is less important for QA testers but it doesn't hurt), as well as easily reproducible to faciliate creating STRs when bugs are found.

So far your change meets this criteria but I think we can take it a step further and make it the default, i.e. running make dev should use all the test data strings.

As Ro pointed out this would add a lot of time when starting make dev. The only problem I see with not doing this is that we will have to know the magic number to pass to NUM_SOURCES in order to see all the test strings.

This doesn't need to hold up your PR, but it is confusing as is, because when you run NUM_SOURCES=1 CLIENT_TEST_DATA=10 make dev you get 10 sources with 2 different special strings each in 2 different messages.

[eloquence]

The only problem I see with not doing this is that we will have to know the magic number to pass to NUM_SOURCES in order to see all the test strings.

I really don't think that's desirable - testers should not have to count up the number of test cases or look for a magic number somewhere. If we don't want to default to importing all, IMO there should be a symbolic way to specify this, e.g., NUM_SOURCES=ALL.

[sssoleileraaa]

Personally, I still prefer:

make dev - sets up a test server with all the strings in the test data file
NUM_SOURCES=1 make dev - sets up a test server with 1 source that has the special strings from the top of the test data file list
NUM_SOURCES=10 make dev - sets up a test server with 10 sources that have the special strings from the top of the test data file list (might not cycle through all the strings)
NUM_SOURCES=1000 make dev - sets up a test server with 1000 sources that have the special strings from the test data file list (cycles through all the strings multiple times)

And in the future... maybe something like:

NUM_MESSAGES=200 NUM_SOURCES=10 make dev - sets up a test server with 10 sources that each have 200 messages+replies that have special strings from the the test data file list (cycles through all the strings multiple times)

[sssoleileraaa]

Agreed, if we want to make make dev default to a smaller number of sources to speed things up, then NUM_SOURCES=ALL make dev is a nice solution.

[lgtm-com]

This pull request introduces 1 alert when merging 46a5d87 into beced1f - view on LGTM.com

new alerts:

• 1 for Implicit string concatenation in a list

[sssoleileraaa]

I think this needs to be kicked twice due to the test timeout issue.

[sssoleileraaa]

(referring to Too long with no output (exceeded 35m0s): context deadline exceeded)

[eloquence]

Taking a quick look at 46a5d87 ...

[lgtm-com]

This pull request introduces 1 alert when merging c83d8c4 into ebc193c - view on LGTM.com

new alerts:

• 1 for Implicit string concatenation in a list

[eloquence]

LGTM.

Test plan:

• make dev should have 2 sources
• NUM_SOURCES=10 make dev should create 10 sources
• ❌ NUM_SOURCES=ALL make dev should create 43 sources -- test plan is inaccurate
• NUM_SOURCES=ALL make dev creates the required number of sources to ingest all strings

Recent changes by Allie LGTM. May benefit from a squash before merge.

[lgtm-com]

This pull request introduces 1 alert when merging d370fb0 into ebc193c - view on LGTM.com

new alerts:

• 1 for Implicit string concatenation in a list

[conorsch]

OK to merge once CI passes. Thank you @creviera for your patient review on this one!

[eloquence]

(Squashed)

[conorsch]

Re-approving post-squash.

[lgtm-com]

This pull request introduces 1 alert when merging 9d82324 into ebc193c - view on LGTM.com

new alerts:

• 1 for Implicit string concatenation in a list

[eloquence]

Erm, that LGTM alert actually looks legit, looks like we're concatenating strings that should be separate per https://github.com/freedomofpress/securedrop-client/wiki/Message-Test-Data

[conorsch]

@eloquence Nice catch, I agree, we should fix that prior to merge. Thanks, LGTM: https://lgtm.com/projects/g/freedomofpress/securedrop/rev/pr-7c7409194c43f3b116ae35ee0fa18f076d1125dfh

[conorsch]

LGTLGTM, LGTM.