Adds wordwrap and text selectability for messages and replies

[kushaldas]

Description

Fixes #815

Wraps the text at the default 70 length

Test Plan

Follow the STR at #998

Initial test data:

• Should be one line:

WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW

• Should be two lines:

WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW

• Should be one line:

.................................................................................................................................

• Should be two lines:

..................................................................................................................................

thisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashes

thisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashes

thisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashes

Checklist

If these changes modify code paths involving cryptography, the opening of files in VMs or network (via the RPC service) traffic, Qubes testing in the staging environment is required. For fine tuning of the graphical user interface, testing in any environment in Qubes is required. Please check as applicable:

• I have tested these changes in the appropriate Qubes environment
• I do not have an appropriate Qubes OS workstation set up (the reviewer will need to test these changes)
• These changes should not need testing in Qubes

If these changes add or remove files other than client code, the AppArmor profile may need to be updated. Please check as applicable:

• I have updated the AppArmor profile
• No update to the AppArmor profile is required for these changes
• I don't know and would appreciate guidance

[sssoleileraaa]

We need to make sure your fix does not reintroduce the issue of removing paragraphs. Also the lines should wrap when the reach teh width plus some padding of the QLabel - in this PR the lines wrap before they get close to the end of the label. I think we should continue to use Qt's wordwrapping feature and specifically fix the issue where long strings without spaces or dashes don't wrap.

[kushaldas]

I think we should continue to use Qt's wordwrapping feature and specifically fix the issue where long strings without spaces or dashes don't wrap.

@creviera any tips on how do you want to go ahead for this?

[kushaldas]

• One way would be to calculate the width of the font and multiply by the number of chars to find the number of chars can be shown in the current size, has to update in every resize. But, do we want to maintain that code?
• Or we mane the labels into a QTextArea, and make them look like QLabel

[kushaldas]

For a note, https://github.com/qt/qtbase/blob/60feaae1967aca5c7b1b0f6cb732962ddd1b2115/src/widgets/widgets/qlabel.cpp#L1197 is the code doing the wrap inside of the Qt.

[sssoleileraaa]

• One way would be to calculate the width of the font and multiply by the number of chars to find the number of chars can be shown in the current size, has to update in every resize. But, do we want to maintain that code?
• Or we mane the labels into a QTextArea, and make them look like QLabel

The first method varies quite a bit with the font that we're using, meaning the widest character is a lot wider than the smallest character, so we would have to add a newline before the very long string with no spaces or dashes gets close to the end of the speech bubble. I ran into this issue with elided text, so you can see how this is done there.

We've talked about the second way before, because it would give us more controller over the wrapping policy - with labels we are only given a boolean config to turn wrapping on or off.

I'll see if there's a third option this morning.

and we'd have to watch out any time we make changes to the

[sssoleileraaa]

After more research I think the cleanest thing to use is QTextEdit, and we should make a SecureQTextEdit class the way we did with SecureQLabel. And then we can update our speech bubble messages from SecureQLabel to SecureQTextEdit. This also means add a little CSS around speech bubble size to make sure it's fitted to the text.

[kushaldas]

One issue is how to automagically resize the speechbubble. I tried what I count on this branch.

Btw, we will get free copy text feature by using this.

[sssoleileraaa]

changing all our QLabels into QTextEdits will make this a bigger change to test, so I think instead of switching SecureQLabel to derive from a QTextEdit we should create an entirely new SecureQTextEdit class and use it only for speech bubbles for now. for instance, one regression is that source selection doesn't work if you click on the preview snippet area.

I thought updating the size policy of the speech bubbles would fit the textedit to the text. I can give this a try and see why it isn't working for you (might be an issue with a parent's size policy). also we shouldn't see the scrollbar for messages or replies.

[kushaldas]

changing all our QLabels into QTextEdits will make this a bigger change to test, so I think instead of switching SecureQLabel to derive from a QTextEdit we should create an entirely new SecureQTextEdit class and use it only for speech bubbles for now. for instance, one regression is that source selection doesn't work if you click on the preview snippet area.

I thought updating the size policy of the speech bubbles would fit the textedit to the text. I can give this a try and see why it isn't working for you (might be an issue with a parent's size policy). also we shouldn't see the scrollbar for messages or replies.

I currently just kept the same name SecureQLabel and used a QPlainTextEdit as the parent class of the SecureQLabel :)

[eloquence]

I agree with @creviera that this PR should not alter the behavior of anything other than speech bubbles, which is the only instance we know of where there is currently a rendering issue. If we change widgets we may also want to compare rendering performance with long conversations (>50 but <100 messages).

[sssoleileraaa]

@kushal feel free to undo the last commit but i wanted you to see what i mean by only changing the SpeechBubble by creating a separate SecureQPlainTextEdit class. Since this does nothing more than setPlainText, I think we could actually do without it all together and just create self.message = QPlainTextEdit(text), what do you think? Also I didn't get to the size policy issue, but I think you may need to override setSizeHint to the size of the underlying QTextDocument's height. Unfortunately the documentation on this is limited... I found a cycle in docs and forums where people say "use QLabel" if you want readonly text and to adjust to the size of the text, and at the same time say "use QTextDocument if you want to display larger sized text with more complexity (for us, the complexity is that we want to wrap words that have more characters than the width of the text box and contain no spaces or dashes).

[sssoleileraaa]

@kushaldas last commit is a solution that sets the height of the textedit based on number of lines of each block in the document and line height. this is not a finalized solution rather an idea i want you to take a look at. the solution is to (1) get the height of the QPlainTextEdit after setting its text, (2) set the height of the QPlainTextEdit to what we calculate, (3) set the height of speechbubble to the height of its child message widget. I was surprised that Qt didn't have a way to get the (real) number of lines of each block so maybe i'm missing something in the docs. Also the values like lineheight may need to be tweaked.

I'm not super happy with my implementation, but i think the overall idea we should consider keeping: create a generic way to get and set the height of a QPlainTextEdit based on the number of lines of text.

We need a new reviewer for this since I added commits to get the resizing of the speech bubbles to work

[sssoleileraaa]

Okay, now I'm happy with my implementation to fix the textedit resize issue, and I updated our unit tests, so this is ready for a review from someone other than me. I also updated the PR description to resolve #1052. We will need to add support for multi-message-select in a separate PR.

[sssoleileraaa]

How to test?

In addition to the test plan above, send messages and replies with the following strings (plus any other test strings you can think of):

one line

one line
two line

one line
two line
three line

thisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashes

Veggies es bonus vobis, proinde vos postulo essum magis kohlrabi welsh onion daikon amaranth tatsoi tomatillo melon azuki bean garlic.

Gumbo beet greens corn soko endive gumbo gourd. Parsley shallot courgette tatsoi pea sprouts fava bean collard greens dandelion okra wakame tomato. Dandelion cucumber earthnut pea peanut soko zucchini.

Turnip greens yarrow ricebean rutabaga endive cauliflower sea lettuce kohlrabi amaranth water spinach avocado daikon napa cabbage asparagus winter purslane kale. Celery potato scallion desert raisin horseradish spinach carrot soko. Lotus root water spinach fennel kombu maize bamboo shoot green bean swiss chard seakale pumpkin onion chickpea gram corn pea. Brussels sprout coriander water chestnut gourd swiss chard wakame kohlrabi beetroot carrot watercress. Corn amaranth salsify bunya nuts nori azuki bean chickweed potato bell pepper artichoke.

Nori grape silver beet broccoli kombu beet greens fava bean potato quandong celery. Bunya nuts black-eyed pea prairie turnip leek lentil turnip greens parsnip. Sea lettuce lettuce water chestnut eggplant winter purslane fennel azuki bean earthnut pea sierra leone bologi leek soko chicory celtuce parsley jícama salsify.

Celery quandong swiss chard chicory earthnut pea potato. Salsify taro catsear garlic gram celery bitterleaf wattle seed collard greens nori. Grape wattle seed kombu beetroot horseradish carrot squash brussels sprout chard.

Pea horseradish azuki bean lettuce avocado asparagus okra. Kohlrabi radish okra azuki bean corn fava bean mustard tigernut jícama green bean celtuce collard greens avocado quandong fennel gumbo black-eyed pea. Grape silver beet watercress potato tigernut corn groundnut. Chickweed okra pea winter purslane coriander yarrow sweet pepper radish garlic brussels sprout groundnut summer purslane earthnut pea tomato spring onion azuki bean gourd. Gumbo kakadu plum komatsuna black-eyed pea green bean zucchini gourd winter purslane silver beet rock melon radish asparagus spinach.

Beetroot water spinach okra water chestnut ricebean pea catsear courgette summer purslane. Water spinach arugula pea tatsoi aubergine spring onion bush tomato kale radicchio turnip chicory salsify pea sprouts fava bean. Dandelion zucchini burdock yarrow chickpea dandelion sorrel courgette turnip greens tigernut soybean radish artichoke wattle seed endive groundnut broccoli arugula.

Soko radicchio bunya nuts gram dulse silver beet parsnip napa cabbage lotus root sea lettuce brussels sprout cabbage. Catsear cauliflower garbanzo yarrow salsify chicory garlic bell pepper napa cabbage lettuce tomato kale arugula melon sierra leone bologi rutabaga tigernut. Sea lettuce gumbo grape kale kombu cauliflower salsify kohlrabi okra sea lettuce broccoli celery lotus root carrot winter purslane turnip greens garlic. Jícama garlic courgette coriander radicchio plantain scallion cauliflower fava bean desert raisin spring onion chicory bunya nuts. Sea lettuce water spinach gram fava bean leek dandelion silver beet eggplant bush tomato.

How to test select-copy-paste?

This supports single message select. Even if you select multiple messages, you can only right-click copy over the selected text you want to copy and then you can test pasting by pasting it into the reply box. We can look into deselecting text from speech bubbles when you click outside of them in a separate PR after we spend more time discussing in this week's UX meeting.

[sssoleileraaa]

I think this approach is a smart way to handle setting the height. In double checking this, I just noticed that it underestimates the number of lines (at least for me locally), e.g. I have a line_count of 4 for the following block which is actually taking up 5 lines in the UI:

(just noting this in case anyone else has seen this and has an idea why it might be happening)

I see this too. When trying to come up with a way to get the real number of lines, which will allow us to resize the QPlainTextEdit based on its contents, I made the following discoveries:

• A document's height is equal to its number of blocks: blockCount == height

• A block is a paragraph or blank line, e.g. the following example is 3 blocks:

  Cat ipsum dolor sit amet, scratch leg; meow for can opener to feed me leave hair on owner's clothes, dream about hunting birds i like fish.

  Cough hairball, eat toilet paper licks paws yet while happily ignoring when being called.

• A block's line count is equal to the number of blocks because a block's line count is always 1 (this was totally unexpected and i plan to dig into the C++ Qt code to figure out why on earth lineCount doesn't report the actual number of lines in a block), so if you add up each block's lineCount you'll end up with its blockCount, e.g. the following example is reported to have 3 lines, instead of 5:

  

• my original idea, which i committed to this PR, was to set the document's textWidth, see https://doc.qt.io/qt-5/qtextdocument.html#textWidth-prop, which says:

  The text width specifies the preferred width for text in the document. If the text (or content in general) is wider than the specified with it is broken into multiple lines and grows vertically. If the text cannot be broken into multiple lines to fit into the specified text width it will be larger and the size() and the idealWidth() property will reflect that.

  Unfortunately, lineCount continued to equal 1 for each block, no matter how many lines were in each block. My next idea was to look at each block's QTextLayout to see if it contained the actually number of lines in a block, see https://doc.qt.io/qt-5/qtextblock.html#layout, which says:

  Returns the QTextLayout that is used to lay out and display the block's contents.

  And see https://doc.qt.io/qt-5/qtextlayout.html#lineCount, which says:

  Returns the number of lines in this text layout.

  But lineCount returned 0 for each block's layout except for the first block, which was always 1. You can see this with the 3-block example above with the following print statements:

   print('document has {} block(s)'.format(self.blockCount()))
   for block_num in range(0, self.blockCount()):
       block = self.document().findBlockByNumber(block_num)
       print('block {} has {} line(s)'.format(block_num, block.lineCount())) # this will always be 1
       line_count = math.ceil(block.length() / self.document().idealWidth())
       total_line_count = total_line_count + line_count
       print("block {}'s layout has {} line(s)".format(block_num, block.layout().lineCount())) # this will always be 1 or 0

  which prints:

   document has 3 block(s)
   block 0 has 1 line(s)
   block 0's layout has 1 line(s)
   block 1 has 1 line(s)
   block 1's layout has 0 line(s)
   block 2 has 1 line(s)
   block 2's layout has 0 line(s)
  

• Since Qt doesn't provide a way to get the actual number of lines in a block (perhaps this works in C++ and the issue is with how these classes and methods were wrapped for Python -- would have to investigate further to verify), my next idea was to somehow get a total sum of the pixel width of each character in a block, and then divide that by the maximum pixel width of a line, and take the ceiling of that quotient. You can see this idea by looking at what we currently have committed to this PR -- in the block loop we get this by doing:

line_count = math.ceil(block.length() / self.document().idealWidth())
total_line_count = total_line_count + line_count

---

I think this approach is a smart way to handle setting the height. In double checking this, I just noticed that it underestimates the number of lines (at least for me locally), e.g. I have a line_count of 4 for the following block which is actually taking up 5 lines in the UI:

(just noting this in case anyone else has seen this and has an idea why it might be happening)

So now I can finally answer why this message is calculated to be 4 lines instead of 5. We currently don't correctly calculate the total sum of the pixel width of each character in a block, because block.length() only gets the total number of characters in a block rather than the pixel width of each character, include spaces, in a block. To clearly demonstrate this, see the following example:

The dots character length is 130 and the W's is 42. So next we need to loop through each character in the block, get its width, and total everything up. Then we need to divide by the maximum pixel width allowed for a line. I'm still not 100% sure idealWidth will provide this, but according to the docs, it should.

[redshiftzero]

Thanks for this detailed explanation! This makes sense - if idealWidth doesn't do the trick, one other idea that I had on Friday is using QFontMetrics::horizontalAdvance since it returns the pixel width of text for a given font.

[sssoleileraaa]

horizontalAdvance is what we want! you're right, that's what we did for get_elided_text. idealWidth is the ideal width in terms of number of characters. whoops, been away too long perhaps.

[sssoleileraaa]

Thinking more, that's what we can use for getting the total sum of the pixel width of each character in a block, but we'll want to use self.size().width() minus whatever the padding is to get the maximum size of a line.

[sssoleileraaa]

Looks like horizontalAdvance gives us a different pixel width for the dots and W example above. The width of the dots is 526 and the width of the Ws is 625.

Update

I found out that fontMetrics wasn't using the right font when calculating the widths so I set the font directly in the SecureQPlainTextEdit class and that did the trick... almost. For some reason I have to add 4 pixels to self.size().width() to get this to resize correctly, so I'll commit what I have so far and keep investigating. Also, fontMetrics().lineHeight() returns 26 for me, which isn't quite right. A hard-coded line-height of 20 seems to work much better for the calculated pixel height of the textedits. So I'll look into this as well.

But feel free to test this now to see how our example messages and replies look correct now... at least they do for me so far. I'll create an updated list of text strings to test. So far I've been using: #1050 (comment) and the following:

• Should be one line:

WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW

• Should be two lines:

WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW

• Should be one line:

.................................................................................................................................

• Should be two lines:

..................................................................................................................................

thisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashes

thisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashes

thisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashesthisisalongwordwithoutspacesordashes

[sssoleileraaa]

Looks like this isn't quite ready because horizontalAdvance returns an unexpected size for tabs, and there could be more special characters that aren't quite right. To see this you can test with the following message text:

0	0
000000	0
00000000
000000000000

This looks like:

But the length of each line, calculated using horizontalAdvance is incorrect. the lengths are as follows, in order of first line to last:

line 1=90
line 2=90
line 3=80
line 4=119

---

So here's what's left (cc @kushaldas, since you're in a diff timezone and in case you can take a look until I come back to this tomorrow):

• We need to update this PR's description with a link to a complete list of test strings to test against, for example, right now if you copy tables and anything with tabs, the speech bubbles aren't sized correctly
• We should try to figure out how to set line-height so that it is consistent across platforms since it seems like fontMetrics doesn't return a consistent or correct lineHeight (for me it returned 26 and see Adds wordwrap and text selectability for messages and replies #1050 (review)). Otherwise we should ensure 20 works for Qubes and create a separate issue.
• We should try to figure out why we need to add 4 or any value to self.size().width() when calculating the maximum pixel size allowed for each line in a document.

[kushaldas]

Here is a screenshot with lines including one TAB and then multiple TABs.
I am also adding the test string I used.

............................................................................................................................		.
............................................................................................................................							.

The second bubble has less height, in my mind it should be having more as the length is bigger.

[redshiftzero]

Excellent progress, this is looking much closer.

We should try to figure out why we need to add 4 or any value to self.size().width() when calculating the maximum pixel size allowed for each line in a document.

There is a document margin that is 4 pixels (self.document().documentMargin()), so we can compute that explicitly to remove the magic number

[redshiftzero]

(One relevant finding on the tab business I want to drop here is that QFontMetrics.size is another method that returns pixel width that allows one to pass a flag (Qt::TextExpandTabs) to expand tabs)

[sssoleileraaa]

We separated out adding a message-copy feature from this PR, which has already been merged (#1063). The work in this PR has entirely been focused on fixing the issue #815 (message copying was more of a side effect and distraction from the original problem since this was opened to solve #815).

We are leaving this PR open while we look at taking a different path to fixing the QLabel wrapping issue described in #815. @kushaldas and I will be pasting research notes in the issue instead of in this PR.

[kushaldas]

Update from today's research on the topic.

• https://code.qt.io/cgit/qt/qtbase.git/tree/src/widgets/widgets/qlabel.cpp#n1035 this is the function doing the actual drawing on the screen
• Tried with TextBrowser and can see similar level of breaking (I have screenshot attached below)
• Figthing with webview proper
  • I have to somehow restrict the document to grow in width and then increase the height accordingly, also asked in the mailing list about this problem.
  • JS must be stopped via https://doc.qt.io/qtforpython/PySide2/QtWebEngineWidgets/QWebEngineSettings.html

[kushaldas]

I think I have a break through. I will report back on the issue at night.

[kushaldas]

Created a related issue about the size of the message https://github.com/freedomofpress/securedrop-client/issues/1072

[kushaldas]

Now I am rendering the text inside of QWebEngineView, with some amazing CSS hack from @SaptakS, this way we are stopping the overflow of the text, can show the emoji and random other text properly.

I will work on the branch tomorrow to clean up more.

[eloquence]

Cross-referencing Allie's and Jen's comments here:
#815 (comment)

Per those comments I'm moving this PR to the near-term backlog for now. I would actually advocate in favor of closing it and keeping the branch, for the avoidance of confusion, since the scope has changed significantly since the PR was opened.

[redshiftzero]

Closing but let's keep the branch around