Make test and suite messages available as plain text and HTML separately

[simonmeggle]

Test results merged with rebot contain by default HTML in the status message field.
It could be helpful to have an option to change that to text-only.

[pekkaklarck]

Possible solutions:

1. Make it possible to tell that the merge message should be plain text.
2. Make it possible to convert any message from HTML to plain text.
3. Decide not to do anything and expect tools using Robot APIs to handle HTML to plain text conversion.

I'm not a big fan of option 1 because it's not too generic (also other messages can be in HTML), it would make messages in normal log less good, and I don't like this kind of configuration too much in general. Option 3 means doing nothing which isn't ideal either so let's concentrate on option 2. For that we have have several possibilities:

1. Add separate message_as_html and message_as_text attributes (actual names subject to change) to test and suite objects. message would be as it's nowadays and these new ones would be read-only properties generated based on it.
2. Make message always plain text and add new html_message (name, again, can be something else). Could be implemented so that there's separate _message that contains the actual message and these others are generated based on it. Both message and html_message would thus need to be settable properties (they'd set that _message).
3. Add new utility functions to convert message into plain text and HTML.

Option 2 above sounds best to me but has a problem that it's slightly backwards incompatible because message changes. That ought to be fine in a major release, though. Regardless the approach, we'd then need to have a way to convert HTML to plain text. I wouldn't like to have any external dependencies so we'd need to implement it ourselves. Probably just stripping HTML tags would be fine.

[pekkaklarck]

Converting HTML to text ought to be pretty easy with html.parser. We'd only need to implement handle_data go get the text but could also easily implement e.g. <b> to * conversion by adding handle_start/endtag methods. Should fine-tune the implementation so that it works well with our merge message.

[pekkaklarck]

After thinking this a bit more, I believe going with the option 2 discussed above is the best idea. I set the issue title and labels accordingly. Next we'd need someone to implement this. I know I have other more important tasks for the forthcoming RF 5.0 and won't have time for this. I'd obviously be happy to help anyone interested to get started, review pull requests, and so on. If @simonmeggle or someone else is interested, let me know!

[pekkaklarck]

Added a label indicating help would be appreciated. Although this is far from trivial to implement, I also added a label telling that this is ok for new contributors.

[simonmeggle]

(I‘m currently on vacation) Pekka Klärck ***@***.***> schrieb am Mi. 1. Sept. 2021 um 11:53:

After thinking this a bit more, I believe going with the option 2 discussed above is the best idea. I set the issue title and labels accordingly. Next we'd need someone to implement this. I know I have other more important tasks for the forthcoming RF 5.0 and won't have time for this. I'd obviously be happy to help anyone interested to get started, review pull requests, and so on. If @simonmeggle <https://github.com/simonmeggle> or someone else is interested, let me know! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4068 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOPHQVRAYNOIVDMIERQW5TT7XSXJANCNFSM5C2RDUAQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

--

…

______________________________________________ *ELABIT GmbH* IT Monitoring | IT Automation | E2E Testing Simon Meggle (CEO) Gewerbering 3 | D-86504 Merching Phone +49 (8233) 2171680 | Mobile +49 (179) 5447309 ***@***.*** | www.elabit.de

[pekkaklarck]

Design option 2 discussed in my earlier comment would in practice mean something like this:

1. Create html_to_text utility. It can utilize html.parser as discussed above.

2. Add code like this to handle setting and getting message:

   def __init__(...):
       ...
       self._message = ''
   
   @property
   def message(self):
       if self._message.startswith('*HTML*'):
           return html_to_text(self._message[6:])    # html_to_text needs to be implemented
       return self._message
   
   @message.setter
   def message(self, message):
       self._message = message
   
   @property
   def htm_message(self):
       if self._message.startswith('*HTML*'):
           return self._message[6:]
       return html_escape(self._message)    # We already have html_escape
   
   @html_message.setter
   def html_message(self, message):
       self._message = message
           self._message = '*HTML* ' + message

Instead of storing message internally in _message attribute as a string with possible *HTML* prefix, we could store it as a tuple containing the message and a Boolean telling should it be interpreted as HTML or not. Yet another solution would be making it an object that contains most of the logic so that the properties above could be simplified.

If we wanted to, we could make message itself and object that would have separate attributes for getting the message as plain text and HTML. The object itself should obviously have string representation as well so that it could be used as a string in formatting.

The reason I want to discuss about the design is that we have #2549 about making it possible to make specify test and suite documentation as HTML. There we probably needed to have documentation available both as plain text and HTML as well. These needs are similar so we should use the same design in both places. That also means that the aforementioned html_to_text could be used in both cases and possible objects containing logic for returning either plain text or HTML version could also be reused.

[fixedforward]

@pekkaklarck Is someone already working on this? If not do you mind assigning me?

[pekkaklarck]

Apparently @simonmeggle hasn't worked on this lately so go ahead @makeevolution!

[fixedforward]

@simonmeggle @pekkaklarck So I'm investigating this, and was thinking about the HTML_to_text function. Would it be something like if the original message is "*HTML* Number is not my <b>MAGIC</b> number", then the output is "Number is not my MAGIC number"? If so, I played around with the html parser library and found the following code to work. What do you think? Maybe I can make a PR if you agree?

I'm also unsure about how to create the atests for this. How should I test the status message of rebot? Any ideas just to help me get started?

from html.parser import HTMLParser

class HTMLToText(HTMLParser):
    def __init__(self):
        super().__init__()
        self._HTMLAsText = ''

    def handle_data(self, data):
        self._HTMLAsText = self._HTMLAsText + data
    
    @property
    def HTMLAsText(self):
        return self._HTMLAsText

def html_to_text(HTML): 
    test = HTMLToText()
    test.feed(HTML)
    return test.HTMLAsText()