TestRunnerAgent: "TypeError: <...> is not JSON serializable" when register_run_keyword is not used

[edbrannin]

EDIT: Repro steps below.

If I pass an ElementTree object to a keyword, I get the below error.

I'm doing this in a for-loop, so it's causing excessive warnings to appear in the Test Execution Errors and keyword-logging portions of log.html.

What does RIDE want with JSON-serializing keyword arguments? If it's just for the "Running this keyword..." ticker, could we replace this exception with a repr() fallback?

TypeError: <Element EntityPayload at 0x45cbc60> is not JSON serializable
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\robotide\contrib\testrunner\TestRunnerAgent.py", line 206, in _send_socket
    self.streamhandler.dump(packet)
  File "C:\Python27\lib\site-packages\robotide\contrib\testrunner\TestRunnerAgent.py", line 430, in dump
    s = self._json_encoder(obj)
  File "C:\Python27\lib\json\encoder.py", line 209, in encode
    chunks = list(chunks)
  File "C:\Python27\lib\json\encoder.py", line 431, in _iterencode
    for chunk in _iterencode_list(o, _current_indent_level):
  File "C:\Python27\lib\json\encoder.py", line 332, in _iterencode_list
    for chunk in chunks:
  File "C:\Python27\lib\json\encoder.py", line 332, in _iterencode_list
    for chunk in chunks:
  File "C:\Python27\lib\json\encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "C:\Python27\lib\json\encoder.py", line 332, in _iterencode_list
    for chunk in chunks:
  File "C:\Python27\lib\json\encoder.py", line 442, in _iterencode
    o = _default(o)
  File "C:\Python27\lib\json\encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")```

[HelioGuilherme66]

Can you show an example test case?
This way we could include it into the unit tests (or acceptance tests).
(I would also like to know what does @mkorpela think about the issue and proposed fix.)

Thanks

[edbrannin]

I'll write up a minimal test case tonight or tomorrow.

Untested, typed-right-here guess:

* Settings *
Library    XML

* Test Cases *
Should Gracefully Pickle XML ElementTrees
    ${xml}=    Parse XML    <hovercraft><eels/></hovercraft>
    Run Keyword    My Keyword    ${xml}

* Keywords *
My Keyword
    [Arguments]    ${xml}
    Log    ${xml}

(In case it matters, I was using BuiltIn().run_keyword("My Keyword", xml) from a Python library)

Pre-patch behavior:

• Log a warning from the RIDE execution-listener plugin about ${xml} not being JSON-serializable every time "My Keyword" is invoked. For most spectacular results, invoke from a for-loop.

[HelioGuilherme66]

Sorry, I still cannot reproduce. Even with your other suggestion of: ${xml}= Evaluate re.compile('djdbdhs') re.
Tried with Linux, Python 2.7.10 and Windows 7, Python 2.7.11, both with Robot Framework 3.0 and 64bit versions.

[edbrannin]

OK, I got it to reproduce. The key elements are:

1. The test passes an object that cannot be JSON-serialized to a Python keyword.
2. The Python keyword passes that object to another keyword with BuiltIn().run_keyword()
3. The Python library does not register_run_keyword the requested keyword (whoops).

I've pasted my test code into a gist: https://gist.github.com/edbrannin/f4f5d1b01e67b93b8ef19622f2a27a60

...May as well also put it right here:

* Settings *
Library    lib.py

* Test Cases *
Should Gracefully Pickle Regex Pattern
    ${Pattern}=    Evaluate    re.compile('w4rsdg')    re
    My Keyword    ${Pattern}

# Lib.py
from robot.libraries.BuiltIn import BuiltIn, register_run_keyword

def my_keyword(thing):
    BuiltIn().run_keyword("Log", thing)

# Uncommenting this also resolves the issue.
# register_run_keyword(__name__, my_keyword)

This is on Windows with Python 2.7.11 x32, robotframework==2.9.2, robotframework-ride==1.5.2.1

I understand if you want to close this as wontfix because I made a mistake in my test library, but this still seems like a low-cost way to make the Test Runner Agent more robust against bad input.