ConsoleMessageCollection doesn't collect all console messages

[marimeireles]

Checklist

• I added a descriptive title
• I searched for other issues and couldn't find a solution or duplication
• I already searched in Google and didn't find any good information or help

What happened?

We have a helper object to collect console messages:

pyscript/pyscriptjs/tests/integration/support.py

Lines 274 to 339 in b674515

	class ConsoleMessageCollection:
	"""
	Helper class to collect and expose ConsoleMessage in a Pythonic way.
	Usage:
	console.log.messages: list of ConsoleMessage with type=='log'
	console.log.lines: list of strings
	console.log.text: the whole text as single string
	console.debug.* same as above, but with different types
	console.info.*
	console.error.*
	console.warning.*
	console.all.* same as above, but considering all messages, no filters
	"""
	class View:
	"""
	Filter console messages by the given msg_type
	"""
	def __init__(self, console, msg_type):
	self.console = console
	self.msg_type = msg_type
	@property
	def messages(self):
	if self.msg_type is None:
	return self.console._messages
	else:
	return [
	msg for msg in self.console._messages if msg.type == self.msg_type
	]
	@property
	def lines(self):
	return [msg.text for msg in self.messages]
	@property
	def text(self):
	return "\n".join(self.lines)
	_COLORS = {
	"error": "red",
	"warning": "brown",
	}
	def __init__(self, logger):
	self.logger = logger
	self._messages = []
	self.all = self.View(self, None)
	self.log = self.View(self, "log")
	self.debug = self.View(self, "debug")
	self.info = self.View(self, "info")
	self.error = self.View(self, "error")
	self.warning = self.View(self, "warning")
	def add_message(self, msg):
	# log the message: pytest will capute the output and display the
	# messages if the test fails.
	category = f"console.{msg.type}"
	color = self._COLORS.get(msg.type)
	self.logger.log(category, msg.text, color=color)
	self._messages.append(msg)

I can't figure out why but this class doesn't collect all messages. Here's an example:

I'm trying to throw an error while using the REPL and catch the console output. Here's the code that I'm using to achieve that:

    def test_repl_error_ouput_console(self):
        self.pyscript_run(
            """
            <py-repl id="my-repl" auto-generate="true"> </py-repl>
            """
        )
        self.page.locator("py-repl").type("this is an error")
        self.page.locator("button").click()
        console_text = self.console.all.lines
        for t in console_text:
             print('🔥', t)
        print('☀️')
        self.page.on("console", lambda msg: print('🌸', msg.text))
        print('☀️')

This will produce the following output:

🔥 [pyscript/main] checking for py-confing
🔥 [py-config] config set: {autoclose_loader: true, runtimes: Array(1)}
🔥 [py-config] Initializing runtimes
🔥 [pyscript/pyodide] Runtime config: {name: pyodide-default, lang: python, src: https://cdn.jsdelivr.net/pyodide/v0.21.2/full/pyodide.js}
🔥 [pyscript/main] add py-loader
🔥 [py-loader] Loading runtime...
🔥 [pyscript/pyodide] Loading pyodide
🔥 Python initialization complete
🔥 [pyscript/pyodide] pyodide.loadPackage: micropip
🔥 [pyscript/pyodide] Loading micropip, pyparsing, packaging, distutils
🔥 [pyscript/pyodide] Loaded micropip, pyparsing, packaging, distutils
🔥 [pyscript/pyodide] importing pyscript.py
🔥 [pyscript/pyodide] pyodide loaded and initialized
🔥 [py-loader] Runtime created...
🔥 [py-loader] Initializing components...
🔥 [py-script] py-mount: found 0 elements
🔥 [py-loader] Initializing scripts...
🔥 [py-loader] Running post initializers...
🔥 [py-repl] element my-repl successfully connected
🔥 [py-loader] Closing
🔥 [py-script] Initializing py-* event handlers...
🔥 [pyscript/runtime] PyScript page fully initialized
☀️
☀️
PASSED[  3.06 console.error  ] [pyscript/base] PythonError: Traceback (most recent call last):
  File "/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception
  File "/lib/python3.10/asyncio/tasks.py", line 232, in __step
    result = coro.send(None)
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 506, in eval_code_async
    await CodeRunner(
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 241, in __init__
    self.ast = next(self._gen)
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 142, in _parse_and_compile_gen
    mod = compile(source, filename, mode, flags | ast.PyCF_ONLY_AST)
  File "<exec>", line 1
    this is an error
               ^^^^^
SyntaxError: invalid syntax

🌸 [pyscript/base] PythonError: Traceback (most recent call last):
  File "/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception
  File "/lib/python3.10/asyncio/tasks.py", line 232, in __step
    result = coro.send(None)
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 506, in eval_code_async
    await CodeRunner(
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 241, in __init__
    self.ast = next(self._gen)
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 142, in _parse_and_compile_gen
    mod = compile(source, filename, mode, flags | ast.PyCF_ONLY_AST)
  File "<exec>", line 1
    this is an error
               ^^^^^
SyntaxError: invalid syntax

As you can see the last console.error is not being caught by the ConsoleMessageCollection class... I wonder if it's some async method behind it because it's not printed sequentially (in between the sun emojis)...
Should we just add this method? page.on("console", ...) into our object and try to incorporate this there?
Or any idea on what's happening here? @antocuni

What browsers are you seeing the problem on? (if applicable)

No response

Console info

No response

Additional Context

No response

[antocuni]

uhm, this works on my machine:

    def test_repl_error_ouput_console(self):
        self.pyscript_run(
            """
            <py-repl id="my-repl" auto-generate="true"> </py-repl>
            """
        )
        self.page.locator("py-repl").type("this is an error")
        self.page.locator("button").click()
        expected_message = """\
[pyscript/base] PythonError: Traceback (most recent call last):
  File "/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception
  File "/lib/python3.10/asyncio/tasks.py", line 232, in __step
    result = coro.send(None)
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 506, in eval_code_async
    await CodeRunner(
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 241, in __init__
    self.ast = next(self._gen)
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 142, in _parse_and_compile_gen
    mod = compile(source, filename, mode, flags | ast.PyCF_ONLY_AST)
  File "<exec>", line 1
    this is an error
               ^^^^^
SyntaxError: invalid syntax
"""
        assert self.console.all.lines[-1] == expected_message
        assert self.console.error.lines[-1] == expected_message

        for line in self.console.all.lines:
            print('🔥', line)

And the error message is correctly printed.
However, it might be that the test passes by chance: you should keep in mind that the browser behavior is async w.r.t the test: you do self.page.locator("button").click() and then immediately look at the messages, so it is entirely possible that by the time you look at them, the error has not been triggered yet.
What happens if you put e.g. time.sleep(2) after the click()?

So, can you:

1. try to run the test which I pasted above
2. try to modify your original test by putting a time.sleep(2) after the click?

[marimeireles]

Thanks for the review.
I'll check this later this week. :)

[marimeireles]

@antocuni just so we don't loose track of it (maybe you prefer a new issue?)
Not all error logs are being sent to the console object we have in our code.
E.g.: self._page_errors for example, maybe more?
Cheers.

[antocuni]

@marimeireles PR #874 should fix it