@expressify puts shinylive into a strange state

[wch]

In the dev version of Shinylive, @expressify puts it in a strange state where it doesn't seem to recognize changes to the function body.

For example, with this app, if you change the body of the card function and restart the app, the card doesn't change.

Live example

from shiny.express import expressify, render, ui

@expressify
def card(i: int):
    with ui.card():
        with ui.span():
            "This is card "
            ui.span(str(i), style="color: red;")

card(1)
card(2)

expressify-shinylive-half.mp4

Although the problem is happening in Shinylive, I think that's probably just a symptom of a deeper issue that's in the implementation of expressify.

[wch]

I also found that if you load the app, then change the function name to card1 (along with the function calls), then press the play button, it will do this:

>>> Traceback (most recent call last):
  File "<exec>", line 379, in _start_app
  File "<exec>", line 357, in __init__
  File "/lib/python3.10/site-packages/shiny/express/_run.py", line 42, in wrap_express_app
    app_ui = run_express(file).tagify()
  File "/lib/python3.10/site-packages/shiny/express/_run.py", line 93, in run_express
    exec(
  File "/home/pyodide/app_vjx2c2sazx2j5o8fraqf/app.py", line 5, in <module>
    def card1(i: int):
  File "/lib/python3.10/site-packages/shiny/express/expressify_decorator/_expressify.py", line 128, in expressify
    return decorator(fn)
  File "/lib/python3.10/site-packages/shiny/express/expressify_decorator/_expressify.py", line 103, in decorator
    fcode = _transform_body(cast(types.FunctionType, fn))
  File "/lib/python3.10/site-packages/shiny/express/expressify_decorator/_expressify.py", line 169, in _transform_body
    raise RuntimeError(
RuntimeError: Failed to find function 'card1' in AST. This should never happen, please file an issue!

[cpsievert]

Appears @render.express has the same issue

[jcheng5]

This has to do with the python stdlib module linecache. If you add these lines to the top, the problem goes away AFAICT:

import linecache
linecache.clearcache()

We look in the linecache for source code instead of on disk, for performance reasons but also because there was some mode I was running in where the source .py file was being generated as a temp file and somehow deleted before these decorators had a chance to run.

We might just need shinylive to call linecache.clearcache() as part of its teardown of apps, or something like that?

[jcheng5]

There's also a linecache.checkcache(filename=None); by default, it checks all of the files it's cached and flushes entries for files that have changed. Or you can have it check a specific file.

[wch]

I think we can use linecache.checkcache() on any file just before doing special evaluation of it?

[jcheng5]

I think you’ll have the same problem without special evaluation, if the app.py loads another .py that uses those decorators? Maybe linecache.checkcache() (with no args) when the run button is pushed?

[wch]

checkcache() and clearcache() seem to partially work. In the video here, it doesn't work if I change the body of a function, but seems to work if I change the name of a function. I'll have to poke at this some more.

clearcache-half.mp4

[jcheng5]

It doesn’t explain what you’re seeing, I don’t think, but we do also have our own cache here: https://github.com/posit-dev/py-shiny/blob/main/shiny/express/expressify_decorator/_expressify.py#L33

[wch]

Yes, it looks like the code_cache is the source of the problem. This might be a bug with this specific version of Pyodide. (I'm not sure if linecache is also causing problems.)

This is where it checks if the bytecode object fn.__code__ is in the cache. What I'm seeing is that, after the first time the app has been run, the test if fn.__code__ in code_cache evaluates to True in some cases even when the code is different.

py-shiny/shiny/express/expressify_decorator/_expressify.py

Lines 99 to 100 in 64d8036

	if fn.__code__ in code_cache:
	fcode = code_cache[fn.__code__]

Here's a minimal reprex.

import contextlib

def card():
    with contextlib.nullcontext():
        "This is text"

print(card)
print(card.__code__.__hash__())


def card():
    with contextlib.nullcontext():
        "This is other text"

print(card)
print(card.__code__.__hash__())

If I run this in my normal Python in a terminal, I get the following output:

<function card at 0x102799ee0>
5123763730382191169
<function card at 0x10279a020>
-5359592953186025408

This is what I would expect. However, if I run it in Pyodide (via Shinylive), this is what it looks like:

<function card at 0x17361d0>
1335897378
<function card at 0x17a5338>
1335897378

In Pyodide, both of the __code__ objects have the same hash, even though their contents are slightly different. (Other changes, like changing the name of the function, or removing the with statement, do result in a different hash.)

Shinylive using Pyodide 0.22.1. I also tested that code in their web console at https://pyodide.org/en/0.22.1/console.html and was able to reproduce the problem there as well.

However, when I tried a newer version, like https://pyodide.org/en/0.23.0/console.html, the hashes are different. So it might have been fixed in that version.