Import PyScript Version into Python Runtime #958

JeffersGlass · 2022-11-17T14:49:39Z

After Ted's PR earlier this week (#952), here's a small-ish PR to access that version in Python as well. It adds PyScript.__version__ and PyScript.version_info:

# namedtuple, mimics sys.version_info
>>> PyScript.version_info
version_info(year=2023, month=2, patch=1, releaselevel='dev')

# human-readable version, similar to what ipython and others do
>>> PyScript.__version__
'2023.02.1.dev'

This fixes #546 , and supercedes #833.

for more information, see https://pre-commit.ci

docs/reference/API/__version__.md

antocuni · 2022-11-17T14:54:04Z

pyscriptjs/src/python/pyscript.py

+            version_dict["releaselevel"] = version_parts[3]
+        except IndexError:
+            version_dict["releaselevel"] = ""
+


ah, this seems to contradict my previous comment. But then I don't understand how "normal" versions are supposed to like, with or without the .final?

So I figured if the "canonical" name of the version is "2022.09.1.RC2", then version_info should match that, as version_info(year=2022, month=9, minor=1, releaselevel='RC2')

But if the cannonical name of the version is "2022.09.1", then either

version-info should respect that the releaselevel is nothing version_info(year=2009, month=9, minor=1, releaselevel='') (this is what the PR currently does)

Or

since we've been using "2022.09.1" to mean the actual release of "2022.09.1", we could put a signifier in the releaselevel for this version_info(year=2009, month=9, minor=1, releaselevel='final')

The term "final" I'm not set on - could be "release" or something.

I am inclined towards final or release instead of "" but have no strong opinions to be honest.

JeffersGlass · 2022-11-17T14:59:18Z

Sadly, this breaks in pyodide.test.ts in a very strange way that I'm hoping someone can help me figure out.

These lines build fine - if you open a REPL you can see the two new values:

pyscript/pyscriptjs/src/pyodide.ts

Lines 74 to 75 in 2740e98

    
           this.run(pyscript as string); 
        
           this.run(`PyScript.set_version_info('${version}')`)

But pyodide.test.ts fails:

● PyodideRuntime › should check if runtime is an instance of abstract Runtime

  PythonError: Traceback (most recent call last):
    File "/lib/python3.10/site-packages/_pyodide/_base.py", line 435, in eval_code
      .run(globals, locals)
    File "/lib/python3.10/site-packages/_pyodide/_base.py", line 304, in run
      coroutine = eval(self.code, globals, locals)
    File "<exec>", line 1, in <module>
  NameError: name 'PyScript' is not defined

I can't for the life of me figure out why the second this.run seems to not be able to access the objects in the initial one, but only in jest.

Any ideas?

FabioRosado · 2022-11-17T18:37:41Z

Sadly, this breaks in pyodide.test.ts in a very strange way that I'm hoping someone can help me figure out.

These lines build fine - if you open a REPL you can see the two new values:

pyscript/pyscriptjs/src/pyodide.ts

Lines 74 to 75 in 2740e98

this.run(pyscript as string);

this.run(`PyScript.set_version_info('${version}')`)

But pyodide.test.ts fails:
● PyodideRuntime › should check if runtime is an instance of abstract Runtime

  PythonError: Traceback (most recent call last):
    File "/lib/python3.10/site-packages/_pyodide/_base.py", line 435, in eval_code
      .run(globals, locals)
    File "/lib/python3.10/site-packages/_pyodide/_base.py", line 304, in run
      coroutine = eval(self.code, globals, locals)
    File "<exec>", line 1, in <module>
  NameError: name 'PyScript' is not defined
I can't for the life of me figure out why the second this.run seems to not be able to access the objects in the initial one, but only in jest.

Any ideas?

Apologies for the delay Jeff! The pyodide unit tests are importing pyodide and loading it dynamically, I wonder if maybe that's why it's it's complainig that PyScript is not available?

@madhur-tandon perhaps you have more context into this? 🤔

JeffersGlass · 2022-11-17T20:08:23Z

Thanks @FabioRosado !

For an extra bit of weirdness, if I change that bit of loadInterpreter to:

this.run(pyscript as string);
logger.info("Dir Results: " + this.run('dir()'))

Then this shows up in the jest logs:

console.info
    [pyscript/pyodide] Dir Results: ['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'pyversion', 'version_info']

So dir() is running fine... but with none of the symbols/globals that should be there after running pyscript.py? Very odd.

JeffersGlass · 2022-11-17T20:36:12Z

Ok some further oddities which make me think it's something odd about async interactions between pyodide and jest. Both of the following examples work when building for the browser, but...

This (the above example) breaks jest:

//main.ts
async afterRuntimeLoad(){
    ...
    await runtime.loadInterpreter();
    ...
    }

//pyodide.ts
async loadInterpreter(){
    ...
    this.run(pyscript as string);
    this.run(`PyScript.set_version_info('${version}')`)
    ...
    }

BUT this works in jest:

//main.ts
async afterRuntimeLoad(){
    ...
    await runtime.loadInterpreter();
    runtime.run(`PyScript.set_version_info('${version}')`)
    ...
    }

//pyodide.ts
async loadInterpreter(){
    ...
    this.run(pyscript as string);
    ....
    }

🤯

Not proposing this as a solution, it's just... odd and interesting.

FabioRosado · 2022-11-17T20:47:43Z

That's interesting, the pyodide tests are doing this:

pyscript/pyscriptjs/tests/unit/pyodide.test.ts

Line 41 in 0e1c396

await runtime.loadInterpreter();

I wonder if this is related? 🤔 Wonder if perhaps runtime didn't finish load (not sure if this even makes sense haha)

JeffersGlass · 2022-11-17T21:23:21Z

I figured it out!

jest is using a mocked version of pyscript.py for the tests:
https://github.com/JeffersGlass/pyscript/blob/803988342304d576425d48fd79ce3ebc50047226/pyscriptjs/jest.config.js#L19-L21

And that mock is.... empty!
https://github.com/JeffersGlass/pyscript/blob/803988342304d576425d48fd79ce3ebc50047226/pyscriptjs/__mocks__/fileMock.js#L12

So of course "PyScript isn't defined".

I think the solution is: change fileMock.js so that it copies and exports the contents of PyScript.py instead of "". I think this gives us the best approximation of the in-browser environment going forward. I'm just not sure the best way to do that... but I'll figure it out. Unless @madhur-tandon has suggestions?

FabioRosado · 2022-11-17T21:43:13Z

Oh dang! That's a great find, awesome work!

Copying over seems like a good idea or we could copy only the needed methods if that works, what do you think?

madhur-tandon · 2022-11-18T05:18:35Z

@JeffersGlass @FabioRosado

Sorry for the late reply. Yeah, I did the mocking thing with an empty string because jest
threw errors when trying to import a .py script.

The project overall builds without failure since our rollup.config.js has

pyscript/pyscriptjs/rollup.config.js

Lines 44 to 46 in 0e1c396

    
           string({ 
        
             include: "./src/**/*.py", 
        
           }),

aka the contents of .py imports are interpreted as raw strings. If we can somehow apply this setting to jest, that will be the best solution IMO but a JS ninja 🥷 could probably weigh in on how to do that.

Else, it seems like copying relevant stuff that is needed into the mocked string should be decently sufficient I guess.

FabioRosado · 2022-11-18T10:38:24Z

@JeffersGlass @FabioRosado

Sorry for the late reply. Yeah, I did the mocking thing with an empty string because jest threw errors when trying to import a .py script.

The project overall builds without failure since our rollup.config.js has

pyscript/pyscriptjs/rollup.config.js

Lines 44 to 46 in 0e1c396

string({

include: "./src/**/*.py",

}),

aka the contents of .py imports are interpreted as raw strings. If we can somehow apply this setting to jest, that will be the best solution IMO but a JS ninja 🥷 could probably weigh in on how to do that.

Else, it seems like copying relevant stuff that is needed into the mocked string should be decently sufficient I guess.

You can get jest to accept .py files with the config:

    moduleFileExtensions: ["js", "mjs", "cjs", "jsx", "ts", "tsx", "json", "node", "py"],

But then the tests explode because of the python imports

SyntaxError: /Users/fabiorosado/github/pyscript/pyscriptjs/src/python/pyscript.py: Unexpected token, expected "from" (2:0)

      1 | import ast
    > 2 | import asyncio

Maybe we go for the easier path and expand the mocked file 😄

JeffersGlass · 2022-11-18T15:10:52Z

Maybe we go for the easier path and expand the mocked file

Thanks y'all - for now, I've added __mocks__/_pyscript.js as an explicit mock of pyscript.py that jest can use at test time. It loads the contents of "pyscript.py" and exports them as a string.

pyscript/pyscriptjs/__mocks__/_pyscript.js

Lines 13 to 14 in af24043

    
           const fs = require('fs') 
        
           module.exports = fs.readFileSync('./src/python/pyscript.py', 'utf8');

All other .py files are still caught by the original mapping and mocked with fileMock.js (i.e. as an empty string), but I've added a console warning there that files are being mocked as "" for the next time someone runs across this.

pyscript/pyscriptjs/__mocks__/fileMock.js

Lines 13 to 16 in af24043

    
            console.warn(`.py files that are not explicitly mocked in \ 
        
            jest.config.js and /__mocks__/ are mocked as empty strings`) 
        
            module.exports = ""

pyscriptjs/__mocks__/fileMock.js

FabioRosado · 2022-11-18T17:50:41Z

pyscriptjs/__mocks__/_pyscript.js

+ */
+
+const fs = require('fs');
+module.exports = fs.readFileSync('./src/python/pyscript.py', 'utf8');


Apologies a bit late to the party but... NICE! 😄

JeffersGlass and others added 8 commits November 16, 2022 13:56

Rebase from main

c2676cc

Fix test

62c5a3e

Split match string in test

8e8aa34

version_info year/month/patch are ints

062f2ef

'patch' -> 'minor' version

f90025d

Add docs

834bd95

Add python unit test

ad92027

[pre-commit.ci] auto fixes from pre-commit.com hooks

2740e98

for more information, see https://pre-commit.ci

antocuni approved these changes Nov 17, 2022

View reviewed changes

Docs fixes

8039883

JeffersGlass added the help wanted Extra attention is needed label Nov 17, 2022

Mock pyscript.py with __mocks__/pyscript.js

af24043

JeffersGlass requested a review from FabioRosado November 18, 2022 15:10

madhur-tandon approved these changes Nov 18, 2022

View reviewed changes

madhur-tandon reviewed Nov 18, 2022

View reviewed changes

pyscriptjs/__mocks__/fileMock.js Outdated Show resolved Hide resolved

Address comments (typos, formatting)

a34c4d1

JeffersGlass merged commit 687b93d into pyscript:main Nov 18, 2022

FabioRosado reviewed Nov 18, 2022

View reviewed changes

JeffersGlass removed the help wanted Extra attention is needed label Nov 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Import PyScript Version into Python Runtime #958

Import PyScript Version into Python Runtime #958

JeffersGlass commented Nov 17, 2022 •

edited

Loading

antocuni Nov 17, 2022

JeffersGlass Nov 17, 2022

madhur-tandon Nov 18, 2022

JeffersGlass commented Nov 17, 2022

FabioRosado commented Nov 17, 2022

JeffersGlass commented Nov 17, 2022

JeffersGlass commented Nov 17, 2022 •

edited

Loading

FabioRosado commented Nov 17, 2022

JeffersGlass commented Nov 17, 2022

FabioRosado commented Nov 17, 2022

madhur-tandon commented Nov 18, 2022 •

edited

Loading

FabioRosado commented Nov 18, 2022

JeffersGlass commented Nov 18, 2022 •

edited

Loading

FabioRosado Nov 18, 2022

Import PyScript Version into Python Runtime #958

Import PyScript Version into Python Runtime #958

Conversation

JeffersGlass commented Nov 17, 2022 • edited Loading

antocuni Nov 17, 2022

Choose a reason for hiding this comment

JeffersGlass Nov 17, 2022

Choose a reason for hiding this comment

madhur-tandon Nov 18, 2022

Choose a reason for hiding this comment

JeffersGlass commented Nov 17, 2022

FabioRosado commented Nov 17, 2022

JeffersGlass commented Nov 17, 2022

JeffersGlass commented Nov 17, 2022 • edited Loading

FabioRosado commented Nov 17, 2022

JeffersGlass commented Nov 17, 2022

FabioRosado commented Nov 17, 2022

madhur-tandon commented Nov 18, 2022 • edited Loading

FabioRosado commented Nov 18, 2022

JeffersGlass commented Nov 18, 2022 • edited Loading

FabioRosado Nov 18, 2022

Choose a reason for hiding this comment

JeffersGlass commented Nov 17, 2022 •

edited

Loading

JeffersGlass commented Nov 17, 2022 •

edited

Loading

madhur-tandon commented Nov 18, 2022 •

edited

Loading

JeffersGlass commented Nov 18, 2022 •

edited

Loading