Add a Pyodide runner based on the python-execution-prototypes #893

tuzz · 2024-01-18T12:57:44Z

This pull request adds experimental support for running Python programs using Pyodide. The code is based on the python-execution-prototypes and the technical analysis document. The related GitHub issue is #891.

The PyodideRunner.jsx can be used instead of the current PythonRunner.jsx (Skulpt) by provided a ?pyodide=true parameter in the URL as shown in the screenshot below:

Notes

The PyodideRunner supports the following features:

Execution of Python programs
Importing of most of Python's standard library
Importing of many common Python packages
Importing of pure Python packages from PyPi
Stopping execution
Sending standard input to Python programs
Closing the standard input stream by pressing Ctrl-d
Visualising turtle graphics using the modified turtle package
Visualising pygal charts using the modified pygal package

Due to time constraints, the following tasks remain outstanding:

Visualising sense_hat graphics using the modified sense_hat package
- There is a work-in-progress pull request here
Apply HTTP headers in a service worker (see Investigate setting Pyodide HTTP headers via a service worker #903)
- Note that crossorigin will need to be added to cross-domain resources once the HTTP headers are added
Add unit/integration tests that run real Python programs through the PyodideRunner.jsx
Manual testing for backwards compatibility of existing Python programs, projects and the WebComponent
Implementing the p5 package to work with Pyodide
- See Support "python mode" for the p5 package and support the py5 package #949
Implementing a dedicated SQL editor for use in the Ada CS project
- Note that Pyodide supports importing sqlite3 so we can add a UI wrapper around it

github-actions · 2024-01-18T13:45:55Z

Github Run 7570888691
Test coverage: 85%

We need to be able to conditionally set headers for pyodideWorker.js in pull request #893 but this feature isn’t supported in webpack-dev-server 3.x.x. We therefore need to upgrade to version 4.0.0 which allows a function to be provided in the headers option. I’ve tried to follow the migration guide here: https://github.com/webpack/webpack-dev-server/blob/master/migration-v4.md

… 3.11.1 to 4.0.0 (#895) We need to be able to conditionally set headers for pyodideWorker.js in pull request #893 but this feature isn’t supported in webpack-dev-server 3.x.x. We therefore need to upgrade to version 4.0.0 which allows a function to be provided in the headers option. When attempting this upgrade, I ran into [this issue](storybookjs/storybook#22431) and only managed to resolve it by also updating yarn to version 3. I’ve tried to follow the migration guide here: https://github.com/webpack/webpack-dev-server/blob/master/migration-v4.md

github-actions · 2024-01-19T12:36:27Z

Github Run 7583828967
Test coverage: 85%

github-actions · 2024-01-19T13:09:57Z

Github Run 7584198244
Test coverage: 85%

Jest doesn’t support Web Workers and was erroring because the importScripts function isn’t available. Best practice seems to be to mock the module and returned canned responses. Currently, we don’t have Jest tests for the experimental PyodideRunner.js so an empty implementation is sufficient but we could change this later. https://stackoverflow.com/questions/42567535#answer-62436219

github-actions · 2024-01-30T11:41:55Z

Github Run 7710553658
Test coverage: 78%

github-actions · 2024-01-30T12:02:20Z

Github Run 7710751536
Test coverage: 78%

sra405 · 2024-02-12T13:35:20Z

src/components/Editor/Runners/PyodideRunner/PyodideRunner.jsx

@@ -0,0 +1,364 @@
+/* eslint import/no-webpack-loader-syntax: off */
+/* eslint-disable react-hooks/exhaustive-deps */


⛏️ I'm not a fan of this rule for sure, especially with empty dependency arrays but 2/3 useEffects that are effected can have handleRun and handleStop in the array without any impact

raising just so I can make a TODO ticket 👍

This was copied over from PythonRunner.jsx when I started working on PyodideRunner.jsx. I tried reinstating the eslint but it caused quite a lot of errors that will take some time to fix. Maybe we can address this later in a separate ticket as you suggest. I've addressed the other pull request comments now.

src/components/Editor/Runners/PyodideRunner/PyodideRunner.jsx

src/components/Editor/Runners/PyodideRunner/PyodideWorker.js

sra405

👍 only a couple of minor things and want to get this in to build on. Great work 🥳

github-actions · 2024-02-12T14:44:07Z

Github Run 7873408006
Test coverage: 78%

These ternaries were copied over from PythonRunner.

github-actions · 2024-02-12T18:48:28Z

Github Run 7876456277
Test coverage: 78%

sra405 · 2024-02-13T14:03:54Z

Due to time constraints, the following tasks remain outstanding:

Visualising sense_hat graphics using the modified sense_hat package

There is a work-in-progress pull request here

Apply HTTP headers in a service worker (see Investigate setting Pyodide HTTP headers via a service worker #903)

Note that crossorigin will need to be added to cross-domain resources once the HTTP headers are added

Add unit/integration tests that run real Python programs through the PyodideRunner.jsx

Manual testing for backwards compatibility of existing Python programs, projects and the WebComponent

Implementing the p5 package to work with Pyodide

See Support "python mode" for the p5 package and support the py5 package python-execution-prototypes#1

Implementing a dedicated SQL editor for use in the Ada CS project

Note that Pyodide supports importing sqlite3 so we can add a UI wrapper around it

I'll make a general ticket to list the remaining work, feel free to add any additional bits or comments 👍

sra405

👍 this is ace! Let's get it in and build on top 😄

tuzz force-pushed the pyodide-runner branch from 11cd01c to 72047ea Compare January 18, 2024 12:58

tuzz had a problem deploying to previews/pyodide-runner January 18, 2024 12:58 — with GitHub Actions Error

tuzz force-pushed the pyodide-runner branch from 72047ea to 85a7f7c Compare January 18, 2024 13:43

tuzz had a problem deploying to previews/pyodide-runner January 18, 2024 13:43 — with GitHub Actions Failure

tuzz mentioned this pull request Jan 18, 2024

Update yarn from version 1.22.21 to 3.4.1 and webpack-dev-server from 3.11.1 to 4.0.0 #895

Merged

tuzz force-pushed the pyodide-runner branch from 85a7f7c to 2ee3530 Compare January 19, 2024 12:32

tuzz had a problem deploying to previews/pyodide-runner January 19, 2024 12:32 — with GitHub Actions Error

tuzz force-pushed the pyodide-runner branch from 2ee3530 to 53b5a04 Compare January 19, 2024 12:34

tuzz temporarily deployed to previews/pyodide-runner January 19, 2024 12:34 — with GitHub Actions Inactive

tuzz temporarily deployed to previews/pyodide-runner January 19, 2024 13:07 — with GitHub Actions Inactive

tuzz temporarily deployed to previews/pyodide-runner January 19, 2024 16:04 — with GitHub Actions Inactive

tuzz temporarily deployed to previews/pyodide-runner January 19, 2024 16:11 — with GitHub Actions Inactive

tuzz had a problem deploying to previews/pyodide-runner January 22, 2024 14:55 — with GitHub Actions Failure

tuzz temporarily deployed to previews/pyodide-runner January 30, 2024 10:42 — with GitHub Actions Inactive

tuzz added 2 commits January 30, 2024 11:31

Ensure the editor.codeRunLoading state is set un RunButton.test.js

a1bae39

tuzz temporarily deployed to previews/pyodide-runner January 30, 2024 11:39 — with GitHub Actions Inactive

tuzz marked this pull request as ready for review January 30, 2024 11:55

Fix the interface for pyodide.FS.writeFile

35810c1

tuzz temporarily deployed to previews/pyodide-runner January 30, 2024 11:59 — with GitHub Actions Inactive

sra405 linked an issue Feb 1, 2024 that may be closed by this pull request

Integrate the Pyodide prototype with the editor-ui project #891

Closed

sra405 mentioned this pull request Feb 1, 2024

Integrate the Pyodide prototype with the editor-ui project #892

Closed

sra405 reviewed Feb 12, 2024

View reviewed changes

sra405 mentioned this pull request Feb 12, 2024

Remove pyodide runner disabling react-hook/exhaustive-deps #920

Open

sra405 reviewed Feb 12, 2024

View reviewed changes

src/components/Editor/Runners/PyodideRunner/PyodideRunner.jsx Outdated Show resolved Hide resolved

sra405 reviewed Feb 12, 2024

View reviewed changes

src/components/Editor/Runners/PyodideRunner/PyodideRunner.jsx Outdated Show resolved Hide resolved

sra405 reviewed Feb 12, 2024

View reviewed changes

src/components/Editor/Runners/PyodideRunner/PyodideWorker.js Show resolved Hide resolved

sra405 previously approved these changes Feb 12, 2024

View reviewed changes

Merge branch 'main' into pyodide-runner

19dc454

sra405 temporarily deployed to previews/pyodide-runner February 12, 2024 14:42 — with GitHub Actions Inactive

tuzz added 2 commits February 12, 2024 18:32

Replace ternaries with && expressions

a47452f

These ternaries were copied over from PythonRunner.

Simplify getInputElement in PyodideRunner

2d6931a

tuzz dismissed sra405’s stale review via 2d6931a February 12, 2024 18:46

tuzz deployed to previews/pyodide-runner February 12, 2024 18:46 — with GitHub Actions View deployment

sra405 mentioned this pull request Feb 13, 2024

Pyodide migration plan #921

Open

sra405 approved these changes Feb 13, 2024

View reviewed changes

tuzz merged commit f791ee3 into main Feb 13, 2024
8 checks passed

tuzz deleted the pyodide-runner branch February 13, 2024 14:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a Pyodide runner based on the python-execution-prototypes #893

Add a Pyodide runner based on the python-execution-prototypes #893

tuzz commented Jan 18, 2024 •

edited

github-actions bot commented Jan 18, 2024

github-actions bot commented Jan 19, 2024

github-actions bot commented Jan 19, 2024

github-actions bot commented Jan 30, 2024

github-actions bot commented Jan 30, 2024

sra405 Feb 12, 2024

sra405 Feb 12, 2024

tuzz Feb 12, 2024

sra405 left a comment

github-actions bot commented Feb 12, 2024

github-actions bot commented Feb 12, 2024

sra405 commented Feb 13, 2024

sra405 left a comment

		@@ -0,0 +1,364 @@
		/* eslint import/no-webpack-loader-syntax: off */
		/* eslint-disable react-hooks/exhaustive-deps */

Add a Pyodide runner based on the python-execution-prototypes #893

Add a Pyodide runner based on the python-execution-prototypes #893

Conversation

tuzz commented Jan 18, 2024 • edited

Notes

github-actions bot commented Jan 18, 2024

github-actions bot commented Jan 19, 2024

github-actions bot commented Jan 19, 2024

github-actions bot commented Jan 30, 2024

github-actions bot commented Jan 30, 2024

sra405 Feb 12, 2024

Choose a reason for hiding this comment

sra405 Feb 12, 2024

Choose a reason for hiding this comment

tuzz Feb 12, 2024

Choose a reason for hiding this comment

sra405 left a comment

Choose a reason for hiding this comment

github-actions bot commented Feb 12, 2024

github-actions bot commented Feb 12, 2024

sra405 commented Feb 13, 2024

sra405 left a comment

Choose a reason for hiding this comment

tuzz commented Jan 18, 2024 •

edited