Port some code to Python3 #27999

ghostd · 2020-12-28T21:39:32Z

With this PR i wish to share modifications to run mach with ./mach instead of python3 ./mach.

I'm using a Debian machine. The following commands work for me:

./mach build --dev
./mach test-tidy
./mach fmt

I still have errors with the WPT scripts, but maybe the root causes is a bad resolution of requirements.txt

I used 2to3 to migrate some code before applying some manual fixes. For PLY, I copied/paste the code from the github repository.

If someone can test...
For now i'm not sure how to continue the migration while i can't run WPT :-/

./mach build -d does not report any errors
./mach test-tidy does not report any errors
These changes fix #___ (GitHub issue number if applicable)

There are tests for these changes OR
These changes do not require tests because ___

highfive · 2020-12-28T21:39:35Z

Heads up! This PR modifies the following files:

@asajeffrey: components/script/dom/bindings/codegen/CodegenRust.py, components/script/dom/bindings/codegen/ply/README, components/script/dom/bindings/codegen/ply/ply/lex.py, components/script/dom/bindings/codegen/run.py, components/script/build.rs and 4 more
@wafflespeanut: python/requirements.txt
@KiChjang: components/script/dom/bindings/codegen/CodegenRust.py, components/script/dom/bindings/codegen/ply/README, components/script/dom/bindings/codegen/ply/ply/lex.py, components/script/dom/bindings/codegen/run.py, components/script/build.rs and 4 more

highfive · 2020-12-28T21:39:37Z

Warning

These commits modify script code, but no tests are modified. Please consider adding a test!

ghostd · 2021-01-02T13:58:34Z

I updated and rebased the PR. My last issue needs a WPT fix (and is specific to Python 3.9)

I can run bootstrap, build, test-wpt, fmt and test-tidy.

jdm · 2021-01-02T17:26:53Z

Running the WPT lint on ./tests/wpt/web-platform-tests/...
{"path": "import-maps/data-driven/tools/format_json.py", "lineno": 15, "rule": "PARSE-FAILED", "message": "Unable to parse file"}

This is an interesting failure.

jdm · 2021-01-02T17:28:18Z

That presumably comes from https://github.com/web-platform-tests/wpt/blob/98049d97b75b4edbc71ee857b2b424bfec82524d/tools/lint/lint.py#L687-L696.

jdm · 2021-01-03T00:38:30Z

@bors-servo try

bors-servo · 2021-01-03T00:38:34Z

⌛ Trying commit 5a7da27 with merge c8b57e6...

Port some code to Python3 With this PR i wish to share modifications to run `mach` with `./mach` instead of `python3 ./mach`. I'm using a Debian machine. The following commands work for me: * `./mach build --dev` * `./mach test-tidy` * `./mach fmt` I still have errors with the WPT scripts, but maybe the root causes is a bad resolution of `requirements.txt` I used `2to3` to migrate some code before applying some manual fixes. For PLY, I copied/paste the code from the github repository. If someone can test... For now i'm not sure how to continue the migration while i can't run WPT :-/ ---  - [X] `./mach build -d` does not report any errors - [X] `./mach test-tidy` does not report any errors - [ ] These changes fix #___ (GitHub issue number if applicable)  - [ ] There are tests for these changes OR - [ ] These changes do not require tests because ___

jdm

The code changes look fine to me.

bors-servo · 2021-01-03T00:44:05Z

💔 Test failed - status-taskcluster

jdm · 2021-01-03T01:06:02Z

We should update https://github.com/servo/servo/blob/master/etc/taskcluster/decision_task.py to always use python3.

ghostd · 2021-01-04T20:36:47Z

Running the WPT lint on ./tests/wpt/web-platform-tests/...
{"path": "import-maps/data-driven/tools/format_json.py", "lineno": 15, "rule": "PARSE-FAILED", "message": "Unable to parse file"}

This is an interesting failure.

I proposed a PR to fix it

ghostd · 2021-01-04T21:33:23Z

We should update https://github.com/servo/servo/blob/master/etc/taskcluster/decision_task.py to always use python3.

I added python3 for each mach command and tried to add a with_python3 method for the Windows build (but i have no clue if it works). This commit really needs a careful review :-)

jdm · 2021-01-05T01:26:59Z

etc/taskcluster/decisionlib.py

+            python -m ensurepip
+            pip install virtualenv==20.2.1


I think these may need to be python3 and pip3 respectively.

jdm · 2021-01-05T01:27:16Z

etc/taskcluster/decisionlib.py

+        .with_early_script("""
+            python -m ensurepip
+            pip install virtualenv==20.2.1
+        """)


I believe this also requires .with_path_from_homedir("python3", "python3\\Scripts")

bors-servo · 2021-01-26T01:28:30Z

☔ The latest upstream changes (presumably #28094) made this pull request unmergeable. Please resolve the merge conflicts.

tomoverlund · 2021-01-28T01:27:32Z

How did you get ./mach build --dev without changes to mozjs?

jdm · 2021-01-28T01:43:13Z

Presumably it was still using python2 for the mozjs build.

tomoverlund · 2021-01-28T01:45:10Z

Presumably it was still using python2 for the mozjs build.

Oh, that makes sense. I'm trying to build in Debian unstable with no python2 in sight.

ghostd · 2021-01-29T21:38:56Z

I rebased and squashed the commit. The WPT PR has been merged \o/

jdm · 2021-02-02T15:04:49Z

I'm working on incorporating these changes into #28092.

Port CI to Python3 Based on the work from #27999 and #28092.

jdm · 2021-02-04T04:55:06Z

I'm going to attempt to wrestle CI into shape in #28141.

Port CI to Python3 Based on the work from #27999 and #28092.

bors-servo · 2021-02-06T20:37:44Z

☔ The latest upstream changes (presumably #28147) made this pull request unmergeable. Please resolve the merge conflicts.

jdm · 2021-02-06T21:10:40Z

I'm moving ahead with merging this code as part of #28092, because my attempts to get Windows CI working in #28141 didn't work out. I'm going to go ahead and close this PR—thank you so much for doing this work! It was a big help.

highfive added the S-awaiting-review There is new code that needs to be reviewed. label Dec 28, 2020

ghostd force-pushed the migrate-to-py3 branch from 72911fa to 5a7da27 Compare January 2, 2021 13:54

ghostd marked this pull request as ready for review January 2, 2021 13:58

jdm approved these changes Jan 3, 2021

View reviewed changes

highfive added the S-tests-failed The changes caused existing tests to fail. label Jan 3, 2021

highfive removed the S-tests-failed The changes caused existing tests to fail. label Jan 4, 2021

jdm reviewed Jan 5, 2021

View reviewed changes

jdm mentioned this pull request Jan 24, 2021

Update mozjs #28092

Merged

highfive added the S-needs-rebase There are merge conflict errors. label Jan 26, 2021

jdm mentioned this pull request Jan 26, 2021

Support Python 3 #23607

Closed

Port some code to Python3

b964306

ghostd force-pushed the migrate-to-py3 branch from 59fe92e to b964306 Compare January 29, 2021 21:38

jdm mentioned this pull request Jan 31, 2021

Mach bootstrap does not work on Fedora 33 #28120

Closed

andreubotella mentioned this pull request Jan 31, 2021

Mach says that virtualenv is not installed even though it is #28121

Closed

jdm mentioned this pull request Feb 4, 2021

Port CI to Python3 #28141

Closed

bors-servo added a commit that referenced this pull request Feb 4, 2021

Auto merge of #28141 - jdm:py3, r=<try>

b45a931

Port CI to Python3 Based on the work from #27999 and #28092.

bors-servo added a commit that referenced this pull request Feb 4, 2021

Auto merge of #28141 - jdm:py3, r=<try>

4dacb05

Port CI to Python3 Based on the work from #27999 and #28092.

bors-servo added a commit that referenced this pull request Feb 4, 2021

Auto merge of #28141 - jdm:py3, r=<try>

2e15f1a

Port CI to Python3 Based on the work from #27999 and #28092.

bors-servo added a commit that referenced this pull request Feb 4, 2021

Auto merge of #28141 - jdm:py3, r=<try>

3e4290e

Port CI to Python3 Based on the work from #27999 and #28092.

bors-servo added a commit that referenced this pull request Feb 4, 2021

Auto merge of #28141 - jdm:py3, r=<try>

71a9f67

Port CI to Python3 Based on the work from #27999 and #28092.

bors-servo added a commit that referenced this pull request Feb 4, 2021

Auto merge of #28141 - jdm:py3, r=<try>

86b542f

Port CI to Python3 Based on the work from #27999 and #28092.

jdm closed this Feb 6, 2021

ghostd deleted the migrate-to-py3 branch February 21, 2021 17:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Port some code to Python3 #27999

Port some code to Python3 #27999

ghostd commented Dec 28, 2020

highfive commented Dec 28, 2020

highfive commented Dec 28, 2020

ghostd commented Jan 2, 2021

jdm commented Jan 2, 2021

jdm commented Jan 2, 2021

jdm commented Jan 3, 2021

bors-servo commented Jan 3, 2021

jdm left a comment

bors-servo commented Jan 3, 2021

jdm commented Jan 3, 2021

ghostd commented Jan 4, 2021

ghostd commented Jan 4, 2021

jdm Jan 5, 2021

ghostd Jan 29, 2021

jdm Jan 5, 2021

ghostd Jan 29, 2021

bors-servo commented Jan 26, 2021

tomoverlund commented Jan 28, 2021

jdm commented Jan 28, 2021

tomoverlund commented Jan 28, 2021

ghostd commented Jan 29, 2021

jdm commented Feb 2, 2021

jdm commented Feb 4, 2021

bors-servo commented Feb 6, 2021

jdm commented Feb 6, 2021

Port some code to Python3 #27999

Port some code to Python3 #27999

Conversation

ghostd commented Dec 28, 2020

highfive commented Dec 28, 2020

highfive commented Dec 28, 2020

ghostd commented Jan 2, 2021

jdm commented Jan 2, 2021

jdm commented Jan 2, 2021

jdm commented Jan 3, 2021

bors-servo commented Jan 3, 2021

jdm left a comment

Choose a reason for hiding this comment

bors-servo commented Jan 3, 2021

jdm commented Jan 3, 2021

ghostd commented Jan 4, 2021

ghostd commented Jan 4, 2021

jdm Jan 5, 2021

Choose a reason for hiding this comment

ghostd Jan 29, 2021

Choose a reason for hiding this comment

jdm Jan 5, 2021

Choose a reason for hiding this comment

ghostd Jan 29, 2021

Choose a reason for hiding this comment

bors-servo commented Jan 26, 2021

tomoverlund commented Jan 28, 2021

jdm commented Jan 28, 2021

tomoverlund commented Jan 28, 2021

ghostd commented Jan 29, 2021

jdm commented Feb 2, 2021

jdm commented Feb 4, 2021

bors-servo commented Feb 6, 2021

jdm commented Feb 6, 2021