Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Python 3 #23607

Open
saschanaz opened this issue Jun 23, 2019 · 33 comments
Open

Support Python 3 #23607

saschanaz opened this issue Jun 23, 2019 · 33 comments

Comments

@saschanaz
Copy link
Contributor

@saschanaz saschanaz commented Jun 23, 2019

We'll probably want to migrate to Python 3 because Python 2 will die next year.

@highfive
Copy link

@highfive highfive commented Jun 28, 2019

@saschanaz saschanaz mentioned this issue Jul 6, 2019
4 of 4 tasks complete
@marmeladema
Copy link
Contributor

@marmeladema marmeladema commented Aug 9, 2019

What is left to do here? I am willing to help if needed

@jdm
Copy link
Member

@jdm jdm commented Aug 9, 2019

Ok, so pieces that I know about:

I think that's probably the full set of dependencies.

@marmeladema
Copy link
Contributor

@marmeladema marmeladema commented Aug 9, 2019

Thanks! I've updated most of the pieces i knew of to be python3 compatible but ultimately ended up fighting with mach itself:

Traceback (most recent call last):                                                                                                                                                                                                                             
  File "./mach", line 103, in <module>                                                                                                                                                                                                                         
    main(sys.argv)                                                                                                                                                                                                                                             
  File "./mach", line 33, in main                                                                                                                                                                                                                              
    mach = mach_bootstrap.bootstrap(topdir)                                                                                                                                                                                                                    
  File "/home/adema/code/servo/python/mach_bootstrap.py", line 288, in bootstrap                                                                                                                                                                               
    import mach.main                                                                                                                                                                                                                                           
  File "/home/adema/code/servo/python/_virtualenv/lib/python3.6/site-packages/mach/main.py", line 34, in <module>                                                                                                                                              
    from .config import ConfigSettings                                                                                                                                                                                                                         
  File "/home/adema/code/servo/python/_virtualenv/lib/python3.6/site-packages/mach/config.py", line 166                                                                                                                                                        
    raise AttributeError().__class__, exc, tb                                                                                                                                                                                                                  

On the mach bugzilla ticket, i see there is a lot a work going on. How can I be of any help?

@saschanaz
Copy link
Contributor Author

@saschanaz saschanaz commented Aug 9, 2019

I think we can be progressive and first do some syntactic changes, even if it doesn't work on Py3 yet.

@saschanaz
Copy link
Contributor Author

@saschanaz saschanaz commented Aug 9, 2019

we need to import changes to the python WebIDL parser to support python 3 (https://bugzilla.mozilla.org/show_bug.cgi?id=1563937)

Done by #23748

@jdm
Copy link
Member

@jdm jdm commented Aug 9, 2019

On the mach bugzilla ticket, i see there is a lot a work going on. How can I be of any help?

I recommend reaching out to @ahal in that bug directly.

@marmeladema
Copy link
Contributor

@marmeladema marmeladema commented Aug 9, 2019

@marmeladema
Copy link
Contributor

@marmeladema marmeladema commented Aug 13, 2019

I checked the upstream mach repository at https://hg.mozilla.org/mozilla-central/file/default/python/mach/mach/config.py
and in fact the bug i am facing has already been fixed but there are no new release since May 2, 2016: https://pypi.org/project/mach/#history
I guess we have to wait until everything is ready upstream and a new version published.

@mvashishth
Copy link

@mvashishth mvashishth commented Sep 28, 2019

Is anybody still working on this?

@marmeladema
Copy link
Contributor

@marmeladema marmeladema commented Sep 28, 2019

I am regularly looking for upstream mach updates. Things are landing but still no new release at this point so we are stuck for the time being.

@ahal
Copy link

@ahal ahal commented Sep 28, 2019

Mach supports Python 3 now, but I had no idea anyone was using the pypi package. I'll see if I can get a new version uploaded.. though we haven't been super careful about keeping Mozilla central specifics out, so it might not work out of the box anymore. (This is something I wish it supported)

It's been over three years since the last release.

@SimonSapin
Copy link
Member

@SimonSapin SimonSapin commented Sep 28, 2019

I think it might be viable to vendor a copy from mozilla-central into the servo/servo repository, instead of relying on PyPI.

@ahal
Copy link

@ahal ahal commented Sep 28, 2019

Yeah that would work too.

Either way I'm happy to version and release as needed. Just let me know next time you need an update :). Tbh I'm happy to learn of out of tree consumers.

@mvashishth
Copy link

@mvashishth mvashishth commented Sep 29, 2019

By default, it runs on Python2.7. I'd love to work on it if this issue is still open.

@ahal
Copy link

@ahal ahal commented Sep 30, 2019

Bug to bump + release mach:
https://bugzilla.mozilla.org/show_bug.cgi?id=1585146

@marmeladema
Copy link
Contributor

@marmeladema marmeladema commented Oct 1, 2019

@mvashishth I already started, I am just waiting for a mach release to go forward. This should happens soon thanks to @ahal :-)

@marmeladema
Copy link
Contributor

@marmeladema marmeladema commented Oct 4, 2019

@ahal do you plan to release version 1.0.0 on pypi soon? In the mean time, i'll try to vendor it in servo directly as proposed by @SimonSapin

@ahal
Copy link

@ahal ahal commented Oct 5, 2019

Done.

@marmeladema
Copy link
Contributor

@marmeladema marmeladema commented Oct 6, 2019

Thank you very much! I made some progress in the past few days but i wonder if I should keep compatibility with Python2.7 or if i can migrate everything to Python 3?

Right now most things work, but to properly finish migration it will require some work in web-platform-tests because even though they are some ongoing efforts to support Python 3, its not done yet

@filips123
Copy link

@filips123 filips123 commented Oct 6, 2019

if I should keep compatibility with Python2.7 or if i can migrate everything to Python 3?

I'm not from official Servo team, but I think support for Python 2 is not needed anymore. Python 2.7 will not be suported in 2020 so it would be insecure to use it.

@SimonSapin
Copy link
Member

@SimonSapin SimonSapin commented Oct 6, 2019

It would be unfortunate IMO to require both Python 2.7 and 3.x in order to build Servo (or some other task like running tests). Therefore, it would be nice if we could maintain compatibility with 2.7 until we are confident that everything supports 3.x.

If such compat proves difficult or costly to implement or maintain, we can reconsider.

@jdm
Copy link
Member

@jdm jdm commented Oct 6, 2019

We already require 2.7 and 3.x to build servo - mozjs won't build without pyhthon 3 available in your path.

@SimonSapin
Copy link
Member

@SimonSapin SimonSapin commented Oct 7, 2019

oh no

bors-servo added a commit that referenced this issue Dec 10, 2019
Enable some mach commands to be run with python3

This change finally enable the following commands to be run with python3:
* `build`
* `test-unit`
* `package`

As previously explained, `test-tidy` will require more work in the wpt repository directly. Maybe `test-tidy --no-wpt` is achievable relatively quickly though.

For possible remaining bits that might need to be worked on, see #23607

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors

<!-- Either: -->
- [x] There are tests for these changes

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
bors-servo added a commit that referenced this issue Dec 10, 2019
Enable some mach commands to be run with python3

This change finally enable the following commands to be run with python3:
* `build`
* `test-unit`
* `package`

As previously explained, `test-tidy` will require more work in the wpt repository directly. Maybe `test-tidy --no-wpt` is achievable relatively quickly though.

For possible remaining bits that might need to be worked on, see #23607

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors

<!-- Either: -->
- [x] There are tests for these changes

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
bors-servo added a commit that referenced this issue Dec 10, 2019
Enable some mach commands to be run with python3

This change finally enable the following commands to be run with python3:
* `build`
* `test-unit`
* `package`

As previously explained, `test-tidy` will require more work in the wpt repository directly. Maybe `test-tidy --no-wpt` is achievable relatively quickly though.

For possible remaining bits that might need to be worked on, see #23607

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors

<!-- Either: -->
- [x] There are tests for these changes

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
bors-servo added a commit that referenced this issue Dec 11, 2019
Make `mach test-tidy --no-wpt` compatible with Python3

Make `mach test-tidy --no-wpt` compatible with Python3
See also #23607

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors

<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
bors-servo added a commit that referenced this issue Dec 11, 2019
Make `mach test-tidy --no-wpt` compatible with Python3

Make `mach test-tidy --no-wpt` compatible with Python3
See also #23607

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors

<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
bors-servo added a commit that referenced this issue Dec 11, 2019
Make `mach test-tidy --no-wpt` compatible with Python3

Make `mach test-tidy --no-wpt` compatible with Python3
See also #23607

After this pull request, all python files (except WPT) will be checked for Python3 syntax compatibility.

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors

<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
bors-servo added a commit that referenced this issue Dec 11, 2019
Make `mach test-tidy --no-wpt` compatible with Python3

Make `mach test-tidy --no-wpt` compatible with Python3
See also #23607

After this pull request, all python files (except WPT) will be checked for Python3 syntax compatibility.

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors

<!-- Either: -->
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
bors-servo added a commit that referenced this issue Dec 14, 2019
…=<try>

Make `mach test-tidy --self-test` compatible with Python3

It now runs in CI to avoid future regressions.
See #23607

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors

<!-- Either: -->
- [x] There are tests for these changes

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
bors-servo added a commit that referenced this issue Dec 14, 2019
…=jdm

Make `mach test-tidy --self-test` compatible with Python3

It now runs in CI to avoid future regressions.
See #23607

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors

<!-- Either: -->
- [x] There are tests for these changes

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
bors-servo added a commit that referenced this issue Dec 14, 2019
…=jdm

Make `mach test-tidy --self-test` compatible with Python3

It now runs in CI to avoid future regressions.
See #23607

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors

<!-- Either: -->
- [x] There are tests for these changes

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
bors-servo added a commit that referenced this issue Dec 14, 2019
…=jdm

Make `mach test-tidy --self-test` compatible with Python3

It now runs in CI to avoid future regressions.
See #23607

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors

<!-- Either: -->
- [x] There are tests for these changes

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
bors-servo added a commit that referenced this issue Dec 14, 2019
…=jdm

Make `mach test-tidy --self-test` compatible with Python3

It now runs in CI to avoid future regressions.
See #23607

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors

<!-- Either: -->
- [x] There are tests for these changes

<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
@marmeladema
Copy link
Contributor

@marmeladema marmeladema commented Dec 21, 2019

Current support for Python3 is partially complete in the sense that most of servo itself is compatible with Python3 but WPT itself is not. The following commands are tested in CI under linux;

  • python3 ./mach test-tidy --no-progress --all --no-wpt
  • python3 ./mach build --dev
  • python3 ./mach test-unit
  • python3 ./mach package --dev
  • python3 ./mach build --dev --features canvas2d-raqote
  • python3 ./mach build --dev --features layout-2020
  • python3 ./mach build --dev --libsimpleservo
  • python3 ./mach build --dev -p servo-gst-plugin
  • python3 ./mach test-tidy --no-progress --self-test

But for example, this does not work yet:

  • python3 ./mach test-tidy --no-progress --all

Maybe the readme should be updated for guidance on how to build using python3?

bors-servo added a commit that referenced this issue Mar 25, 2020
Make tests/jquery/run_jquery.py Python3 compatible

Attempt to make it py3 compatible.

Ran the "python-migrate" tool, and did a couple of manual adjustments, which are directly noted with comments below.

re: #23607

```
(.env) [dalley@localhost servo]$ python3 ./mach test-jquery
Already up to date.
Testing jQuery on Servo!
	selector
		OK: element - jQuery only
		OK: id
		OK: class - jQuery only
		OK: name
		OK: selectors with comma
		OK: child and adjacent
		OK: attributes
		OK: disconnected nodes
		OK: disconnected nodes - jQuery only
		OK: attributes - jQuery.attr
		OK: jQuery.contains
		OK: jQuery.uniqueSort
		OK: Sizzle cache collides with multiple Sizzles on a page
		FAIL: Iframe dispatch should not affect jQuery (#13936): WAS ok=0 fail=1 total=1 NOW ok=1 fail=0 total=1
	Ran 14 test groups. 1 unexpected results.
	163 tests succeeded of 167 (97.60%)

```

Not sure if the test failure on the last one is an issue.
@renich
Copy link

@renich renich commented May 7, 2020

Something's wrong with pip in the virtual environment. I've tried using mach but it complains about no virtualenv being installed; which is false (Fedora 32 here).

In any case, after activating the virtualenv; I tried to install the requirements. It fails:

Traceback (most recent call last):
  File "/home/renich/src/servo/python/_virtualenv3.8/bin/pip", line 8, in <module>
    sys.exit(main())
  File "/home/renich/src/servo/python/_virtualenv3.8/lib/python3.8/site-packages/pip/_internal/cli/main.py", line 73, in main
    command = create_command(cmd_name, isolated=("--isolated" in cmd_args))
  File "/home/renich/src/servo/python/_virtualenv3.8/lib/python3.8/site-packages/pip/_internal/commands/__init__.py", line 104, in create_command
    module = importlib.import_module(module_path)
  File "/home/renich/src/servo/python/_virtualenv3.8/lib64/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/renich/src/servo/python/_virtualenv3.8/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 21, in <module>
    from pip._internal.cache import WheelCache
  File "/home/renich/src/servo/python/_virtualenv3.8/lib/python3.8/site-packages/pip/_internal/cache.py", line 16, in <module>
    from pip._internal.models.link import Link
ModuleNotFoundError: No module named 'pip._internal.models.link'


@jdm
Copy link
Member

@jdm jdm commented May 7, 2020

You need to run python -m pip install virtualenv. I don't know what to make of that error message, however.

@renich
Copy link

@renich renich commented May 7, 2020

This is the next error. I've deleted the incomplete virtualenv and did

You need to run python -m pip install virtualenv. I don't know what to make of that error message, however.

[renich@introdesk servo]$ python -m pip install virtualenv
Requirement already satisfied: virtualenv in /usr/lib/python3.8/site-packages (16.7.10)
@renich
Copy link

@renich renich commented May 7, 2020

OK, now, its eems it wants the devel package for python:

[renich@introdesk servo]$ python3 ./mach build --release
"/home/renich/src/servo/python/_virtualenv3.8/bin/python" "-m" "pip" "install" "-I" "-r" "/home/renich/src/servo/tests/wpt/web-platform-tests/tools/wptrunner/requirements_firefox.txt" failed with error code 1:
Output:
Collecting marionette_driver==3.0.0
  Using cached marionette_driver-3.0.0-py3-none-any.whl (47 kB)
Collecting mozcrash==2.0.0
  Using cached mozcrash-2.0.0-py2.py3-none-any.whl (10 kB)
Processing /home/renich/.cache/pip/wheels/40/c3/14/f9cd06f54132b85b6f60ac4a0899e1b9456c754ad6b85f9853/mozdownload-1.26.0-py3-none-any.whl
Processing /home/renich/.cache/pip/wheels/89/56/08/9668ed765dbfb8f2aba4d9d6a18f67d1054a527e67de73ac0c/mozInstall-2.0.0-py2.py3-none-any.whl
Collecting mozleak==0.2
  Using cached mozleak-0.2-py2.py3-none-any.whl (6.9 kB)
Processing /home/renich/.cache/pip/wheels/89/6a/d8/b8519773d14645ae1fde253c968ee09b430c0ce8d467d3ed75/moznetwork-1.1.0-py3-none-any.whl
Collecting mozprocess==1.0.0
  Using cached mozprocess-1.0.0-py2.py3-none-any.whl (21 kB)
Collecting mozprofile==2.5.0
  Using cached mozprofile-2.5.0-py2.py3-none-any.whl (23 kB)
Collecting mozrunner==7.8.0
  Using cached mozrunner-7.8.0-py2.py3-none-any.whl (35 kB)
Collecting mozversion==2.3.0
  Using cached mozversion-2.3.0-py2.py3-none-any.whl (5.2 kB)
Collecting psutil==5.7.0
  Using cached psutil-5.7.0.tar.gz (449 kB)
Collecting six
  Using cached six-1.14.0-py2.py3-none-any.whl (10 kB)
Processing /home/renich/.cache/pip/wheels/46/fb/8f/e1971224236ced53c9e370abe1c35f76fa408fcddfdd00622e/mozfile-2.1.0-py2.py3-none-any.whl
Collecting mozlog>=6.0
  Using cached mozlog-6.0-py2.py3-none-any.whl (68 kB)
Collecting redo==2.0.3
  Using cached redo-2.0.3-py2.py3-none-any.whl (5.2 kB)
Collecting packaging>=19.0
  Using cached packaging-20.3-py2.py3-none-any.whl (37 kB)
Collecting treeherder-client<6.0.0,>=5.0.0
  Using cached treeherder_client-5.0.0-py2.py3-none-any.whl (5.2 kB)
Collecting requests<3.0.0,>=2.21.0
  Using cached requests-2.23.0-py2.py3-none-any.whl (58 kB)
Collecting progressbar2>=3.34.3
  Using cached progressbar2-3.51.3-py2.py3-none-any.whl (51 kB)
Collecting mozinfo>=1.0.0
  Using cached mozinfo-1.2.1-py2.py3-none-any.whl (7.1 kB)
Collecting mozdevice>=3.0.1
  Using cached mozdevice-3.2.2-py2.py3-none-any.whl (32 kB)
Collecting mozterm
  Using cached mozterm-1.0.0-py2.py3-none-any.whl (3.4 kB)
Collecting blessings>=1.3
  Using cached blessings-1.7-py3-none-any.whl (18 kB)
Collecting pyparsing>=2.0.2
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting idna<3,>=2.5
  Using cached idna-2.9-py2.py3-none-any.whl (58 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2020.4.5.1-py2.py3-none-any.whl (157 kB)
Collecting chardet<4,>=3.0.2
  Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
  Using cached urllib3-1.25.9-py2.py3-none-any.whl (126 kB)
Collecting python-utils>=2.3.0
  Using cached python_utils-2.4.0-py2.py3-none-any.whl (12 kB)
Collecting distro==1.4.0
  Using cached distro-1.4.0-py2.py3-none-any.whl (17 kB)
Building wheels for collected packages: psutil
  Building wheel for psutil (setup.py): started
  Building wheel for psutil (setup.py): finished with status 'error'
  Running setup.py clean for psutil
Failed to build psutil
Installing collected packages: six, mozterm, blessings, mozlog, mozfile, distro, mozinfo, mozprocess, mozprofile, mozdevice, mozrunner, mozversion, marionette-driver, mozcrash, redo, pyparsing, packaging, idna, certifi, chardet, urllib3, requests, treeherder-client, python-utils, progressbar2, mozdownload, mozinstall, mozleak, moznetwork, psutil
    Running setup.py install for psutil: started
    Running setup.py install for psutil: finished with status 'error'
Error:
  ERROR: Command errored out with exit status 1:
   command: /home/renich/src/servo/python/_virtualenv3.8/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-efhm2776/psutil/setup.py'"'"'; __file__='"'"'/tmp/pip-install-efhm2776/psutil/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-xgxued1p
       cwd: /tmp/pip-install-efhm2776/psutil/
  Complete output (44 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.8
  creating build/lib.linux-x86_64-3.8/psutil
  copying psutil/_psosx.py -> build/lib.linux-x86_64-3.8/psutil
  copying psutil/_pssunos.py -> build/lib.linux-x86_64-3.8/psutil
  copying psutil/_pslinux.py -> build/lib.linux-x86_64-3.8/psutil
  copying psutil/_psaix.py -> build/lib.linux-x86_64-3.8/psutil
  copying psutil/__init__.py -> build/lib.linux-x86_64-3.8/psutil
  copying psutil/_psposix.py -> build/lib.linux-x86_64-3.8/psutil
  copying psutil/_common.py -> build/lib.linux-x86_64-3.8/psutil
  copying psutil/_compat.py -> build/lib.linux-x86_64-3.8/psutil
  copying psutil/_psbsd.py -> build/lib.linux-x86_64-3.8/psutil
  copying psutil/_pswindows.py -> build/lib.linux-x86_64-3.8/psutil
  creating build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/runner.py -> build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/test_contracts.py -> build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/__main__.py -> build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/test_linux.py -> build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/test_process.py -> build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/test_osx.py -> build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/test_memory_leaks.py -> build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/__init__.py -> build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/test_sunos.py -> build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/test_misc.py -> build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/test_windows.py -> build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/test_posix.py -> build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/test_unicode.py -> build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/test_connections.py -> build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/test_bsd.py -> build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/test_system.py -> build/lib.linux-x86_64-3.8/psutil/tests
  copying psutil/tests/test_aix.py -> build/lib.linux-x86_64-3.8/psutil/tests
  running build_ext
  building 'psutil._psutil_linux' extension
  creating build/temp.linux-x86_64-3.8
  creating build/temp.linux-x86_64-3.8/psutil
  gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=570 -DPSUTIL_LINUX=1 -I/usr/include/python3.8 -c psutil/_psutil_common.c -o build/temp.linux-x86_64-3.8/psutil/_psutil_common.o
  psutil/_psutil_common.c:9:10: fatal error: Python.h: No such file or directory
      9 | #include <Python.h>
        |          ^~~~~~~~~~
  compilation terminated.
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for psutil
    ERROR: Command errored out with exit status 1:
     command: /home/renich/src/servo/python/_virtualenv3.8/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-efhm2776/psutil/setup.py'"'"'; __file__='"'"'/tmp/pip-install-efhm2776/psutil/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-ohz03zi6/install-record.txt --single-version-externally-managed --compile --install-headers /home/renich/src/servo/python/_virtualenv3.8/include/site/python3.8/psutil
         cwd: /tmp/pip-install-efhm2776/psutil/
    Complete output (44 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.8
    creating build/lib.linux-x86_64-3.8/psutil
    copying psutil/_psosx.py -> build/lib.linux-x86_64-3.8/psutil
    copying psutil/_pssunos.py -> build/lib.linux-x86_64-3.8/psutil
    copying psutil/_pslinux.py -> build/lib.linux-x86_64-3.8/psutil
    copying psutil/_psaix.py -> build/lib.linux-x86_64-3.8/psutil
    copying psutil/__init__.py -> build/lib.linux-x86_64-3.8/psutil
    copying psutil/_psposix.py -> build/lib.linux-x86_64-3.8/psutil
    copying psutil/_common.py -> build/lib.linux-x86_64-3.8/psutil
    copying psutil/_compat.py -> build/lib.linux-x86_64-3.8/psutil
    copying psutil/_psbsd.py -> build/lib.linux-x86_64-3.8/psutil
    copying psutil/_pswindows.py -> build/lib.linux-x86_64-3.8/psutil
    creating build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/runner.py -> build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/test_contracts.py -> build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/__main__.py -> build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/test_linux.py -> build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/test_process.py -> build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/test_osx.py -> build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/test_memory_leaks.py -> build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/__init__.py -> build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/test_sunos.py -> build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/test_misc.py -> build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/test_windows.py -> build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/test_posix.py -> build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/test_unicode.py -> build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/test_connections.py -> build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/test_bsd.py -> build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/test_system.py -> build/lib.linux-x86_64-3.8/psutil/tests
    copying psutil/tests/test_aix.py -> build/lib.linux-x86_64-3.8/psutil/tests
    running build_ext
    building 'psutil._psutil_linux' extension
    creating build/temp.linux-x86_64-3.8
    creating build/temp.linux-x86_64-3.8/psutil
    gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=570 -DPSUTIL_LINUX=1 -I/usr/include/python3.8 -c psutil/_psutil_common.c -o build/temp.linux-x86_64-3.8/psutil/_psutil_common.o
    psutil/_psutil_common.c:9:10: fatal error: Python.h: No such file or directory
        9 | #include <Python.h>
          |          ^~~~~~~~~~
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/renich/src/servo/python/_virtualenv3.8/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-efhm2776/psutil/setup.py'"'"'; __file__='"'"'/tmp/pip-install-efhm2776/psutil/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-ohz03zi6/install-record.txt --single-version-externally-managed --compile --install-headers /home/renich/src/servo/python/_virtualenv3.8/include/site/python3.8/psutil Check the logs for full command output.
@renich
Copy link

@renich renich commented May 7, 2020

OK, after installing python3-devel in Fedora 32, it cofails again:

Exception: Your system's gstreamer libraries are out of date (we need at least 1.16). Please run ./mach bootstrap-gstreamer

  File "/home/renich/src/servo/python/servo/build_commands.py", line 241, in build
    env = self.build_env(target=target, is_build=True, uwp=uwp, features=features)
  File "/home/renich/src/servo/python/servo/command_base.py", line 648, in build_env
    if is_build and self.needs_gstreamer_env(target or host_triple(), env, uwp, features):
  File "/home/renich/src/servo/python/servo/command_base.py", line 552, in needs_gstreamer_env
    raise Exception("Your system's gstreamer libraries are out of date \

So, it seems I need a bunch of devel libraries. Will stop posting until I get it right. Sorry for the SPAM.

@joebarbere
Copy link

@joebarbere joebarbere commented May 27, 2020

Is there a specific branch/repo out there for python3 support/work? I'm getting the following error when building the master branch.

Compiling ws v0.9.1
error: failed to run custom build command for script v0.0.1 (/home/joebarbere/code/servo/components/script)

Caused by:
process didn't exit successfully: /home/joebarbere/code/servo/target/debug/build/script-a15c9cbb53f68e5d/build-script-build (exit code: 1)
--- stderr
./parser/WebIDL.py:5481: SyntaxWarning: str indices must be integers or slices, not tuple; perhaps you missed a comma?
raise WebIDLError("[CrossOriginCallable] is only allowed on non-static "
Traceback (most recent call last):
File "dom/bindings/codegen/run.py", line 119, in
main()

@jdm
Copy link
Member

@jdm jdm commented May 27, 2020

There is not. We don't yet support python3 for the script crate code generation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.