New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make PlatformIO Python 3 compatible #895

Closed
StefanD986 opened this Issue Feb 3, 2017 · 12 comments

Comments

Projects
None yet
9 participants
@StefanD986
Copy link

StefanD986 commented Feb 3, 2017

Configuration

Operating system:
Windows and Mac

PlatformIO Version (platformio --version):
PlatformIO, version 3.2.1

Description of problem

PlatformIO is not compatible to Python 3 (as stated on the website). It would be nice and would ease the install process if PlatformIO would also work with Python 3.

@ivankravets ivankravets added the feature label Feb 3, 2017

@ivankravets

This comment has been minimized.

Copy link
Member

ivankravets commented Feb 3, 2017

PlatformIO's Build System depends on SCons tool that doesn't support Python 3. Please note that SCons team works in this way.

@skorokithakis

This comment has been minimized.

Copy link

skorokithakis commented Aug 5, 2017

@ivankravets How hard would it be to remove this dependency, or replace it with something else?

Would it be feasible to write an adapter module that called out to SCons from Python 3? Although that would be a hassle when installing...

Python 2 is quickly becoming irrelevant, and I'm finding myself having to keep a Python 2 env around just for PlatformIO. I think this should be a higher priority, as PlatformIO risks finding itself in a position where it can't be easily installed soon (with OSes moving to Py3).

I would be willing to contribute some code to the migration to Py3.

@ivankravets

This comment has been minimized.

Copy link
Member

ivankravets commented Aug 5, 2017

PlatformIO risks finding itself in a position where it can't be easily installed soon (with OSes moving to Py3).

Python 2 is a stable and supported branch. I think, minimum 5 years all major OSs will include Python 2 distribution by default (as they now do). Hundred OS-dependent libraries and software are written in Python 2: macOS, Linux, BSD.

Nevertheless, SCons Team hardly working on the first Python3 compatible release.

@ivankravets ivankravets added this to the 3.5.0 milestone Sep 18, 2017

@tehnorm

This comment has been minimized.

Copy link

tehnorm commented Sep 28, 2017

Just noticed that it looks like SCons 3.0.0 just released support for python 3.5+!

http://scons.org/scons-300-is-available.html

@ivankravets ivankravets removed this from the 3.5.0 milestone Oct 31, 2017

ivankravets added a commit that referenced this issue Jan 26, 2018

@ivankravets

This comment has been minimized.

Copy link
Member

ivankravets commented Jan 26, 2018

Upgraded to SCons 3.0 in PlatformIo Core 3.5.2 ( 59337c7 ). Now, need to upgrade PIO Core to Python 3.

ivankravets added a commit that referenced this issue Jan 30, 2018

cfobel added a commit to sci-bots/platformio-core that referenced this issue Feb 14, 2018

Merge branch 'feat(py3)' into develop
Add Python 3 support (backwards compatible with Python 2.7).

See platformio#895.

Note: platforms/packages containing Python code, e.g.,
[`platform-atmelavr`][1], still need to be updated to support Python 3.

The [`python-modernize`][2] can help with this considerably, e.g.:

    python-modernize . -w

will automatically use the [`six`][3] module to apply numerous
transformations required for Python 2/3 compatability.  The
[`python-modernize`][2] tool was used for most Python 3 updates to
`platformio-core` introduced by this merge, with the notable exception of
unicode string handling, which required manual intervention (see below).

Unicode issues
--------------

In Python 3, **all** string literals (i.e., `str` objects) are **unicode** -
**there is NO `unicode` type** in Python 3.  In Python 3, the **`bytes` type**
is equivalent to **`str` in Python 2**.  Note that in Python 2 the `bytes` type
may be used interchangeably with the `str` type.

However, in Python 3 many library functions, e.g., `hashlib.md5.update` require
`bytes` objects; **NOT `str` objects**.  Thus, to work properly with Python 3,
code ported from Python 2 must be updated to **explicitly decode** `str`
unicode string objects to `bytes`, e.g., `my_string.encode('utf8')`.

Furthermore, some Python 3 library functions now **return** `bytes` objects,
e.g., `Popen.communicate`.  Python 2 code interpreting the output of such
functions as `str` string objects **MUST** be updated to explicitly encode the
output to unicode strings to work properly in Python 3, e.g.:

```python
process = subprocess.Popen(...)

stdout, stderr = process.communicate()

print(stdout.decode('utf8'))
```

[1]: https://github.com/platformio/platform-atmelavr.git
[2]: https://github.com/python-modernize/python-modernize
[3]: https://pythonhosted.org/six/

dbrgn added a commit to gfroerli/firmware that referenced this issue Mar 30, 2018

Require Python 2.7 in Pipfile
Platformio does not run on Python 3 yet.

Tracking issue:
platformio/platformio-core#895

dbrgn added a commit to gfroerli/firmware that referenced this issue Mar 31, 2018

Require Python 2.7 in Pipfile (#21)
Platformio does not run on Python 3 yet.

Tracking issue:
platformio/platformio-core#895
@mintisan

This comment has been minimized.

Copy link

mintisan commented Apr 29, 2018

How can I install the specific version for Python3?

It seems to be not OK, if I use pip3 install platformio.

➜  ~ pip3 install platformio
Collecting platformio
  Downloading https://files.pythonhosted.org/packages/7c/95/899efc1264c63d1ba1d32b4160b2fe3d48020de00f7b8c91dc5b712aa05f/platformio-3.5.2.tar.gz (95kB)
    100% |████████████████████████████████| 102kB 238kB/s
    Complete output from command python setup.py egg_info:
    PlatformIO Core v3.5.2 does not run under Python version 3.5.1 (default, Jul  9 2016, 11:42:58)
    [GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)].
    Minimum supported version is 2.7, please upgrade Python.
    Python 3 is not yet supported.

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/7z/j_bymh4x3l7cp_m_xgk132mr0000gn/T/pip-install-dnnmbov9/platformio/
@Peque

This comment has been minimized.

Copy link

Peque commented Apr 30, 2018

@mintisan As you can see in your pip3 command output: "Python 3 is not yet supported".

That means Platformio is currently not Python 3 compatible. Hence this issue. 😉

You may want to try pip2 install platformio (within a system/virtual environment that uses Python 2 instead).

@Cabalist

This comment has been minimized.

Copy link

Cabalist commented May 4, 2018

There is a pull request open for this and I would be happy to help get polish it up if someone could share what is blocking it at the moment. :)

@ivankravets ivankravets modified the milestones: Backlog, 4.0 Aug 6, 2018

ivankravets added a commit that referenced this issue Oct 27, 2018

ivankravets added a commit that referenced this issue Oct 27, 2018

ivankravets added a commit that referenced this issue Oct 27, 2018

@ivankravets

This comment has been minimized.

Copy link
Member

ivankravets commented Dec 26, 2018

Hi all,

Sorry for the delay. We have finally added full support for Python 3.5+. Of course, Python 2.7 is also in the supported list.

We would be thankful if you re-test with $ pio upgrade --dev. It should be PlatformIO 4.0.

Does it work for you?

@13rac1

This comment has been minimized.

Copy link

13rac1 commented Dec 26, 2018

Awesome! Confirming install/run in Ubuntu 18.04 LTS! 🎉 👍 An existing platform.io project builds without issue. All tests, except one pass!

$ python --version
Python 3.6.7
$ python3 -m venv venv # snip
$ source venv/bin/activate # snip
$ git clone https://github.com/platformio/platformio-core --recursive
$ cd platformio-core
$ python setup.py install # snip
$ platformio --version
PlatformIO, version 4.0.0a1
$ pip install pytest # snip
# Run the test command listed in tox.ini
$ py.test -v --basetemp="/tmp/piotest" tests --ignore tests/test_examples.py # Almost Pass!
tests/commands/test_test.py::test_local_env FAILED
____________________________________________________ test_local_env ____________________________________________________

    def test_local_env():
        result = util.exec_command([
            "platformio", "test", "-d",
            join("examples", "unit-testing", "calculator"), "-e", "native"
        ])
        if result['returncode'] != 1:
            pytest.fail(result)
>       assert all([s in result['out']
                    for s in ("PASSED", "IGNORED", "FAILED")]), result['out']
E       AssertionError: Details: http://docs.platformio.org/page/userguide/account/index.html
E         
E       assert False
E        +  where False = all([False, False, False])

tests/commands/test_test.py:29: AssertionError
$ pio test -d examples/unit-testing/calculator -e native
You are not logged in. Please log in to PIO Account using PlatformIO IDE > PIO Home > Account or `pio account login` command and try again. 

If you don't have PIO Account yet, please create it using PlatformIO IDE > PIO Home > Account or `pio account register` command.
Details: http://docs.platformio.org/page/userguide/account/index.html

I have to log in? Ok.

$ pio account register # snip
$ pio account login # snip
$ pio test -d examples/unit-testing/calculator -e native
Verbose mode can be enabled via `-v, --verbose` option
Collected 3 items

========================================= [test/test_common] Building... (1/2) =========================================
Please wait...

========================================= [test/test_common] Testing... (2/2) =========================================
test/test_common/test_calculator.cpp:48:test_function_calculator_addition	[PASSED]
test/test_common/test_calculator.cpp:49:test_function_calculator_subtraction	[PASSED]
test/test_common/test_calculator.cpp:50:test_function_calculator_multiplication	[PASSED]
test/test_common/test_calculator.cpp:51:test_function_calculator_division	[PASSED]

-----------------------
4 Tests 0 Failures 0 Ignored
OK

======================================== [test/test_desktop] Building... (1/2) ========================================
Please wait...

========================================= [test/test_desktop] Testing... (2/2) =========================================
test/test_desktop/test_calculator.cpp:48:test_function_calculator_addition	[PASSED]
test/test_desktop/test_calculator.cpp:49:test_function_calculator_subtraction	[PASSED]
test/test_desktop/test_calculator.cpp:50:test_function_calculator_multiplication	[PASSED]
test/test_desktop/test_calculator.cpp:43:test_function_calculator_division:FAIL: Expected 32 Was 33	[FAILED]

-----------------------
4 Tests 1 Failures 0 Ignored
FAIL

==================================================== [TEST SUMMARY] ====================================================
test/test_common/env:uno	[IGNORED]
test/test_common/env:nodemcu	[IGNORED]
test/test_common/env:native	[PASSED]
test/test_desktop/env:uno	[IGNORED]
test/test_desktop/env:nodemcu	[IGNORED]
test/test_desktop/env:native	[FAILED]
test/test_embedded/env:uno	[IGNORED]
test/test_embedded/env:nodemcu	[IGNORED]
test/test_embedded/env:native	[IGNORED]
============================================== [FAILED] Took 0.71 seconds ==============================================

Edit Again: it fails if I run this test with pio, but passes if I run it, while logged in, with pytest. Seems odd.

tests/commands/test_test.py::test_local_env PASSED                                                               [ 98%]
@rab852

This comment has been minimized.

Copy link

rab852 commented Dec 29, 2018

Using a windows machine I can download the dev branch but still has the python 2.7.13 dependency

@OttoWinter OttoWinter referenced this issue Dec 30, 2018

Merged

Make compatible with python 3 #281

1 of 2 tasks complete
@ivankravets

This comment has been minimized.

Copy link
Member

ivankravets commented Jan 1, 2019

@rab852 PlatformIO IDE currently depends on Python 2.7. Please open an issue here https://github.com/platformio/platformio-vscode-ide/issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment