Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UnicodeDecodeError in platformio\util.py line 420 #2100

Closed
JustACatRu opened this issue Feb 5, 2019 · 9 comments

Comments

2 participants
@JustACatRu
Copy link

commented Feb 5, 2019

  • PlatformIO Core.
    Error: UnicodeDecodeError: Non-ASCII characters found in build environment
    Real error: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x88 in position 0: invalid start byte

Configuration

  • Windows 10 x64 (ru)
  • PlatformIO (Core CLI through "pip install -U https..."), version 4.0.0a3
  • Python 3.7.2
  • Eclipse as IDE (not matter, it happens with all IDEs)

Description of problem

When using "pio init" with "--ide" parameter (no matter what ide) with platform = espressif8266.
2019-02-01_001338
And there is no Non-ASCII characters in my paths as I can see:
2019-02-01_001404

Details

If I select some avr board like "pio init --board pro16MHzatmega328" then "pio init --ide eclipse" works without errors:
2019-02-02_013425
If I select esp board like "pio init --board esp07" then "pio init --ide eclipse" produce error:
2019-02-02_013619
If there is 2 envs in a project and avr goes first in "platformio.ini" then all is workig fine too:
2019-02-02_014015
But if I put "env:esp07" first, "pio init --ide eclipse" stops working again.

@JustACatRu

This comment has been minimized.

Copy link
Author

commented Feb 5, 2019

I've modified code a little bit and found traceback of error (this catch hides traceback):
2019-02-02_023330
Here:

result[k] = result[k].decode()

For me it starts working, if I change that line to any of this variants:

result[k] = str(result[k])
result[k] = str(result[k], "windows-1251")
result[k] = result[k].decode("windows-1251")

2019-02-02_024315
I do not know if it's a right solution. And which variant is the best and correct. And why it is working with "windows-1251" encoding. My Python knowledge is way too low for this :)
But hope all this will help solve the problem quicker.

PS: I've transferred this issue and details from there: platformio/platformio-atom-ide#768 (comment) - because it is not Atom IDE specific.

@JustACatRu

This comment has been minimized.

Copy link
Author

commented Feb 6, 2019

I've found out, that this variant:

result[k] = str(result[k])

coused this bug. Because here:

Passing a bytes object to str() without the encoding or errors arguments falls under the first case of returning the informal string representation (see also the -b command-line option to Python).
For example:

str(b'Zoot!')
"b'Zoot!'"

So, the encoding must be specified explicitly:

result[k] = result[k].decode("windows-1251")
# Or:
result[k] = str(result[k], "windows-1251")

But it should be "windows-1251", not "utf-8". I still do not get why.

@JustACatRu

This comment has been minimized.

Copy link
Author

commented Feb 6, 2019

When I saw this line:

with open(path, encoding="latin-1") as f:

I tried this variant:

result[k] = result[k].decode("latin-1")

And it works too. So decode("latin-1") or decode("windows-1251") both working for me.
But not decode("utf-8") or decode() which is equivalent.

@ivankravets ivankravets added this to the 4.0 milestone Feb 23, 2019

@ivankravets ivankravets added the bug label Feb 23, 2019

@ivankravets ivankravets added help wanted and removed bug labels Apr 18, 2019

@ivankravets

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

Could you reproduce this issue with the latest pio upgrade --dev?

@ivankravets

This comment has been minimized.

Copy link
Member

commented Apr 24, 2019

Please reopen if you still need help

@JustACatRu

This comment has been minimized.

Copy link
Author

commented Apr 27, 2019

Could you reproduce this issue with the latest pio upgrade --dev?

Yep. Same issue:
2019-04-27_133423

@ivankravets ivankravets reopened this Apr 27, 2019

@ivankravets ivankravets added this to Bugs in PlatformIO 4.0 May 24, 2019

@ivankravets ivankravets added this to Low priority in Known Issues May 30, 2019

@ivankravets ivankravets moved this from Low priority to High priority in Known Issues May 30, 2019

@ivankravets ivankravets removed this from Bugs in PlatformIO 4.0 May 30, 2019

@ivankravets ivankravets added bug and removed help wanted labels Jun 1, 2019

@ivankravets ivankravets removed this from High priority in Known Issues Jun 3, 2019

@ivankravets ivankravets added this to Done in PlatformIO 4.0 via automation Jun 3, 2019

@ivankravets

This comment has been minimized.

Copy link
Member

commented Jun 3, 2019

Please re-test with pio upgrade --dev. Thanks!

@JustACatRu

This comment has been minimized.

Copy link
Author

commented Jun 3, 2019

Please re-test

Just tested on version 4.0.0a18 and it is working without issues now! Great and thanks.

@ivankravets

This comment has been minimized.

Copy link
Member

commented Jun 3, 2019

Thanks for the report! We are so close for the amazing release :)

ivankravets added a commit that referenced this issue Jun 5, 2019

Fix numerous issues related to "UnicodeDecodeError" and international…
… locales, or when project path contains non-ASCII chars // Resolve #143, Resolve #1342, Resolve #1959, Resolve #2100
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.