Problem with Python 3.8+ on Windows when network drive is used

[Hofei90]

The cause of the issue: https://bugs.python.org/issue37993

Solution 1

1. Open Windows Registry regedit.exe
2. Create a new key named Command Processor at HKEY_CURRENT_USER\Software\Microsoft
   
3. Create a new DWORD (32-bit) Value named DisableUNCCheck
   
4. Set the value of DisableUNCCheck to 1 (Hex)
   
5. Restart your Windows machine.

Solution 2

1. Download our portable Python 3.7
2. Extract and replace contents in %HOMEPATH%/.platformio/python3.
3. Remove %HOMEPATH%/.platformio/penv folder.
4. Restart VSCode.

---

Configuration

Operating system:
Windows_NT x64 10.0.17763

PlatformIO Version (platformio --version):
4.2.2b1
and
4.2.1

Python Version
3.8 32bit

Description of problem

During compilation I get the Python Traceback UnicodeDecodeError: 'charmap' codec can't decode byte.

Full Message:

> Executing task in folder IR_Sender_WB: C:\Users\***\.platformio\penv\Scripts\platformio.exe run <

Processing nodemcuv2 (platform: espressif8266; board: nodemcuv2; framework: arduino)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/nodemcuv2.html
PLATFORM: Espressif 8266 2.3.3 > NodeMCU 1.0 (ESP-12E Module)
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
PACKAGES:
- framework-arduinoespressif8266 3.20603.200130 (2.6.3)
- tool-esptool 1.413.0 (4.13)
- tool-esptoolpy 1.20800.0 (2.8.0)
- toolchain-xtensa 2.40802.191122 (4.8.2)
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 31 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <IRremoteESP8266> 2.7.4
Building in release mode
Compiling .pio\build\nodemcuv2\src\main.cpp.o
Generating LD script .pio\build\nodemcuv2\ld\local.eagle.app.v6.common.ld
Exception in thread Thread-3:
Traceback (most recent call last):
  File "C:\Users\***\AppData\Local\Programs\Python\Python38-32\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
  File "C:\Users\***\AppData\Local\Programs\Python\Python38-32\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "c:\users\***\.platformio\penv\lib\site-packages\platformio\proc.py", line 46, in run
    self.do_reading()
  File "c:\users\***\.platformio\penv\lib\site-packages\platformio\proc.py", line 69, in do_reading
    for byte in iter(lambda: self._pipe_reader.read(1), ""):
  File "c:\users\***\.platformio\penv\lib\site-packages\platformio\proc.py", line 69, in <lambda>
    for byte in iter(lambda: self._pipe_reader.read(1), ""):
  File "C:\Users\***\AppData\Local\Programs\Python\Python38-32\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 167: character maps to <undefined>
Compiling .pio\build\nodemcuv2\lib70a\IRremoteESP8266\IRac.cpp.o
Compiling .pio\build\nodemcuv2\lib70a\IRremoteESP8266\IRrecv.cpp.o
Compiling .pio\build\nodemcuv2\lib70a\IRremoteESP8266\IRsend.cpp.o
Compiling .pio\build\nodemcuv2\lib70a\IRremoteESP8266\IRtext.cpp.o
Compiling .pio\build\nodemcuv2\lib70a\IRremoteESP8266\IRtimer.cpp.o
Compiling .pio\build\nodemcuv2\lib70a\IRremoteESP8266\IRutils.cpp.o
======================================================================================== [FAILED] Took 5.42 seconds ========================================================================================
The terminal process terminated with exit code: 1

Terminal will be reused by tasks, press any key to close it.

If problems with PlatformIO Build System:

The content of platformio.ini:

; PlatformIO Project Configuration File
;
;   Build options: build flags, source filter
;   Upload options: custom upload port, speed and extra flags
;   Library options: dependencies, extra library storages
;   Advanced options: extra scripting
;
; Please visit documentation for the other options and examples
; https://docs.platformio.org/page/projectconf.html

[env:nodemcuv2]
platform = espressif8266
board = nodemcuv2
framework = arduino
monitor_speed = 115200

Additional info

On another PC with Python version 3.7 there are no problems compiling.
Here are the data of the PC where there is no problem:

Version: 1.43.0 (user setup)
Commit: 78a4c91400152c0f27ba4d363eb56d2835f9903a
Date: 2020-03-09T19:47:57.235Z
Electron: 7.1.11
Chrome: 78.0.3904.130
Node.js: 12.8.1
V8: 7.8.279.23-electron.0
OS: Windows_NT x64 10.0.17763

PlatformIO: Core 4.2.1, Home 3.1.1

[ivankravets]

1. Does both machines use the same user name?
2. Do you have non-ASCII chars in the user name on a machine that fails?
3. Do you use the same locale?

[Hofei90]

1. Does both machines use the same user name?

no the machines have different user names

2. Do you have non-ASCII chars in the user name on a machine that fails?

no, the names contain only characters from [A-Z] [a-z] and char \ from the domain Name. But Domainname is same on both maschines.

3. Do you use the same locale?

Do you mean the character encoding setting like UTF-8? In Visual Studio Code is it UTF-8 or you mean on the OS?

[ivankravets]

Where did you get this Python 3.8? Could you try to uninstall it and install again the latest Python 3? Also, please remove .platformio/penv folder form user home folder. Restart VSCode.

[Hofei90]

Hello,
all Python versions were downloaded from the official website. (https://www.python.org/downloads/)

What did I do:

1. uninstalled Python 3.8 and installed the last release with version 3.8.2 and deleted the penv folder and restarted VS Code
   Result: Problem still exists
2. platformIO completely deleted from VS Code and reinstalled (Version 4.2.1)
   Result Problem still exists
3. uninstall Python 3.8.2 and install Python 3.7.3, so that the versions are exactly the same as the working PC. delete the penv folder and restart VS Code.
   Result: Compilation works!

Conclusion:
Python 3.8 causes the error posted above
In Python 3.7 these problems do not yet exist

Translated with www.DeepL.com/Translator (free version)

[ivankravets]

@Hofei90 could you provide a simple project ot reproduce this issue?

[Hofei90]

This is the project where the problems were.
Just removed the docs folder, .vscode, .git and .pio.
IR_Sender_WB.zip

[ivankravets]

I moved this issue to ESP8266 dev-platform. I tried in my VM and cannot reproduce it.

---

@Hofei90 , could you open PlatformIO IDE Terminal and change directory to this project and run:

pio run --jobs 1

Could you provide a new output? It will show where it fails.

[Hofei90]

I tried in my VM and cannot reproduce it.

Did you use Python 3.8.2 32bit too?
I just installed Python 3.8.2 on the other PC where 3.7 was previously installed and it worked.
After upgrading to Python 3.8.2, compiling no longer works.

Here is the output with
pio run --jobs 1

PS U:\gitea\IR_Sender_WB> pio run --jobs 1
Processing nodemcuv2 (platform: espressif8266; board: nodemcuv2; framework: arduino)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/nodemcuv2.html
PLATFORM: Espressif 8266 2.3.3 > NodeMCU 1.0 (ESP-12E Module)
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
PACKAGES:
 - framework-arduinoespressif8266 3.20603.200130 (2.6.3)
 - tool-esptool 1.413.0 (4.13)
 - tool-esptoolpy 1.20800.0 (2.8.0)
 - toolchain-xtensa 2.40802.191122 (4.8.2)
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 30 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <IRremoteESP8266> 2.7.4
Building in release mode
Compiling .pio\build\nodemcuv2\src\main.cpp.o
Exception in thread Thread-3:
Traceback (most recent call last):
  File "C:\Users\***\AppData\Local\Programs\Python\Python38-32\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
  File "C:\Users\***\AppData\Local\Programs\Python\Python38-32\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "c:\users\***\.platformio\penv\lib\site-packages\platformio\proc.py", line 46, in run
    self.do_reading()
  File "c:\users\***\.platformio\penv\lib\site-packages\platformio\proc.py", line 69, in do_reading
    for byte in iter(lambda: self._pipe_reader.read(1), ""):
  File "c:\users\***\.platformio\penv\lib\site-packages\platformio\proc.py", line 69, in <lambda>
    for byte in iter(lambda: self._pipe_reader.read(1), ""):
  File "C:\Users\***\AppData\Local\Programs\Python\Python38-32\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 166: character maps to <undefined>
======================================================================================== [FAILED] Took 5.49 seconds ========================================================================================
PS U:\gitea\IR_Sender_WB>

EDIT, add Output with verbose Option:

PS U:\gitea\IR_Sender_WB> pio run --jobs 1 --verbose
Processing nodemcuv2 (platform: espressif8266; board: nodemcuv2; framework: arduino; monitor_speed: 115200)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/nodemcuv2.html
PLATFORM: Espressif 8266 2.3.3 > NodeMCU 1.0 (ESP-12E Module)
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
PACKAGES:
 - framework-arduinoespressif8266 3.20603.200130 (2.6.3)
 - tool-esptool 1.413.0 (4.13)
 - tool-esptoolpy 1.20800.0 (2.8.0)
 - toolchain-xtensa 2.40802.191122 (4.8.2)
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 30 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <IRremoteESP8266> 2.7.4 (\\***\***\***\gitea\IR_Sender_WB\lib\IRremoteESP8266)
Building in release mode
xtensa-lx106-elf-g++ -o .pio\build\nodemcuv2\src\main.cpp.o -c -fno-rtti -std=c++11 -Os -mlongcalls -mtext-section-literals -falign-functions=4 -U__STRICT_ANSI__ -ffunction-sections -fdata-sections -fno-exceptions -Wall -DPLATFORMIO=40201 -DESP8266 -DARDUINO_ARCH_ESP8266 -DARDUINO_ESP8266_NODEMCU -DF_CPU=80000000L -D__ets__ -DICACHE_FLASH -DARDUINO=10805 -DARDUINO_BOARD=\"PLATFORMIO_NODEMCUV2\" -DFLASHMODE_DIO -DLWIP_OPEN_SRC -DNONOSDK22x_190703=1 -DTCP_MSS=536 -DLWIP_FEATURES=1 -DLWIP_IPV6=0 -DVTABLES_IN_FLASH -Iinclude -Isrc -Ilib\IRremoteESP8266\src -IC:\Users\***\.platformio\Exception in thread Thread-3:
pTraceback (most recent call last):
a  File "C:\Users\***\AppData\Local\Programs\Python\Python38-32\lib\threading.py", line 932, in _bootstrap_inner
ck    self.run()
a  File "C:\Users\***\AppData\Local\Programs\Python\Python38-32\lib\threading.py", line 870, in run
ges\fra    self._target(*self._args, **self._kwargs)
m  File "c:\users\***\.platformio\penv\lib\site-packages\platformio\proc.py", line 46, in run
ewo    self.do_reading()
r  File "c:\users\***\.platformio\penv\lib\site-packages\platformio\proc.py", line 69, in do_reading
k-    for byte in iter(lambda: self._pipe_reader.read(1), ""):
a  File "c:\users\***\.platformio\penv\lib\site-packages\platformio\proc.py", line 69, in <lambda>
rdui    for byte in iter(lambda: self._pipe_reader.read(1), ""):
n  File "C:\Users\***\AppData\Local\Programs\Python\Python38-32\lib\encodings\cp1252.py", line 23, in decode
oes    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
pUnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 166: character maps to <undefined>
ressif8266\tools\sdk\include -IC:\Users\***\.platformio\packages\framework-arduinoespressif8266\tools\sdk\libc\xtensa-lx106-elf\include -IC:\Users\***\.platformio\packages\framework-arduinoespressif8266\cores\esp8266 -IC:\Users\***\.platformio\packages\framework-arduinoespressif8266\tools\sdk\lwip2\include -IC:\Users\***\.platformio\packages\framework-arduinoespressif8266\variants\nodemcu
src\main.cpp
======================================================================================== [FAILED] Took 5.44 seconds ========================================================================================

[ivankravets]

\****\userdaten

Could you try to move this project to C:\test-project. Just for test.

[Hofei90]

Good eye, can you please make the path in your post unrecognizable and also remove it from the history

If the folder is on C: it works.
Funny though that under Python 3.7 it also works on the network folder

[ivankravets]

can you please make the path in your post unrecognizable

Sorry, done!

---

So, I moved back this issue to PIO Core because it looks like a bug. What is a type of this drive? How can I create it?