build py file include Chinese notes show error

[skylin008]

I had a lcd.py file as show:
`#coding:utf-8
"""

LCD 驱动,版本1.0

LCD 为迪文创新的串口屏,型号为 DMG10600T070_A5WTC

使用 RS485 方式进行数据传送,波特率为115200

数据协议格式如下: 起始符(2个字节) + 长度(1个字节) + 命令类型(1个字节) + 地址(2个字节) + 数据内容

16 进制格式 5A A5 0~FF 0x82(写入)/0x83(读取) 详见协议

例子:测距 1-6 跟倒车雷达距离,因为7个数据所在地址是连续的,发送时只给首地址即可

5A A5 11 82 50 00 00 32 00 32 00 32 00 32 00 32 00 32 00 32

"""

from machine import UART,Pin
import gc
import logging
import uasyncio as asyncio
from machine import const
from primitives import delay_ms
import ustruct
from InitConfig import config
from micropython import alloc_emergency_exception_buf

logging.level_name(level = logging.DEBUG)
log = logging.Logger("Lcd")

alloc_emergency_exception_buf(100)

DEBUG = config.get('DEBUG', False)

gc.collect()

version = "1.0"
author = "skylin"

class Lcd(object):

SYNC_BYTE = b'\x5A\xA5'
CMD_WRITE = b'\x82'
CMD_READ  = b'\x83'
SYMSTART  = b'\x5A\xA5\x04\x83\x00\x0F\x01'   # 用于判断硬件连接


def __init__(self, uartId = 3, baudrate = 115200, timeout = 5, ctrlPin = None):
    self._uart    = UART(uartId, baudrate, timeout)
    self.swriter  = asyncio.StreamWriter(self._uart, {})
    self.sreader  = asyncio.StreamReader(self._uart)
    self.timeout  = 30          # interval 
    self.delay    = delay_ms.Delay_ms()
    self.isRuning = False 
    self.response = []
    if ctrlPin == None:
        self.ctrlPin = Pin("485_TR3", Pin.OUT, value = 1)  # high level is transmitter, low level is receiver
    self.send(SYMSTART)
    self.ctrlPin(0)
    asyncio.create_task(self._recv())

async def _recv(self):
    count = 0
    while not self.isRuning:
        count += 1
        self.ctrlPin(0)
        res = await self.sreader.read(9)
        self.ctrlPin(1)
        if self.isRuning == False and res is not None:
            if len(res) == 9:             # return 9 bytes
                if 0x47 and 0x12 in res:  # 0x47 for hardware version， 0x12 for software version
                    self.isRuning = True
                    break

            if count > 5:
                if DEBUG:
                    log.info(f "Couldn't establish a connect event after {count} tries\r\n")
                break                    
        self.delay.trigger(self.timeout)   

async def send(self, command):
    self.response = []
    if command is None:
        return None
    else:
        self.ctrlPin(1)
        await self.swriter.awrite("{}".format(command))
        self.ctrlPin(0)
    if DEBUG:
        log.info("<", command)
    self.delay.trigger(self.timeout)
    await self.dealy.wait()
    #return b''.join(self.response)
    return self.response

def state(self):
    return self.isRuning


def close(self):
    self._uart.deinit()

@micropython.native
def frame(self, addr, data):
    
    ''' addr for address '''
    ''' data for data '''

    if addr or data is None:
        return None

    msg = bytearray()
    msg.extend(SYNC_BYTE)     

    if isinstance(data, int):
        a = 1
    else:
        a = len(data)
    length = a * 2 + 3          
    msg.extend(length.to_bytes(1, 'big'))

    msg.extend(CMD_WRITE)
    
    tmpA = ustruct.pack('>H', addr)
    msg.append(tmpA[0])
    msg.append(tmpA[1])

    if isinstance(data, int):
        a = ustruct.pack('>H', data)
        msg.append(a[0])
        msg.append(a[1])

    else:
        for i in range(len(data)):
            tmp = ustruct.pack('>H', data[i])
            msg.append(a[0])
            msg.append(a[1])
    if DEBUG:
        log.info(f'frame msg:{msg}')

    return msg

`
and lcd directory manifest.py as follow:
'
package(
"lcd",
(
"lcd.py",
),
base_path="..",
opt=3,
)
'
and at the same time, I add some content to the boards/manifest.py file as:

include("$(PORT_DIR)/modules/lcd")

when i build the PYBV11 firmware, the error shows:

MPY lcd/lcd.py
Traceback (most recent call last):
File "E:\Project\TianMa\SoftWare_Micropython\micropython\tools\makemanifest.py", line 265, in
main()
File "E:\Project\TianMa\SoftWare_Micropython\micropython\tools\makemanifest.py", line 196, in main
with manifestfile.tagged_py_file(result.full_path, result.metadata) as tagged_path:
File "C:/msys64/mingw64/lib/python3.10/contextlib.py", line 135, in enter
return next(self.gen)
File "E:/Project\TianMa\SoftWare_Micropython\micropython\tools/manifestfile.py", line 547, in tagged_py_file
contents = src.read()
UnicodeDecodeError: 'gbk' codec can't decode byte 0x8f in position 76: illegal multibyte sequence
make: *** [../../py/mkrules.mk:194: build-PYBV11/frozen_content.c] Error 1
make: *** Waiting for unfinished jobs....

How to solve this issue? Thanks !

[glenn20]

Python's open() call uses the default encoding of your platform (seems to be Windows and is environment dependent). It appears that your environment has encoding set to "gbk", which causes tools/manifestfile.py to fail in decoding some sequences in your text. You can tell python to ignore the environment and use utf-8 for all encodings by setting an environment variable PYTHONUTF8=1 (See https://docs.python.org/3/using/cmdline.html#envvar-PYTHONUTF8). Good luck.

[glenn20]

Oh, and to clarify, when python loads lcd.py as source code (eg. import lcd) it treats that file as utf-8 encoded by default (python treats all source code as utf-8 irrespective of the local platform encoding). However, when the micropython tools are packaging it up, they are just reading it as a text file.

It would probably be better if tools/manifestfile.py:

• opened the file in "rb" mode, instead of "r" mode to avoid the whole encodings issue and just copy the file with it's original encoding
• OR realise that when opening .py files it should always set the encoding to "utf-8" (because that is what python does when importing .py files.

Setting the PYTHONUTF8 env variable is a pretty acceptable work-around which makes tools/manifestfile.py behave as in the second option.

[skylin008]

Thanks @glenn20 , I will be try!