Cross-platform alternative to amlogic usb tool binary

[bishopdynamics]

I found this repo which appears to be source for an earlier version of the amlogic usb tool. Specifically, it appears to be:
Amlogic update USB tool(Ver 1.5) 2017/05
as opposed to the one provided by Frederic, which is:
Amlogic update USB tool(Ver 1.7.2) 2018/04

I had no trouble building it on Ubuntu x86_64, but it doesn't work with superbird, which isn't terribly surprising.

There is another repo, forked from the first one, with some changes made to support p230. Probably not much use for us.

I also found that the khadas repo has the same update tool for Windows, x86_64 Linux, and armhf Linux. The versions here match the one provided by frederic. I got excited at the armhf one, but I'm getting an exec format error when i try to use it, so I think I'm missing some detail. I haven't tried the Windows one. There are some additional tools in that repo that may be useful as well.

So, my point is: If someone understands what's going on better than I, they could tweak that old source so that it works with superbird, and then we would have a tool that could potentially be ported to other platforms.

I am particularly interested in this, because my main dev machine is aarch64, and I had to haul out an old Intel machine to work on superbird; It

[bishopdynamics]

Update: I previously knew about pyamlboot but thought all it could do was boot images.

I was wrong, and in-fact you can totally use bulkcmd.py as a replacement for update bulkcmd !

On macOS, I had to use python3 and libusb from homebrew:

# install libusb and python3 from homebrew
brew install libusb python3

# need to use the homebrew version of python3, so that it will find libusb properly
PY_CMD="/opt/homebrew/bin/python3"

# install pyamlboot from source, as bulkcmd.py does not ship with the pypy packages
git clone https://github.com/superna9999/pyamlboot pyamlboot
$PY_CMD -m pip install pyamlboot/.

# Now lets try the "continue booting" command
# instead of: update bulkcmd 'mw.b 0x17f89754 1'
$PY_CMD pyamlboot/bulkcmd.py 'mw.b 0x17f89754 1'

[bishopdynamics]

In an effort to produce a cross-platform example of how to practically use pyamlboot, I have implemented fredric's upload-kernel.sh as a python script using pyamlboot

I tested this working on macOS and Linux, aarch64 and x86_64 for both.
Edit: updated, tested working on windows too

boot-adb-kernel.py:

#!/usr/bin/env python3
"""
Boot using kernel and initrd in images/
"""

import sys
import traceback

KERNEL_ADDR = '0x01080000'
INITRD_ADDR = '0x13000000'
ENV_ADDR = '0x13000000'

FILE_ENV = 'images/env.txt'
FILE_KERNEL = 'images/superbird-adb.kernel.img'
FILE_INITRD = 'images/superbird-adb.initrd.img'

try:
    from pyamlboot import pyamlboot
    from usb.core import USBTimeoutError, USBError
except ImportError:
    print("""
    ###########################################################################################

    Error while importing pyamlboot!
    
    on macOS, you must install python3 and libusb from homebrew, 
    and execute using that version of python
        brew install python3 libusb
        /opt/homebrew/bin/python3 -m pip install git+https://github.com/superna9999/pyamlboot
        /opt/homebrew/bin/python3 boot-adb-kernel.py
    root is not needed on macOS
    
    on Linux, you just need to install pyamlboot
    root is needed on Linux, unless you fiddle with udev rules, 
    which means the pip package also needs to be installed as root
        sudo python3 -m pip install git+https://github.com/superna9999/pyamlboot
        sudo ./boot-adb-kernel.py
    
    on Windows, you need to download and install python3 from https://www.python.org/downloads/windows/
    and execute using "python" instead of "python3"
        python -m pip install git+https://github.com/superna9999/pyamlboot
        python boot-adb-kernel.py
    
    You need to install pyamlboot from github because the current pypy package is too old, missing bulkcmd
    
    ############################################################################################
    """)
    sys.exit(1)


class AmlDevice:
    """ convenience wrapper """
    def __init__(self) -> None:
        self.device = pyamlboot.AmlogicSoC()
        try:
            self.device.bulkCmd('echo Testing bulkcmd')
        except AttributeError as exaa:
            if exaa.name == 'bulkCmd':
                self.print('Detected an old version of pyamlboot which lacks AmlogicSoC.bulkCmd')
                self.print('Need to install from the github master branch')
                self.print(' need to uninstall the current version, then install from github')
                self.print('  python3 -m pip uninstall pyamlboot')
                self.print('  python3 -m pip install git+https://github.com/superna9999/pyamlboot')
                sys.exit(1)
            raise exaa

    @staticmethod
    def decode(response):
        """ decode a response """
        return response.tobytes().decode("utf-8")

    @staticmethod
    def print(message:str):
        """ print a message to console
            on Windows, need to flush after printing
            or nothing will show up until script is complete
        """
        print(message)
        sys.stdout.flush()

    def bulkcmd(self, message:str):
        """ perform a bulkcmd, separated by semicolon """
        self.print(f' executing bulkcmd: "{message}"')
        try:
            response = self.device.bulkCmd(message)
            self.print(f'  result: {self.decode(response)}')
        except USBTimeoutError as exut:
            # if you use booti, it wont return, thus will raise USBTimeoutError
            if 'booti' in message:
                self.print('  booting...')
            else:
                self.print('  Error: bulkcmd timed out!')
                raise exut
        except USBError as exut:
            # on Windows, throws USBError instead of USBTimeoutError
            if 'booti' in message:
                self.print('  booting...')
            else:
                self.print('  Error: bulkcmd timed out!')
                raise exut

    def write(self, address:str, data, chunk_size=8):
        """ write data to an address """
        self.print(f' writing to: {address}')
        self.device.writeLargeMemory(int(address, 0), data, chunk_size, True)

    def send_env(self, env_string:str):
        """ send given env string to device, space-separated kernel args on one line """
        env_size_bytes = len(env_string.encode('utf-8'))
        env_size = str(hex(env_size_bytes))  # get size of env in bytes
        self.print('clearing runtime env')
        self.bulkcmd('amlmmc env')  # clear runtime env
        self.print(f'sending env ({env_size_bytes} bytes)')
        # self.print(env_string)
        self.write(ENV_ADDR, env_string.encode('utf-8'))  # write env string somewhere
        self.bulkcmd(f'env import -t {ENV_ADDR} {env_size}')  # read env from string

    def send_env_file(self, env_file:str):
        """ read env.txt, strip any newlines, then send it to device
            we remove newlines, so that you can use them in env.txt,
            making it easier to read and edit
        """
        env_data = ''
        with open(env_file, 'r', encoding='utf-8') as envf:
            env_data = ' '.join([line.rstrip('\n') for line in envf])  # strip out newline chars
        self.send_env(env_data)

    def send_file(self, filepath:str, address:str):
        """ write given file to device memory at address """
        self.print(f'writing {filepath} at {address}')
        file_data = None
        with open(filepath, 'rb') as flp:
            file_data = flp.read()
        self.write(address, file_data, chunk_size=512)

    def boot(self, env_file:str, kernel:str, initrd:str):
        """ boot using given env.txt, kernel, and initrd """
        self.print(f'Booting {env_file}, {kernel}, {initrd}')
        self.send_env_file(env_file)
        self.send_file(kernel, KERNEL_ADDR)
        self.send_file(initrd, INITRD_ADDR)
        self.print('Booting kernel with initrd')
        self.bulkcmd(f'booti {KERNEL_ADDR} {INITRD_ADDR}')


if __name__ == "__main__":
    try:
        print()
        print('Looking for device...')
        aml = AmlDevice()
    except ValueError:
        print('Device not found, is it in usb burn mode?')
        sys.exit(1)
    except USBError as exu:
        if exu.errno == 13:
            # [Errno 13] Access denied (insufficient permissions)
            print(f'{exu}, need to run as root')
        else:
            print(f'Error: {exu}')
            print(traceback.format_exc())
    else:
        try:
            aml.boot(FILE_ENV, FILE_KERNEL, FILE_INITRD)
            sys.exit(0)
        except Exception as ex:
            print(f'Error: {ex}')
            print(traceback.format_exc())
        sys.exit(1)

[worst-computer-user]

I can confirm this works - granted you grab the images (and env.txt) from @bishopdynamics repo. We should probably maintain a single repo with all the tooling/examples, preferably this one?

I'm currently working on a cross-compiler toolchain that would produce binaries compatible with the glibc version present in Car Thing's linux - I'm able to compile SDL and some basic SDL based games already.

[bishopdynamics]

I have tried a USB OTG Cable, along with USB Ethernet adapters, and WiFi adapters, with no luck. They don't show up when i run lsusb. It might be that we need to put the usb port in the right mode to host devices. It also might be that the stock os doesn't actively look for new devices, since the hardware should never change. If someone can figure out how to get external USB devices to populate, then we will be in business

[williamtcastro]

I was wondering about this on the other open issue. As soon mines arrive I will try tinker with it

[bishopdynamics]

oh weird, i thought i was commenting on the other issue, oops!

[big-harry]

I've tested this on multiple Mac devices (MacBookPro16,1 and MacBookPro9,2) and neither of them work with this. (Also dmesg doesn't show the device, but System Information does).

[big-harry]

Tried this on a Raspberry Pi also and got the same result. On the Macs I got error code 60 but otherwise they were the same.

[ckosmic]

I'm currently working on a cross-compiler toolchain that would produce binaries compatible with the glibc version present in Car Thing's linux - I'm able to compile SDL and some basic SDL based games already.

I've been able to compile SDL2 programs, but haven't been able to run any (create a GL context) because of the Car Thing not displaying anything past the Spotify logo at the beginning when plugged into a USB host. Have you gotten any to run? Would I have to run it while unplugged from a computer? Sorry it's a bit off topic this is just the one place I've seen anyone talk about SDL on the Car Thing

[bishopdynamics]

@big-harry I wonder if you have the device connected through a USB hub? I noticed that I got timeout errors much more frequently when using a hub, and that they went away when I connected directly to the device.

I have expanded upon that script to create superbird-tool, hopefully it will encourage more development from others!