Skip to content

Latest commit

 

History

History
162 lines (131 loc) · 6.55 KB

lang_python.adoc

File metadata and controls

162 lines (131 loc) · 6.55 KB
Raw

Kaitai Struct: Python notes

Table of Contents

Overview

Kaitai Struct compiler invocation

To generate Python modules, Kaitai Struct compiler should be invoked with -t python option:

kaitai-struct-compiler -t python my_spec.ksy

This normally generates my_spec.py parser module, which is compatible with both Python 2 and 3. It is strongly recommended to use python 3 because python 2 is EOL since 2020-01-01.

--python-package allows you to select a parent package name used by one generated file to import symbols from another one. If you want to embed KSC-generated files into an own package, it is recommended to set it to ., which means "current package" rather than to the real name of the subpackage, because setting it to a real name will cause breakage when you want to reorganize or rename your package.

Adding runtime library

Code generated by Kaitai Struct compiler will require additional runtime library (kaitaistruct at PyPi, sources at GitHub).

Runtime library API version must match compiler version that you’re using (i.e. if you use ksc v0.8, then you must use compatible v0.8 runtime).

If you are using non-release KSC, you should also use non-release runtime.

Using release version

With stable version of Kaitai Struct compiler, one can use use following methods:

  • On Debian / Ubuntu, one can install it from apt repositories:

    • For Python 3.x:

      sudo apt-get install -y python3-kaitaistruct

      Reference: package info for Debian, package info for Ubuntu

    • For Python 2.x:

      sudo apt-get install -y python-kaitaistruct

      Reference: package info for Debian, package info for Ubuntu

  • Add the following dependency specifier into the package configuration of the packaging tool you use:

    kaitaistruct

    and then run python3 -m pip install --upgrade -e ./ in the dir with your project in order to install your package and all its dependencies in an editable way. If you don’t want to package your software, but only installbits dependencies, add kaitaistruct into requirements.txt.

  • Otherwise, you can just install the package manually using pip (Python package manager):

    python3 -m pip install --upgrade kaitaistruct

Using latest (snapshot) version

If you’re using latest (unreleased AKA "unstable" AKA "snapshot" AKA "nightly") build of Kaitai Struct compiler, that will require latest runtime libraries as well:

  • You can a make pip to use a bleeding edge version of the runtime by adding a PEP 508 dependency specifier into the config of your favorite build tool:

        kaitaistruct @ git+https://github.com/kaitai-io/kaitai_struct_python_runtime.git

    .

  • Otherwise, you can just install the package manually using pip (Python package manager):

    python3 -m pip install --upgrade --pre git+https://github.com/kaitai-io/kaitai_struct_python_runtime.git

Adding dependencies manually

Enums implementation in Python requires a enum support as described in PEP-0435. This support is available out of the box in standard libraries since Python v3.4, or since v2.4 using enum34 PyPi compatibility layer.

  • One usually doesn’t need to install this lib manually, pip will install it automatically if it is needed.

  • One can also install it manually:

    • One can use just sudo python -m pip install enum34. Don’t do it on later pythons or the apps using just enum will be broken!

    • On Debian / Ubuntu, this library can be installed with sudo apt-get install -y python-enum34.

Automation of compilation process

Setuptools is the de-facto standard way of building and installing python packages. This library allows extension with plugins. https://gitlab.com/KOLANICH has developed an extension to setuptools, allowing you to automate compilation of *.ksy files as the part of python module build process. The library provides a mostly declarative syntax to facilitate installation: you add a parameter kaitai to your setup.py where you enumerate files that you want compiled, output directory, compilation flags, and source code post-compilation transformations. Here are some examples: 1, 2, 3, 4, 5.

Here are the docs in JSON Schema format.

Please remember that this lib is VERY UNSTABLE and very likely to be improved in a way that changes API (but due to API simplicity it’d be easy to fix your code), but I guess it’s much better to use this than to reinvent the wheel. Contributions are welcome!

Please note that this lib is NOT available on pypi and if you want to install it automatically, you should add kaitaiStructCompile.setuptools @ https://gitlab.com/kaitaiStructCompile.py/kaitaiStructCompile.setuptools into setup_requires into options section of your setup.cfg and into requires of build_system section of your pyproject.toml.