Python rewrite

[Qonfused]

Test coverage:

• Python: Python rewrite #4 (comment)

Tasks:

CI (Python)

• Handle semver versioning schemes
  • Support automatic versioning schemes (e.g. via git tags)
• Add environment checks for future versions of Python.
• Ensure namespace and module resolution are not mangled
  • Automatically generate API exports for codegen
• Sort import order for packages and modules

CLI (Python)

• Add CLI support for --version flag
• Add shared context for verbosity and debug flags
• Add shared context for config and build file paths
• Commands
  • Build command
  • Lock command
  • Patch command

Library (C++, Python)

• Errors
  • Traceback frame introspection and suppression
  • Custom error classes (runtime/tests)
  • Runtime validation methods
• Filesystem
  • Handle extracting remote archives to temporary directories
  • Expose missing POSIX filesystem operations
• Parsers
  • Tokenizer/AST
    • (Native)
    • Stringified (asserting all values as string)
    • Flat (stringifying all nested keys)
  • Plist - Apple PLIST 1.0
    • Dictionary (root)
    • Array (root)
  • YAML - YAML 1.2
    • (Native)
    • Annotated (see example config.yml)
    • Stringified (asserting all values as string)
    • Flat (stringifying all nested keys)
• Pipeline
  • Build file/configuration
    • Read and parse build file
    • Normalize entries
    • Validate build file against schema
    • Generate a template build file from an existing EFI structure
  • Config
    • Read and parse config file
    • Parse schema from Sample.plist/config.plist file
      • Validate config.plist against a schema
    • Merge and patch config.plist
      • Handle config inheritance
  • Kexts
    • Extract kexts from archives
    • Read and parse kext manifest
    • Validate kext manifest against build configuration
    • Sort kext load order by OSBundleLibraries field
  • Lockfile
    • Read and parse lockfile
    • Handle parsing build configuration specifiers
    • Handle creating or updating the lockfile
    • Validate lockfile against build configuration
    • Cache package sources and checksums
    • Validate lockfile updates against cached packages
  • OpenCore
    • Read and parse the OpenCore (build) configuration
    • Unpack the OpenCore archive to a temporary directory
    • Extract bundled OpenCore binaries, scripts + documentation
  • SSDTs
    • Extract and wrap iASL binary
    • Handle decompilation of SSDT binaries
    • Handle patching of SSDT source files
    • Handle compilation of SSDT source files
    • Sort SSDTs by resolving symbolic references
• Sources
  • Handle remote sources
    • GitHub
      • Commits - raw + tarball sources
      • CI/CD - GitHub Actions
      • Releases
    • Dortania build repo
    • Handle arbitrary archives/URLs
  • Handle local sources
    • Filesystem - Raw path and RFC 8089 file: protocol
    • Local cache
• Versioning
  • Support PEP 440 and PEP 425
  • Add support for semantic versioning
    • Support NPM.js semver resolution.

[Qonfused]

Schema docs generation is mostly uncoupled from the main library, though there currently exists a markdown parser in the ocebuild/parsers/schema.py file that may be worth moving separately under the current docsgen pipeline.

A more robust solution worth exploring is to use pandoc (i.e. ran in CI w/ a docker image), which will more reliably convert each schema's LaTeX entry to markdown. There is currently no action needed for the current schema parser which parses and segments these entries.

[Qonfused]

Schema docs will need to be updated on a regular cadence following each OpenCore release. Pre-release builds of OpenCore will also need to be smoke-tested against any schema validation in case new schema entries are added that must be specially handled.

Additionally, the usage of the '#Revision' dict mimics the same usage by OCLP in its payload config.plist here. Special care must be taken when introducing metadata keys that may conflict with other projects.

[Qonfused]

Unfortunately due to a bug in the paths filtering behavior for GitHub actions, there is currently a block preventing progress on the CI for orchestrating integration testing. This will force current progress in this PR to be merged pre-maturely in order to continue.

The core library is mostly stable and tested, however, what is remaining regarding development and test coverage is primarily the pipeline supporting ocebuild_cli:

For the CLI, the build and lock commands are mostly finished and are pending the patch command to be implemented, which will handle the remaining config.plist patching for the build pipeline.