Migration from v2 to v3

Aim

Lazuli v2 already had some deviations from SwordieDB, to improve re-usability of components, but we feel like it's still no where near good enough.

One of the issues Lazuli faces is the bloated requirements file, which contains libraries that are not used by the end-user. In Lazuli v3, we will maintain separate virtual environments and requirements for deployment, and for end-users.

The other issue is with Python (and dependency) version changes. While Lazuli v2 works fine for Python v3.6+, it's starting to see its age. Users that require older versions of Python are welcome to continue using Lazuli v2, but we're going to target Python v3.10 and up for Lazuli v3. This will allow us to make use of newer versions of our dependencies, and reduce the number of vulnerabilities moving forward.

Alongside these technical changes, we will also add type hints as well as improve on API documentation, to enhance the usability of Lazuli downstream. As part of the improvements to API documentation, we will not only ensure that docstrings comply with Google style, but also be automatically generated into a set of API docs on GitHub Pages as part of a new CI process. We will also refactor the Job-ID-to-common-name mapping system to be compliant and/or compatible with SpiritSuite's existing YAML-based solution(s).

Information for Users

No action is needed other than re-creation of the virtual environment. We highly recommend starting from a new virtual environment, due to the numerous changes in dependencies.

Simply delete your current venv folder, and re-create the virtual environment (see wiki for how to generate venv).

Note: Existing documentation for Lazuli v2 will remain on the gh-pages branch, for posterity.

Information for Contributors

Here are the tasks to be completed in the migration process:

Clean up virtual environment

• Remove distribution- and documentation-related dependencies from end-user's requirements file
• Run unit tests for BOTH virtual environments (to check for dependency-related issues)
• Create separate requirements file for contributors
• Remove unit test-related dependencies from end-user's requirements file

PEP 621 Compliance

• Require version >=61.0 for setuptools
• Migrate metadata to pyproject.toml

Clean up docs

• Change documentation library from Portray to pdoc
• Reformat docstrings in __init__ to follow Google style
• Reformat docstrings in account to follow Google style
• Reformat docstrings in character to follow Google style
• Reformat docstrings in database to follow Google style
• Reformat docstrings in inventory to follow Google style
• Reformat docstrings in utility to follow Google style

Compliance with SpiritSuite

• Replace the JOBS dictionary in __init__ module with the SpiritSuite YAML document
• Run unit tests for BOTH virtual environments (to check for YAML-related issues)

Improve IDE hints downstream/PEP 484 Compliance

• Add type hints for account
• Add type hints for character
• Add type hints for database
• Add type hints for inventory
• Add type hints for utility

Modify setup script for contributors

• Re-export requirements.txt and contributor_requirements.txt
• Add switch to choose between use and test and distribute

Migrate API docs auto-generation from portray to pdocs

• Rewrite build script
• Test publish to testpypi
  • This is to make sure the change in metadata handling (see: PEP 621 Compliance above) does not break the package
• Test install testpypi package
• Merge pdoc branch with main
• Test CI workflow on GitHub

Distribute

• Publish to PyPi