Release v1 API

[phaer]

We believe it's about time to move from the old, experimental API (legacy) to the new one, sketched out in #468 and (partially) implemented in the v1/ directory in the main branch .

The new API is based on drv-parts and does not require flake-enabled nix, but ships examples based on flake parts for those of us who like flakes.

This issue aims to provide an overview of the work we still need to do before deprecating the existing API and moving v1 to the top level of this repo. We've discussed this list in the last weekly call on 2023-04-18, you are welcome to provide feedback either here or in next weeks call (2024-04-25 14h Utc).

Documentation

• update readme
• update manual
• Introduction
• Guides
• Subsystems (those which we'll keep, see below)
• Concepts / API: Explain drv-parts API, lock & eval-cache, nixpkgs-overrides modules
• Contributing
• v1 -> Move to Concepts
• Community Overrides: Implement, document, initialize

Support Policies

• Define a policy for release
  We want git tags per release and a changelog announcing breaking changes with categories for the general API and specific subsystems
• Write a paragraph or table per subsystem module to describe features, limitations and expected stability of the subsystem.

Update Ecosystems

While there's a proof-of-concept wrapper which wraps the old API in terms of the new one (dream2nix-legacy module), this would still mean that we need to keep the old API inside our tree. We've decided to instead do a proper port of some subsystems to the new API.
Due to the different completeness, community interest, and maintenance burden of each subsystem, we are probably going to drop some of them:

• nodejs: will be ported, @DavHau
• rust: will be ported, @DavHau
• python: examples will be ported, old API dropped, see init fetchPipMetadata / pip #504 @phaer

The following subsystems would be dropped. Please contact us here, on matrix or in a weekly call if you are interested into maintaining one of them - we'll happily provide mentoring and support.

• debian
• go
• haskell
• php (@aszenz is interested)
• racket

Checklist

• remove legacy code, outside v1
• generalize nixpkgs-overrides to work without python modules

Testing

• We'd like to have a more flexible test runner, allowing selection of tests per name or tag, like pytest cli,
  in order to split tests in "slow" and "fast" sets. The idea is to make it easier and faster to run unit tests which primarily checks
  api assertions and evaluation, while still being able to run heavier integration tests in CI and locally.
• We also would like to automatically test code examples included in our documentation.

[DavHau]

Some more tasks that should be completed:

• Community Overrides: Implement, document, initialize

[phaer]

some quick notes copied from our chat.

• Provide Code examples, which should ideally be testable (mdsh, tesh, etc)
• Maybe provide relationship graphs in mermaidjs format or similar.
• Generate Reference Documentation for drv-part modules:

1. prose in markdown, describing the purpose of the module, its features, limitations and whether its considered experimental or stable, maintainers, etc
2. List of module-specific options with per-option description and links to relevant examples in tutorial.
3. list of links to module documentation of dependencies, to e.g. get from builldPythonPackage to mkDerivation options.

[aszenz]

I would like to help in porting the php subsystem to the V1 api.

[DavHau]

@aszenz your help would be much appreciated. I opened an issue for the php migration: #550

Feel free to post any questions you have on that issue.

[yusdacra]

If no one is porting the rust subsystem yet, can I pick it up? Some pointers would be helpful since I haven't looked at how all of the code is organized right now.

[phaer]

@yusdacra Great! As far as I know, there's no one working on the rust subsystem yet and contributions are more than welcome.

Some pointers would be helpful since I haven't looked at how all of the code is organized right now.

Any specific questions? I think David's recent presentation might currently be the best conceptual overview, if you haven't seen it yet. Otherwise always happy to help here, on matrix or in our weekly jitsi calls, Tuesday at 14 UTC.

[DavHau]

If no one is porting the rust subsystem yet, can I pick it up? Some pointers would be helpful since I haven't looked at how all of the code is organized right now.

That would be great. I actually started working on it a while ago. I'll rebase my branch once I'm home, push it and ping you.

[DavHau]

I pushed my work here: #606

[DrRuhe]

I'd like to add a task that should be done as well regarding the cleanup of the API. Currently ./modules/flake-parts/all-modules.nix imports all modules and exports them again under dream2nix.modules.<kind>.<name>. This clutters the user-facing API. For example: ./modules/flake-parts/drv-parts-deprecated.nix simply adds a deprecation notice to anyone trying to use dream2nix.modules.drv-parts. But it is also itself exposed as dream2nix.modules.flake-parts.drv-parts-deprecated.