Hydra developers

This page is for developers of Hydra itself. See Contributors.

Dependency tiers

Since Hydra generates much of its own source code, it is challenging to prevent circular dependencies. To make things easier, we categorize Hydra source code into several dependency tiers, and enforce the simple rule that code in a lower tier may not depend on code in a higher tier:

• tier-0 kernel modules: generated sources consisting mainly of type definitions, including hydra/core, hydra/graph, etc. These are not allowed to depend on primitive functions.
• tier-0 DSLs: language-specific DSLs, such as Hydra-Haskell's Types and Terms DSLs, which depend only on each other and on tier-0 kernel modules. The kernel modules are not allowed to depend on the DSLs.
• tier-1 kernel modules: generated code which is allowed to depend on (tier-1) primitive functions, as well as tier-0 kernel modules and DSLs (but not on tier-1 primitive functions)
• tier-1 primitive functions: primitive functions whose implementations are allowed to depend on tier-0 kernel modules and tier-0 DSLs
• tier-1 DSLs: language-specific DSLs which are allowed to depend on tier-1 kernel modules and primitive functions
• tier- 2 kernel modules: the rest of Hydra's generated kernel code
• tier-2 primitive functions: the rest of Hydra's built-in functions
• tier-3 kernel sources: these are the DSL-based definitions of tier-0, tier-1, tier-2, and tier-3 kernel code. For example, Hydra/Sources/Core.hs in Hydra-Haskell is the DSL-based definition of hydra/core, which is generated to produce the tier-0 Hydra/Core.hs module in Haskell, the corresponding hydra/core package in Java, etc.
• tier-4 non-kernel code: built-in language coders and utilities which are provided with an implementation of Hydra, but are not part of the Hydra kernel. For example, Hydra-Haskell provides a Java coder among many others, but this not part of the kernel; other Hydra implementations might not provide a Java coder.
• tier-5 code (aka third-party code): sources, generated code, and user-defined functions which are external to Hydra proper. Developers of third-party code may create their own dependency tiers within this one, but these are not specified by Hydra.

Releases

Hydra-Haskell release process

Find previous Hydra-Haskell releases on Hackage. You will need a Hackage account and appropriate permissions to publish new releases. This is the process I usually follow:

• Pre-release readiness checks
  • Make sure the entire project has been brought up to date for the release. Check open issues, update documentation, etc. as needed.
  • Upgrade dependencies if this hasn't been done since the last release, taking care not to impact downstream consumers of Hydra
    • Bump "resolver: lts-xxx" in hydra-haskell/stack.yaml to the latest LTS release of Stack, then compile and run tests. Corresponding changes to dependencies in hydra-haskell/package.yaml may be required. Bump the version of the base dependency in package.yaml to the latest version supported by Stack. Also consider bumping the versions of the other dependencies in package.yaml, and remove any dependencies which are no longer needed. Compile and run tests.
    • Repeat the above step in hydra-extensions. This is not necessary for the release, but it keeps the repository self-consistent.
  • Commit, review, and push to main any changes we made as part of the readiness checks above.
• Bump the version
  • Update the Hydra version in hydra-haskell/package.yaml. Also update the corresponding versions in hydra-java/build.sh and in hydra-extensions/package.yaml
  • Update CHANGELOG.md, adding release notes.
  • Commit the change using a message like "Bumping Hydra version to 0.x.y".
• Release to Hackage
  • Create the Hackage release package (on main HEAD) with cd hydra-haskell && stack sdist
  • Upload the package here, then periodically check on the status of the release.
• Tag the release
  • As soon as the release is confirmed to have succeeded, create a tag (e.g. git tag 0.x.y -m '0.x.y release' [hash] && git push --tags using the commit hash of main HEAD at the time of the release.

Hydra-Java release process

In most cases, Hydra-Java releases should be coordinated with Hydra-Haskell releases, the exception being bug fix or minor feature releases which concern only Hydra-Haskell or only Hydra-Java. The following additional steps are required for Hydra-Java releases:

• Update JavaDocs
  • After bumping the release version (in hydra-java, hydra-haskell, and hydra-extensions), re-generate JavaDocs with rm -rf hydra-java/javadoc && cd hydra-java && ./bin/update-javadoc.sh. Now add files, commit, and push to main as needed.
  • Check out the remote docs branch of Hydra, then git pull origin main and git push origin docs. This will trigger the generation of the online JavaDocs. You should see notifications in the#github channel on the LambdaGraph Discord server. The generated Hydra JavaDocs can be found here.
• Release to Nexus
  • Once master HEAD is ready for the release, cd hydra-java && ./gradlew publishToSonatype. Then,
  • Log in to oss.sonatype.org, assuming you have the credentials to publish Hydra-Java
  • Find the staging repository in the left sidebar under "Staging Repositories", then click to "close" it (do NOT "drop"). Supply a message like "Hydra-Java 0.x.y release".
  • Wait several minutes, then click "release" when that option is available. Again enter a message like "Hydra 0.x.y release".
  • It will take a variable amount of time for the release to propagate, sometimes up to an hour. When the release is available, you will see it here as net.fortytwo.hydra.
• Tag the release
  • If you haven't already done so as part of the Hydra-Haskell release process (see above).