Referencing How SQLite Is Tested
Reliability of ART is achieved in part by thorough and careful automated testing. The range of testing covers each ART component, it's public API, code samples in the documentation, and supported versions (such as JDK and Clojure versions) as listed in vivid-art-facts.edn. ART is tested on each release of Clojure starting from the minimum supported version, and on the latest release of each Java LTS major version starting from the minimum supported Java version.
Defects reported against ART cannot be considered as resolved until automated tests express the defect and prove remediation, as feasible. These regression tests ensure that prior defects do not re-emerge in the future.
A VCS commit is considered releasable provided that all of its components satisfy the following criteria:
- Code quality assessment tools don't indicate any outstanding problems, within reason:
TODO
s, Clojure & Java library dependency warnings, reflection warnings, ancient, clj-kondo, nvd. - The documentation is synchronized with the code, including version numbers, and automated testing of all examples.
- The behavior of code samples as described in the documentation is confirmed via automated tests.
- Reasonably unified test suite shared between the tools
clj-art
et. al. - All automated tests pass throughout the matrix of supported versions of Clojure and JDKs.
- Test coverage from automated testing indicates a near-total or better coverage rate on each namespace.
- All dependencies are as up-to-date as reasonable, and NVD doesn't indicate serious problems.
- Copyright headers are up-to-date.
- Ensure the contents of the project metadata file vivid-art-facts.edn is correct.
bin/gen.sh
doesn't change any files. In other words, all generated files are up-to-date.- Update CHANGELOG.md to reflect the new version.
- Replace the
_Unreleased_
attribute with the actual date. - Ensure the entries genuinely reflect the nature of the changes in this release.
- Replace the
- Choose a specific VCS commit identifier as the release target.
- Ensure the release criteria are satisfied.
- All README.md files are synchronized in content.
- CI is building the release branch.
- Use
bin/deploy.sh
to deploy all projects to Clojars. - Tag the release and push the tag to upstream Git.
- Smoke test each downloadable deliverable.
- In the GitHub project, change the default branch to this new release.
- Confirm correctness of:
- All project URLs.
- Default branch in GitHub.
- Versions appearing in current documentation.
- CI build results.
- Announce the release to the Clojure community.
- (Vivid Inc. internal) In an env that doesn't have the new release in its
~/.m2
repo, update all project build dependencies to this release of ART and run their full builds on clean VCS checkouts to ensure correct operation of the new release obtained from Clojars.