Skip to content

How to create a Heimdall2 release

Jump to bottom
DMedina6 edited this page Apr 2, 2024 · 87 revisions

Creating a Release

1. Make sure your local Heimdall repo is up-to-date, install dependencies, and check latest commits.
  1. Ensure you have the most recent commits to Heimdall with git pull.
  2. Do not delete or modify the yarn.lock file, it should be identical to the one pulled from main.
  3. Run yarn install --registry https://registry.npmjs.org. Note that this may take a while and could require you to fix broken dependencies.
    • We're using the --registry flag because using the regular yarn registry caused OpenSSL errors.
    • If you receive an openssl error, try referencing guidance in this closed issue
    • Make sure that the yarn install completes successfully.
    • If the cypress command fails (exit code 1) turn cypress checking off by setting the CYPRESS_INSTALL_BINARY variable on the terminal that the yarn command is being executed:
      • Windows: set CYPRESS_INSTALL_BINARY=0
      • Mac: export CYPRESS_INSTALL_BINARY=0
  4. Run yarn frontend build. This will build the latest version of the Tailwind css for inclusion within Heimdall / the HTML export. Make sure to commit the changes! (changes are commit directly to master)
  5. Check the repository to identify if there has been a commit in the frontend since the last release. If there were none (ex: the latest commit in frontend is v2.X.X), make a new commit in frontend.
    • Explanation: At the moment, a commit in the Heimdall frontend directory is required so that Heimdall-lite has a version bump, allowing the Heimdall-lite version number to match the overall repository version number. If the version numbers do not match, a warning message reading 'Heimdall is out of date' will display on Heimdall-lite.
    • What to Change: If there is no new commit in the Heimdall frontend, make a change to a file in a file within apps/frontend. For example, you could add a real or trivial change to a markdown file, .gitignore file, etc.
    • Note: You can verify this step when running the lerna command in step 3, where you will see what will be released. If the frontend (@mitre/heimdall-lite) is not one of those items, return to the instructions above (step 1.4).
2. Update the necessary files
  • VERSION (with the new version number)
  • CHANGELOG (Copy and paste from the draft release markdown) Note: There might not be a draft release until a human-origin commit has been merged (i.e. the dependabot PRs being merged will not trigger the draft release workflow)
  • Commit and push the changes. You can commit directly to master.
3. Create a release tag using lerna
  • Run npx lerna version v2.x.x
  • You will be asked - ? Are you sure you want to create these versions? type y for Yes
    • The results from the lerna command should like something like below. If @mitre/heimdall-lite is not shown, go back to step 1.
lerna notice cli v5.6.2
lerna info current version 2.8.2
lerna info Looking for changed packages since v2.8.2

Changes:
 - heimdall-server: 2.8.1 => 2.8.3 (private)
 - @mitre/heimdall-lite: 2.8.2 => 2.8.3
 - @heimdall/cypress-tests: 2.8.1 => 2.8.3 (private)
  • To check if the step finished successfully go to the Releases page and ensure that a Tag with the same name you created is there (example v2.x.x).
  • Note: Major version increments are a significant decision - please discuss internally before considering changing the major version.
  • Consider prioritizing the actions required for the release by canceling the Dependabot PRs. (SEE BELOW). Heimdall uses Dependabot to keep dependencies up-to-date. Dependabot usually has a large number of PRs open, all of which will try to update when a push is made to the master branch.
4. Verify that the container builds successfully by running Heimdall locally
  1. Verify the containers push successfully to Docker Hub by looking at the workflow status of Pushing Heimdall Lite to Docker Hub and Push Heimdall Server to Docker Hub. They should be triggered by the commit generated by Lerna.
  • Note: This may require clearing cache or running incognito

Build Status Indicators

NOTE: If this is your first time running Heimdall via the docker image, consider referencing the README.

  1. Ensure you have setup your docker secrets with ./setup-docker-env.sh
  2. Build (or pull) the image for testing.
    • If you want to build the image locally:
      • Edit your docker-compose.yml and change image: mitre/heimdall2:release-latest to build: .
      • Run docker compose build.
      • NOTE: Building this on a company network may not work given that building this image requires access to the internet and the image will not have company specific certificates.
    • If you want to pull the latest version of Heimdall from DockerHub
      • Edit your docker-compose.yml and change image: mitre/heimdall2:release-latest to image: mitre/heimdall2:latest. Note that the latest tag for Heimdall in DockerHub reflects the current state of the master branch (make sure the action that updates DockerHub has finished running before you try to pull this image), while release-latest reflects the last tagged release.
      • Run docker compose pull to ensure you have the latest container images specified in your docker-compose.yml;
  3. Remove any existing database content: rm -rf data/ on window use rmdir /s /q data
  4. Ensure the container starts successfully.
    • Run docker compose up.
    • Access Heimdall in your browser at https://localhost:443 Note: You will get a self-signed certificate warning from your browser if you did not provide your own; this is not a problem for testing purposes.
5. Validate and promote the Heroku deployment

  1. It can take some time to automatically push out a new deployment. Verify that mitre-heimdall-staging has been deployed by checking the version number. If it has been automatically deployed, move to step 5.2. If it has not updated, you can manually deploy it with the instructions below (5.1).

    • To manually deploy on Heroku:
    • Login to Heroku:
    • Click the mitre-saf workspace if it is not already selected Heroku Login
    • Click on drop down menu then "Deploy a branch..." Heroku Deploy
    • Make sure that master is selected
    • You may be prompted by GitHub to login
    • Click "Deploy"Deploy

  2. Verify the staging environment before promoting to production by validating the functionality below. If there are issues, address them with the team prior to moving forward with the release.

    • Uploading an Evaluation and Profile
    • Compare View
    • Filtering
    • Database Storage
    • OAuth Login
    • Results Table / Marking Results as Viewed
    • Exports
    • Tags (note: you can change tags from the Load modal and from inside the results view itself in the top left of the screen; try both)
    • Groups
    • API keys

  3. Promote mitre-heimdall-staging to production in Heroku

    • Note: If you receive an error rather than the picture below, you may not have the proper authorization. In this case, reach out to the team to request proper authorization. Screenshot 2023-06-13 at 9 20 43 AM
6. Update and publish the release.
  • Associate the release with the tag created by Lerna.
  • Remove "## New Features" header if it's there
  • Add "## What's New" in the description if there's anything besides dependabot PRs
  • Update the tag by using the dropdown menu on the draft release page. At first, it will be populated with the string untagged followed by a long hash. Click the dropdown, delete the existing tag string, and pick the correct version from the list that should appear.
Screen Shot 2023-01-03 at 4 01 02 PM
  • Click Publish Release
    • Note: You may need to cancel extraneous workflows again. (SEE BELOW)

PublishRelease

7. Validate that the workflows associated with the release succeeded. image
  • Heimdall Lite and Heimdall Server should both have successfully been pushed to the appropriate repositories (Dockerhub, Ironbank, hopefully Github Container Registry soon)
  • The 'Deploy to Github Pages' workflow completing triggers the 'pages build and deployment' workflow, which is what actually updates heimdall-lite.mitre.org (see #8)
  • The Github Packages/NPM workflow may fail. This is expected because the overall workflow attempts to publish individual components (Heimdall Lite, InSpecJS, HDF Converters) and the overall workflow will fail if one of the components has not been updated in this release. Click on the workflow that ran to confirm the components that were updated this release were actually published.
8. Validate heimdall-lite.mitre.org works
  • Go to heimdall-lite.mitre.org.
  • Hard refresh the website to get the latest (you may have to wait several minutes for the Heroku promotion to be in effect).
  • Confirm that there is no warning message about the version on the screen.
9. *Recommended*: Update and Release the SAF CLI
  • Wait for the npm workflows to finish before releasing the SAF-CLI release. Monitor Push-To-NPM Action to validate completion. Note: Any components of the monorepo that do not have a new version (ex: InSpecJS, HDF Converters) may appropriately fail to push to npm because they do not have an updated version.
  • Follow the SAF CLI release instructions
  • Note: This action may fail if HDF-Converters and/or InspectJS does not have a version update. Refer to the results of the lerna command previously ran

Cancelling workflow runs

Dependabot's PRs are currently set to have their CI/CD workflows triggered on every merge into master. Use the following script to cancel all of the dependabot PR based workflows.

Script based on this StackOverflow answer: https://stackoverflow.com/a/60766849/645647

token=YOUR_GITHUB_PERSONAL_ACCESS_TOKEN # needs the 'repo' scope
repo=your_user/your_repo # mitre/heimdall2

# get all the workflow run ids -> https://github.com/mitre/heimdall2/actions/runs/{VERY_LONG_ID_NUMBER}
# might need to add in parameters to increase page size and/or a forloop to iterate over pages OR use the 'gh' tool which seems like it returns all results at once instead of paginating them
# might need to adjust the filter if it's not precisely cancelling any and all of dependabot's PRs' workflows
ids=$(curl -s -H "Authorization: token $token" \
     https://api.github.com/repos/$repo/actions/runs?per_page=100 | \
     jq '.workflow_runs[] | select(([.status] | inside(["in_progress", "queued"])) and (.head_branch | startswith("dependabot/"))) | .id')
# set ids to be a positional value that the for-loop iterates over
set -- $ids
# can't batch cancel workflows so have to iterate
for i; do curl \
     -H "Authorization: token $token" \
     -X POST "https://api.github.com/repos/$repo/actions/runs/$i/cancel"; done
Clone this wiki locally