Add ci:persist-prior-workflows tag to allow prior workflow to run unt…

…il completion (#1449)

.github/CI_README.md

@@ -41,6 +41,12 @@ Currently, the only way to access any running instance that is created from the
 
 If the instance is stopped, then you must request a AWS IAM user account from the FireSim developers to access the EC2 console and restart the instance.
 
+Prior CI Jobs for Pull Requests
+------------------------------
+
+The default behavior is that a new commit to a PR will cancel any existing workflows that are still running. This is to save resources and is done by the `cancel-prior-workflows` job. If you wish to 
+allow all prior workflows to keep running, add the `ci:persist-prior-workflows` tag to your PR. Please use this tag sparingly, and with caution.
+
 GitHub Secrets
 --------------
 * **AWS_ACCESS_KEY_ID**: Passed to `aws configure` on CI containers + manager instances

.github/workflows/firesim-run-tests.yml

@@ -28,13 +28,15 @@ env:
   LANGUAGE: "en_US:en"
   LC_ALL: "en_US.UTF-8"
   CI_LABEL_DEBUG: ${{ contains(github.event.pull_request.labels.*.name, 'ci:debug') }}
+  CI_LABEL_PERSIST: ${{ contains(github.event.pull_request.labels.*.name, 'ci:persist-prior-workflows') }}
 
 jobs:
   cancel-prior-workflows:
     name: cancel-prior-workflows
     runs-on: ubuntu-20.04
     steps:
       - name: Cancel previous workflow runs
+        if: ${{ (env.CI_LABEL_PERSIST != 'true') }}
         uses: styfle/cancel-workflow-action@0.11.0
         with:
           access_token: ${{ github.token }}