Caching #515
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Caching | |
| # Docs on the workflow: | |
| # 1. GitHub cache scoping goes: | |
| # [main branch -> PR main branch -> PR. (essentially in a tree fashion)](https://web.archive.org/web/20211125171853/https://docs.github.com/en/actions/advanced-guides/caching-dependencies-to-speed-up-workflows#restrictions-for-accessing-a-cache). | |
| # Building & keeping caches on `master` allows | |
| # to share the main project state cache be shared to the whole tree. | |
| # 2. GitHub has a [default 10G cache pool limit](https://web.archive.org/web/20211125171853/https://docs.github.com/en/actions/advanced-guides/caching-dependencies-to-speed-up-workflows#usage-limits-and-eviction-policy) per repo. | |
| # HLS is a big monorepo codebase, which means easy cache pool | |
| # invalidation & exhaustion because of the pool limit. | |
| # To keep caches useful - the main state of the main branch should remain | |
| # & so keep caching in the repo well below the limit. | |
| # that means preferring main branch to the PR caches | |
| # (especially internal branch ones), since PRs from internal branches - | |
| # count into the repo 10G pool, while that cache gets used only inside of the PR, | |
| # while exhausting the pool would bork cache for the rest of the community. | |
| # That is a short story why `dist-newstyle` (especially full) currently is not | |
| # includded into `master` or PR caches. | |
| defaults: | |
| run: | |
| shell: bash | |
| # See: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#concurrency. | |
| concurrency: | |
| group: ${{ github.head_ref }}-${{ github.workflow }} | |
| cancel-in-progress: true | |
| on: | |
| workflow_dispatch: | |
| push: | |
| branches: | |
| - master | |
| schedule: | |
| # Refresh snapshot every (02+8*x):25 UTC | |
| # When cache is present it is a light check workflow with early termination. | |
| # When primary cache is not hit - runs the cache generation. | |
| # Why: GitHub repo has 10G pool & on overflow GitHub removes caches in FIFO manner. | |
| # When internal branche PRs save into the same pool - | |
| # their cache is accessible only inside of the scope of the PR. | |
| # If main cache is forced out - there are no cache shared between PRs, | |
| # which implies all PRs would start to create & save their cache. | |
| # Reinstitution of the main chache puts it back into FIFO | |
| # & so it gets shared across all PRs. | |
| - cron: "25 2/8 * * *" | |
| # Not using a explicit target to build the top level haskell-language-server package | |
| # which make build the rest of subpackages *libs* (but shake-bench) | |
| env: | |
| cabalBuild: "v2-build --keep-going" | |
| jobs: | |
| pre_job: | |
| runs-on: ubuntu-latest | |
| outputs: | |
| ghcs: ${{ steps.ghcs.outputs.ghcs }} | |
| should_skip: ${{ steps.skip_check.outputs.should_skip }} | |
| steps: | |
| # Need the repo checked out in order to read the file | |
| - uses: actions/checkout@v3 | |
| - id: ghcs | |
| run: echo "ghcs=$(cat ./.github/workflows/supported-ghc-versions.json)" >> $GITHUB_OUTPUT | |
| - id: skip_check | |
| uses: fkirc/skip-duplicate-actions@v5.3.1 | |
| with: | |
| cancel_others: false | |
| paths_ignore: '["**/docs/**" | |
| , "**.md" | |
| , "**/LICENSE" | |
| , "**.nix" | |
| , "flake.lock" | |
| , "**/README.md" | |
| , "FUNDING.yml" | |
| , ".circleci/**" | |
| , "**/stack*.yaml" | |
| , ".gitlab-ci.yaml" | |
| , ".gitlab/**" | |
| ]' | |
| caching: | |
| if: needs.pre_job.outputs.should_skip != 'true' | |
| needs: pre_job | |
| runs-on: ${{ matrix.os }} | |
| strategy: | |
| fail-fast: false | |
| matrix: | |
| ghc: ${{ fromJSON(needs.pre_job.outputs.ghcs) }} | |
| os: | |
| - ubuntu-latest | |
| - macOS-latest | |
| - windows-latest | |
| exclude: | |
| # We disable this this combo in test.yml due to long path issues, so we also need to disable it here | |
| - os: windows-latest | |
| ghc: "9.2" | |
| steps: | |
| - uses: actions/checkout@v3 | |
| - uses: ./.github/actions/setup-build | |
| with: | |
| ghc: ${{ matrix.ghc }} | |
| os: ${{ runner.os }} | |
| # Download sources for feeding build sources cache | |
| # Fetching from github cache is faster than doing it from hackage | |
| # Sources does not change per ghc and ghc version son only doing it | |
| # for one matrix job (it is arbitrary) | |
| - if: steps.compiled-deps.outputs.cache-hit != 'true' && runner.os == 'Linux' && matrix.ghc == '9.2' | |
| name: Download sources | |
| run: | | |
| cabal $cabalBuild --only-download --enable-benchmarks --enable-tests | |
| # This build agenda is not to have successful code but produce cache as much as possible | |
| - if: steps.compiled-deps.outputs.cache-hit != 'true' | |
| name: Build haskell-language-server | |
| run: | | |
| # repeating builds to workaround segfaults in windows and ghc-8.8.4 | |
| cabal $cabalBuild || cabal $cabalBuild || cabal $cabalBuild | |
| # We build ghcide with benchs and test enabled to include its dependencies in the cache | |
| # (including shake-bench) | |
| # Only for the same ghc and os used in the bench workflow, so we save cache space | |
| - if: steps.compiled-deps.outputs.cache-hit != 'true' && runner.os == 'Linux' && matrix.ghc == '9.2' | |
| name: Build ghcide benchmark | |
| run: | | |
| cabal $cabalBuild ghcide --enable-benchmarks --enable-tests |