Skip to content

Commit

Permalink
Support caching for mono repos and repositories with complex structure
Browse files Browse the repository at this point in the history
  • Loading branch information
MaksimZhukov committed Aug 2, 2021
1 parent aa759c6 commit 9f31dbb
Show file tree
Hide file tree
Showing 7 changed files with 727 additions and 674 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/e2e-cache.yml
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,8 @@ jobs:
node-yarn2-depencies-caching:
name: Test yarn 2 (Node ${{ matrix.node-version}}, ${{ matrix.os }})
runs-on: ${{ matrix.os }}
env:
YARN_ENABLE_IMMUTABLE_INSTALLS: false
strategy:
fail-fast: false
matrix:
Expand Down
13 changes: 7 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ nvm lts syntax: `lts/erbium`, `lts/fermium`, `lts/*`

### Caching packages dependencies

The action has a built-in functionality for caching and restoring npm/yarn dependencies. Supported package managers are `npm`, `yarn`, `pnpm`. The `cache` input is optional, and caching is turned off by default.
The action has a built-in functionality for caching and restoring npm/yarn dependencies. Supported package managers are `npm`, `yarn`, `pnpm`. The `cache` input is optional, and caching is turned off by default. By default, the action searches for the dependency file in the project root and uses its hash as a part of cache key. Use `cache-dependency-path` to specify custom dependency file path. The field accepts wildcards or an array of files to be cached.

**Caching npm dependencies:**
```yaml
Expand Down Expand Up @@ -90,7 +90,7 @@ steps:
- run: pnpm test
```

> At the moment, only `lock` files in the project root are supported.
For more examlpes of caching, please see the [Advanced usage](docs/advanced-usage.md#caching-packages-dependencies) guide.

### Matrix Testing:
```yaml
Expand All @@ -114,10 +114,11 @@ jobs:

1. [Check latest version](docs/advanced-usage.md#check-latest-version)
2. [Using different architectures](docs/advanced-usage.md#architecture)
3. [Using multiple operating systems and architectures](docs/advanced-usage.md#multiple-operating-systems-and-architectures)
4. [Publishing to npmjs and GPR with npm](docs/advanced-usage.md#publish-to-npmjs-and-gpr-with-npm)
5. [Publishing to npmjs and GPR with yarn](docs/advanced-usage.md#publish-to-npmjs-and-gpr-with-yarn)
6. [Using private packages](docs/advanced-usage.md#use-private-packages)
3. [Caching packages dependencies](docs/advanced-usage.md#caching-packages-dependencies)
4. [Using multiple operating systems and architectures](docs/advanced-usage.md#multiple-operating-systems-and-architectures)
5. [Publishing to npmjs and GPR with npm](docs/advanced-usage.md#publish-to-npmjs-and-gpr-with-npm)
6. [Publishing to npmjs and GPR with yarn](docs/advanced-usage.md#publish-to-npmjs-and-gpr-with-yarn)
7. [Using private packages](docs/advanced-usage.md#use-private-packages)

# License

Expand Down
2 changes: 2 additions & 0 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ inputs:
default: ${{ github.token }}
cache:
description: 'Used to specify a package manager for caching in the default directory. Supported values: npm, yarn, pnpm'
cache-dependency-path:
description: 'Used to specify path to a dependencies file: package-lock.json, yarn.lock, etc. Supports wildcards or an array of file names.'
# TODO: add input to control forcing to pull from cloud or dist.
# escape valve for someone having issues or needing the absolute latest which isn't cached yet
# Deprecated option, do not use. Will not be supported after October 1, 2019
Expand Down
1,324 changes: 665 additions & 659 deletions dist/setup/index.js

Large diffs are not rendered by default.

42 changes: 36 additions & 6 deletions docs/advanced-usage.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Advanced usage

### Check latest version:
### Check latest version

The `check-latest` flag defaults to `false`. When set to `false`, the action will first check the local cache for a semver match. If unable to find a specific version in the cache, the action will attempt to download a version of Node.js. It will pull LTS versions from [node-versions releases](https://github.com/actions/node-versions/releases) and on miss or failure will fall back to the previous behavior of downloading directly from [node dist](https://nodejs.org/dist/). Use the default or set `check-latest` to `false` if you prefer stability and if you want to ensure a specific version of Node.js is always used.

Expand All @@ -19,7 +19,7 @@ steps:
- run: npm test
```

### Architecture:
### Architecture

You can use any of the [supported operating systems](https://docs.github.com/en/actions/reference/virtual-environments-for-github-hosted-runners), and the compatible `architecture` can be selected using `architecture`. Values are `x86`, `x64`, `arm64`, `armv6l`, `armv7l`, `ppc64le`, `s390x` (not all of the architectures are available on all platforms).

Expand All @@ -39,7 +39,37 @@ jobs:
- run: npm test
```

### Multiple Operating Systems and Architectures:
### Caching packages dependencies

**Using wildcard patterns to cache dependencies**
```yaml
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: '14'
cache: 'npm'
cache-dependency-path: '**/package-lock.json'
- run: npm install
- run: npm test
```

**Using a list of file paths to cache dependencies**
```yaml
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: '14'
cache: 'npm'
cache-dependency-path: |
server/app/package-lock.json
frontend/app/package-lock.json
- run: npm install
- run: npm test
```

### Multiple Operating Systems and Architectures

```yaml
jobs:
Expand Down Expand Up @@ -74,7 +104,7 @@ jobs:
- run: npm test
```

### Publish to npmjs and GPR with npm:
### Publish to npmjs and GPR with npm
```yaml
steps:
- uses: actions/checkout@v2
Expand All @@ -94,7 +124,7 @@ steps:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
```

### Publish to npmjs and GPR with yarn:
### Publish to npmjs and GPR with yarn
```yaml
steps:
- uses: actions/checkout@v2
Expand All @@ -114,7 +144,7 @@ steps:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
```

### Use private packages:
### Use private packages
```yaml
steps:
- uses: actions/checkout@v2
Expand Down
15 changes: 13 additions & 2 deletions src/cache-restore.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,10 @@ import {
PackageManagerInfo
} from './cache-utils';

export const restoreCache = async (packageManager: string) => {
export const restoreCache = async (
packageManager: string,
cacheDependencyPath?: string
) => {
const packageManagerInfo = await getPackageManagerInfo(packageManager);
if (!packageManagerInfo) {
throw new Error(`Caching for '${packageManager}' is not supported`);
Expand All @@ -22,9 +25,17 @@ export const restoreCache = async (packageManager: string) => {
packageManagerInfo,
packageManager
);
const lockFilePath = findLockFile(packageManagerInfo);
const lockFilePath = cacheDependencyPath
? cacheDependencyPath
: findLockFile(packageManagerInfo);
const fileHash = await glob.hashFiles(lockFilePath);

if (!fileHash) {
throw new Error(
'Some specified paths were not resolved, unable to cache dependencies.'
);
}

const primaryKey = `node-cache-${platform}-${packageManager}-${fileHash}`;
core.debug(`primary key is ${primaryKey}`);

Expand Down
3 changes: 2 additions & 1 deletion src/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,8 @@ export async function run() {
if (isGhes()) {
throw new Error('Caching is not supported on GHES');
}
await restoreCache(cache);
const cacheDependencyPath = core.getInput('cache-dependency-path');
await restoreCache(cache, cacheDependencyPath);
}

const matchersPath = path.join(__dirname, '../..', '.github');
Expand Down

0 comments on commit 9f31dbb

Please sign in to comment.