Skip to content

Commit

Permalink
feat: check integrity with Deno lockfiles (#4)
Browse files Browse the repository at this point in the history
* feat: the one with integrity checks

* docs: add integrity PR link to changelog

* chore: set config defaults at the root
  • Loading branch information
jeroenptrs committed Aug 7, 2020
1 parent b5a7f06 commit c92e93a
Show file tree
Hide file tree
Showing 14 changed files with 218 additions and 63 deletions.
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## Unreleased

### Added

- Source map support in [denofn/denopack/pull/2](https://github.com/denofn/denopack/pull/2)
- Lockfile integrity checking in [denofn/denopack/pull/4](https://github.com/denofn/denopack/pull/4)

### Changed

- Hooks now take one configuration object as parameter where needed in [denofn/denopack/pull/4](https://github.com/denofn/denopack/pull/4)
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ There is absolutely nothing wrong with `deno bundle`, but in its current state i
- [x] [Tree shaking](https://rollupjs.org/guide/en/#tree-shaking) comes built-in with Rollup
- [x] Minification by the usage of the [Terser plugin](./plugin/terserTransform)
- [x] Source Maps (should also come built-in with Rollup, [coming soon](https://github.com/denofn/denopack/pull/2))
- [x] Lock file support, checking checksums from the lockfile against loaded code
- [ ] File watching (pretty sure this can be implemented, coming soon)
- [ ] Lock file support, checking checksums from the lockfile against loaded code

More to come, also see the `deno bundle` roadmap/wishlist over at [denoland/deno/issues/4549](https://github.com/denoland/deno/issues/4549)

Expand Down Expand Up @@ -219,6 +219,7 @@ Contributing a hook follows the following conventions:
- Hooks are stored inside of [hooks.ts](./plugin/hooks.ts)
- Hooks are always functions and always return an array of plugins
- Using plugins that accept configuration options in hooks should always be allowed to pass config down to that plugin
- Configuration options are stored in one object containing all configuration options, see [hooks.ts](./plugin/hooks.ts) for examples

## Acknowledgements

Expand Down
25 changes: 22 additions & 3 deletions plugin/cacheLoader/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
# denopack/plugin/cacheLoader

- [optional] checks if cached file exists and skips loading if it doesn't
- [optional] checks integrity of cached file against a lockfile
- returns local cached file location

## Options

- `lazy [boolean]`: defaults to false. If active, this will skip checking whether the file actually exists locally and will lazily return the assumed path.
- `lazy [boolean]`: defaults to false. If active, this will skip checking whether the file actually exists locally and will lazily return the assumed path
- `cacheOnly [boolean]`: throws if an external dependency is not found inside the cache
- `lockFile [string]`: path to a lockfile (for example: `lock.json`). Throws if the integrity of the loaded cache file does not match the integrity in the lockfile

## Required flags

Expand All @@ -17,11 +20,27 @@
Put this before pluginFileLoader

```ts
import { pluginRootResolver } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/rootResolver/mod.ts";
import { pluginImportResolver } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/importResolver/mod.ts";
import { pluginCacheLoader } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/cacheLoader/mod.ts";
import { pluginFileLoader } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/filLoader/mod.ts";

export default {
plugins: [pluginRootResolver(), pluginCacheLoader(), pluginFileLoader()],
plugins: [pluginImportResolver(), pluginCacheLoader(), pluginFileLoader()],
};
```

### Strict integrity checks

```ts
import { pluginImportResolver } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/importResolver/mod.ts";
import { pluginCacheLoader } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/cacheLoader/mod.ts";
import { pluginFileLoader } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/filLoader/mod.ts";

export default {
plugins: [
pluginImportResolver(),
pluginCacheLoader({ lockFile: "lock.json", cacheOnly: true }),
pluginFileLoader({ lockFile: "lock.json" }),
],
};
```
38 changes: 27 additions & 11 deletions plugin/cacheLoader/mod.ts
Original file line number Diff line number Diff line change
@@ -1,26 +1,42 @@
import { Plugin } from "../../deps.ts";
import { path, Plugin } from "../../deps.ts";
import { buildCacheUrl } from "../../util/buildCacheUrl.ts";
import { checkIntegrity } from "../../util/checkIntegrity.ts";
import { isFile } from "../../util/isFile.ts";
import { isHttpUrl } from "../../util/isHttpUrl.ts";

type Opts = {
export type Opts = {
lazy?: boolean;
cacheOnly?: boolean;
lockFile?: string;
};

async function cacheLoader(
id: string,
opts: Opts,
lockFile: Record<string, string> | undefined
): Promise<string | null> {
const cacheUrl = buildCacheUrl(id);
const isFileInCache = await isFile(cacheUrl);

if (opts.cacheOnly && !isFileInCache) throw new Error(`Cannot find ${id} in Deno cache`);
if (!opts.lazy && !isFileInCache) return null;

const code = await Deno.readTextFile(cacheUrl);
if (lockFile) checkIntegrity(lockFile, id, code);

return code;
}

export function pluginCacheLoader(opts: Opts = {}): Plugin {
const lockFile: Record<string, string> | undefined = opts.lockFile
? JSON.parse(Deno.readTextFileSync(path.resolve(opts.lockFile)))
: undefined;

return {
name: "denopack-plugin-cacheLoader",
async load(id) {
if (!isHttpUrl(id)) return null;

const cacheUrl = buildCacheUrl(id);

if (opts.lazy || (await isFile(cacheUrl))) {
const code = await Deno.readTextFile(cacheUrl);
return code;
}

return null;
return cacheLoader(id, opts, lockFile);
},
};
}
4 changes: 2 additions & 2 deletions plugin/cacheResolver/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,11 @@
## Usage

```ts
import { pluginRootResolver } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/rootResolver/mod.ts";
import { pluginImportResolver } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/importResolver/mod.ts";
import { pluginChainResolver } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/chainResolver/mod.ts";
import { pluginCacheResolver } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/cacheResolver/mod.ts";

export default {
plugins: [pluginChainResolver(pluginRootResolver(), pluginCacheResolver())],
plugins: [pluginChainResolver(pluginImportResolver(), pluginCacheResolver())],
};
```
4 changes: 2 additions & 2 deletions plugin/chainResolver/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@ If files are resolved, Rollup will skip all other resolvers for that file. Somet
## Usage

```ts
import { pluginRootResolver } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/rootResolver/mod.ts";
import { pluginImportResolver } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/importResolver/mod.ts";
import { pluginChainResolver } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/chainResolver/mod.ts";
import { pluginCacheResolver } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/cacheResolver/mod.ts";

export default {
plugins: [pluginChainResolver(pluginRootResolver(), pluginCacheResolver())],
plugins: [pluginChainResolver(pluginImportResolver(), pluginCacheResolver())],
};
```
27 changes: 24 additions & 3 deletions plugin/fileLoader/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,40 @@

- Loads locally resolved files
- Fetches resolved urls
- [optional] checks integrity of files loaded from url against a lockfile

## Options

- `lockFile [string]`: path to a lockfile (for example: `lock.json`). Throws if the integrity of the loaded cache file does not match the integrity in the lockfile

## Required flags

- `--allow-read` for local files
- `--allow-read` for local files and lockfiles
- `--allow-net` for fetching urls

## Usage

```ts
import { pluginRootResolver } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/rootResolver/mod.ts";
import { pluginImportResolver } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/importResolver/mod.ts";
import { pluginFileLoader } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/fileLoader/mod.ts";

export default {
plugins: [pluginRootResolver(), pluginFileLoader()],
plugins: [pluginImportResolver(), pluginFileLoader()],
};
```

### Strict integrity checks

```ts
import { pluginImportResolver } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/importResolver/mod.ts";
import { pluginCacheLoader } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/cacheLoader/mod.ts";
import { pluginFileLoader } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/filLoader/mod.ts";

export default {
plugins: [
pluginImportResolver(),
pluginCacheLoader({ lockFile: "lock.json", cacheOnly: true }),
pluginFileLoader({ lockFile: "lock.json" }),
],
};
```
25 changes: 19 additions & 6 deletions plugin/fileLoader/mod.ts
Original file line number Diff line number Diff line change
@@ -1,16 +1,29 @@
import { Plugin } from "../../deps.ts";
import { path, Plugin } from "../../deps.ts";
import { checkIntegrity } from "../../util/checkIntegrity.ts";
import { isHttpUrl } from "../../util/isHttpUrl.ts";

export function pluginFileLoader(): Plugin {
export type Opts = {
lockFile?: string;
};

export function pluginFileLoader(opts: Opts = {}): Plugin {
const lockFile: Record<string, string> | undefined = opts.lockFile
? JSON.parse(Deno.readTextFileSync(path.resolve(opts.lockFile)))
: undefined;

return {
name: "denopack-plugin-fileLoader",
async load(id) {
if (isHttpUrl(id)) {
const response = await fetch(id);
return response.text();
} else {
if (!isHttpUrl(id)) {
return Deno.readTextFile(id);
}

const response = await fetch(id);
const code = await response.text();

if (lockFile) checkIntegrity(lockFile, id, code);

return code;
},
};
}
91 changes: 68 additions & 23 deletions plugin/hooks.ts
Original file line number Diff line number Diff line change
@@ -1,34 +1,79 @@
import { pluginCacheLoader } from "./cacheLoader/mod.ts";
import { pluginFileLoader } from "./fileLoader/mod.ts";
import { Opts as CacheLoaderOptions, pluginCacheLoader } from "./cacheLoader/mod.ts";
import { Opts as FileLoaderOptions, pluginFileLoader } from "./fileLoader/mod.ts";
import { pluginImportResolver } from "./importResolver/mod.ts";
import { pluginTypescriptCompile } from "./typescriptCompile/mod.ts";
import { pluginTypescriptTransform } from "./typescriptTransform/mod.ts";

export const useAlwaysFetch = (opts: Deno.CompilerOptions = {}) => [
pluginImportResolver(),
pluginFileLoader(),
pluginTypescriptTransform(opts),
];
const useLoadAndTransform = ({
fileLoaderOptions,
compilerOptions,
}: {
compilerOptions?: Deno.CompilerOptions;
fileLoaderOptions?: FileLoaderOptions;
}) => [pluginFileLoader(fileLoaderOptions), pluginTypescriptTransform(compilerOptions)];

export const useCache = (opts: Deno.CompilerOptions = {}) => [
pluginImportResolver(),
pluginCacheLoader(),
pluginFileLoader(),
pluginTypescriptTransform(opts),
];
export const useAlwaysFetch = (
opts: {
compilerOptions?: Deno.CompilerOptions;
fileLoaderOptions?: FileLoaderOptions;
} = {}
) => [pluginImportResolver(), ...useLoadAndTransform(opts)];

export const useCacheLazy = (opts: Deno.CompilerOptions = {}) => [
export const useCache = (
opts: {
cacheLoaderOptions?: CacheLoaderOptions;
compilerOptions?: Deno.CompilerOptions;
fileLoaderOptions?: FileLoaderOptions;
} = {}
) => [
pluginImportResolver(),
pluginCacheLoader({ lazy: true }),
pluginFileLoader(),
pluginTypescriptTransform(opts),
pluginCacheLoader(opts.cacheLoaderOptions),
...useLoadAndTransform({
compilerOptions: opts.compilerOptions,
fileLoaderOptions: opts.fileLoaderOptions,
}),
];

export const useCompile = (opts: Deno.CompilerOptions = {}) => [
pluginTypescriptCompile({ useAsLoader: false, compilerOptions: opts }),
pluginFileLoader(),
];
export const useCacheLazy = (
opts: {
fileLoaderOptions?: FileLoaderOptions;
cacheLoaderOptions?: CacheLoaderOptions;
compilerOptions?: Deno.CompilerOptions;
} = {}
) =>
useCache({
cacheLoaderOptions: { ...opts.cacheLoaderOptions, lazy: true },
compilerOptions: opts.compilerOptions,
fileLoaderOptions: opts.fileLoaderOptions,
});

export const useCacheOnly = (
opts: {
fileLoaderOptions?: FileLoaderOptions;
cacheLoaderOptions?: CacheLoaderOptions;
compilerOptions?: Deno.CompilerOptions;
} = {}
) =>
useCache({
fileLoaderOptions: opts.fileLoaderOptions,
cacheLoaderOptions: { ...opts.cacheLoaderOptions, cacheOnly: true },
compilerOptions: opts.compilerOptions,
});

export const useCompileAsLoader = (opts: Deno.CompilerOptions = {}) => [
pluginTypescriptCompile({ useAsLoader: true, compilerOptions: opts }),
export const useCompile = (
opts: {
cacheLoaderOptions?: CacheLoaderOptions;
compilerOptions?: Deno.CompilerOptions;
fileLoaderOptions?: FileLoaderOptions;
} = {}
) => [
pluginTypescriptCompile({ useAsLoader: false, compilerOptions: opts.compilerOptions }),
pluginCacheLoader(opts.cacheLoaderOptions),
pluginFileLoader(opts.fileLoaderOptions),
];

export const useCompileAsLoader = (
opts: {
compilerOptions?: Deno.CompilerOptions;
} = {}
) => [pluginTypescriptCompile({ useAsLoader: true, compilerOptions: opts.compilerOptions })];
4 changes: 2 additions & 2 deletions plugin/mod.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ export * from "./importResolver/mod.ts";
export * from "./rootResolver/mod.ts";
export * from "./cacheResolver/mod.ts";
export * from "./chainResolver/mod.ts";
export * from "./cacheLoader/mod.ts";
export * from "./fileLoader/mod.ts";
export { pluginCacheLoader, Opts as CacheLoaderOptions } from "./cacheLoader/mod.ts";
export { pluginFileLoader, Opts as FileLoaderOptions } from "./fileLoader/mod.ts";
export * from "./terserTransform/mod.ts";
export * from "./typescriptTransform/mod.ts";
export * from "./typescriptCompile/mod.ts";
8 changes: 6 additions & 2 deletions plugin/terserTransform/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,17 @@ Use Terser to minify/compress/mangle/... your bundle.
## Usage

```ts
import { pluginRootResolver } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/rootResolver/mod.ts";
import { pluginImportResolver } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/importResolver/mod.ts";
import { pluginFileLoader } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/fileLoader/mod.ts";
import { pluginTypescriptTransform } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/typescriptTransform/mod.ts";
import { pluginTerserTransform } from "https://cdn.jsdelivr.net/gh/denofn/denopack@latest/plugin/terserTransform/mod.ts";

export default {
plugins: [pluginRootResolver(), pluginFileLoader(), pluginTypescriptTransform({ ...myOptions })],
plugins: [
pluginImportResolver(),
pluginFileLoader(),
pluginTypescriptTransform({ ...myOptions }),
],
output: {
plugins: [pluginTerserTransform()],
},
Expand Down
Loading

0 comments on commit c92e93a

Please sign in to comment.