Skip to content

Commit

Permalink
Fix URLs in all packages' README (#9)
Browse files Browse the repository at this point in the history
  • Loading branch information
DSoko2 committed Nov 8, 2023
1 parent f2c5994 commit 95b8ff5
Show file tree
Hide file tree
Showing 11 changed files with 32 additions and 23 deletions.
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@
![CI workflow](https://github.com/proti-iac/proti/actions/workflows/ci.yaml/badge.svg)
[![GitHub version](https://badge.fury.io/gh/proti-iac%2Fproti.svg)](https://badge.fury.io/gh/proti-iac%2Fproti)
[![NPM version](https://badge.fury.io/js/@proti-iac%2Fcore.svg)](https://badge.fury.io/js/@proti-iac%2Fcore)
[![License](https://img.shields.io/github/license/proti-iac/proti)](LICENSE)
[![License](https://img.shields.io/github/license/proti-iac/proti)](./LICENSE)
[![DOI](https://zenodo.org/badge/706779109.svg)](https://zenodo.org/doi/10.5281/zenodo.10028479)

[ProTI](https://proti-iac.github.io) is an automated unit testing tool for Infrastructure as Code (IaC) programs. ProTI implements [Automated Configuration Testing (ACT)](https://proti-iac.github.io/#approach) for [Pulumi](https://pulumi.com) TypeScript, minimizing the development effort for unit testing Pulumi TypeScript IaC programs. ProTI is extensible through test generator and oracle plugins and leverages ideas from [fuzzing](https://en.wikipedia.org/wiki/Fuzzing) and [property-based testing](https://en.wikipedia.org/wiki/Software_testing#Property_testing).

ProTI builds upon [Jest](https://jestjs.io/), implementing the Jest runner [`@proti-iac/runner`](./proti-runner/), the Jest test-runner [`@proti-iac/test-runner`](./proti-test-runner/), and the Jest reporter [`@proti-iac/reporter`](./proti-reporter/). [`@proti-iac/core`](./proti-core/) and [`@proti-iac/spec`](./proti-spec/) implement the core abstractions and the inline specification syntax, leveraging [fast-check](https://fast-check.dev) for random-based testing abstractions. [`@proti-iac/pulumi-packages-schema`](./proti-pulumi-packages-schema/) implements the first type-based generator and oracle plugins. [@proti-iac/plugins-demo](./proti-plugins-demo/) demonstrates the setup of an NPM package of configurable ProTI generator and oracle plugins, serving as a blueprint for new ProTI plugins.
ProTI builds upon [Jest](https://jestjs.io/), implementing the Jest runner [`@proti-iac/runner`](./proti-runner/), the Jest test-runner [`@proti-iac/test-runner`](./proti-test-runner/), and the Jest reporter [`@proti-iac/reporter`](./proti-reporter/). [`@proti-iac/core`](./proti-core/) and [`@proti-iac/spec`](./proti-spec/) implement the core abstractions and the inline specification syntax, leveraging [fast-check](https://fast-check.dev) for random-based testing abstractions. [`@proti-iac/pulumi-packages-schema`](./proti-pulumi-packages-schema/) implements the first type-based generator and oracle plugins. [`@proti-iac/plugins-demo`](./proti-plugins-demo/) demonstrates the setup of an NPM package of configurable ProTI generator and oracle plugins, serving as a blueprint for new ProTI plugins.

![Overview of ProTI's NPM packages](https://proti-iac.github.io/assets/img/proti-packages.svg)

Expand Down Expand Up @@ -45,7 +45,7 @@ const config = {
module.exports = config;
```

Add further configuration to the file to augment Jest's, ts-jest's, and ProTI's default configuration. The default configuration configures a simple empty state test generator and an oracle that only checks URN uniqueness across all resources, which are implemented in [`@proti-iac/core`](./proti-core/). Most likely, you want to configure more sophisticated generator and generator plugins. [The next section](#configuring-proti) describes how. Concretely, [@proti-iac/pulumi-packages-schema's README](./proti-pulumi-packages-schema/README.md) describes how to install and configure our first type-based plugins.
Add further configuration to the file to augment Jest's, ts-jest's, and ProTI's default configuration. The default configuration configures a simple empty state test generator and an oracle that only checks URN uniqueness across all resources, which are implemented in [`@proti-iac/core`](./proti-core/). Most likely, you want to configure more sophisticated generator and generator plugins. [The next section](#configuring-proti) describes how. Concretely, [`@proti-iac/pulumi-packages-schema`'s README](./proti-pulumi-packages-schema/README.md) describes how to install and configure our first type-based plugins.

4. Run ProTI by running Jest on the project:

Expand All @@ -61,7 +61,7 @@ The generator plugin and the set of oracle plugins are configured in the test co

The test runner configuration object under `globals.proti.testRunner` is passed as a whole to fast-check's runner in ProTI's test runner, inheriting all [configuration options of fast-check](https://fast-check.dev/api-reference/interfaces/Parameters.html). For instance, you can configure the test runner's verbosity level in `globals.proti.testRunner.verbose`. `0` only shows the final error result, `1` all failing tests, `2` the full tree of passing and failing tests.

ProTI plugins are configured through `globals.proti.plugins.[PLUGIN NAME]`. For instance, the schema registry's schema cache of [@proti-iac/pulumi-packages-schema](./proti-pulumi-packages-schema/) can be disabled by setting `globals.proti.plugins.pulumi-packages-schema.registry.cacheDownloadedSchemas` to `false`.
ProTI plugins are configured through `globals.proti.plugins.[PLUGIN NAME]`. For instance, the schema registry's schema cache of [`@proti-iac/pulumi-packages-schema`](./proti-pulumi-packages-schema/) can be disabled by setting `globals.proti.plugins.pulumi-packages-schema.registry.cacheDownloadedSchemas` to `false`.

### Using ProTI's Inline Specifications

Expand Down Expand Up @@ -111,7 +111,7 @@ Workaround: Use another Pulumi version than 3.72.0 or 3.73.0.

### Parallel Test Runner

We use our own Jest runner `@proti-iac/runner` to pass down the resolver to the test runner `@proti-iac/test-runner`. This works well in single-worker execution. However, currently, parallel multi-worker execution is broken, most likely because serialization and deserialization of the resolver fails when Jest passes the resolver to the separate test runner.
We use our own Jest runner [`@proti-iac/runner`](./proti-runner/) to pass down the resolver to the test runner [`@proti-iac/test-runner`](./proti-test-runner/). This works well in single-worker execution. However, currently, parallel multi-worker execution is broken, most likely because serialization and deserialization of the resolver fails when Jest passes the resolver to the separate test runner.

Workaround: Only use single-worker execution. If multiple projects are tested, this can be enforced with `--runInBand` or `-i`.

Expand All @@ -125,5 +125,5 @@ Workarounds: Run ProTI only individually on programs. Fix the "Parallel Test Run

We aim to leverage some cool concepts in our code:

* Immutable types: Wherever possible we want to immutable `readonly` type definitions, i.e., through `Readonly<...>`, `ReadonlyArray<...>`, `ReadonlySet<...>`, and `ReadonlyMap<..., ...>`. `@proti-iac/core` implements a `DeepReadonly<...>` utility which makes this easier. More about immutable types in TypeScript: https://levelup.gitconnected.com/the-complete-guide-to-immutability-in-typescript-99154f859fdb
* Immutable types: Wherever possible we want to immutable `readonly` type definitions, i.e., through `Readonly<...>`, `ReadonlyArray<...>`, `ReadonlySet<...>`, and `ReadonlyMap<..., ...>`. [`@proti-iac/core`](./proti-core/) implements a `DeepReadonly<...>` utility which makes this easier. More about immutable types in TypeScript: https://levelup.gitconnected.com/the-complete-guide-to-immutability-in-typescript-99154f859fdb
* Stateless arbitraries: fast-check arbitraries are meant to be stateless, allowing safe re-use across tests. We embrace this and try to re-use instantiated arbitraries wherever possible for better run time and resource efficiency.
5 changes: 4 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,17 @@
"g:build": "yarn g:pkgroot $INIT_CWD tsconfig.json build 'yarn tsc'",
"g:build:tspc": "yarn g:pkgroot $INIT_CWD tsconfig.json build 'yarn tspc'",
"build": "yarn g:all build '-vip --topological-dev'",
"g:license:copy": "yarn g:pkgroot $INIT_CWD tsconfig.json 'copy LICENSE' 'cp ../LICENSE .'",
"g:lint": "eslint --ext .ts --ignore-pattern bin/ $INIT_CWD",
"lint": "yarn g:lint",
"g:test": "yarn g:pkgroot $INIT_CWD tsconfig.json test 'yarn build && INIT_CWD=$(pwd) && cd $PROJECT_CWD && jest --roots $INIT_CWD'",
"test": "yarn build && yarn jest",
"pack:all": "yarn workspaces foreach -vi --topological-dev --exclude proti-monorepo pack --out ../%s@%v.tgz",
"g:prepack": "yarn g:pkgroot $INIT_CWD package.json prepack 'yarn clean && yarn build'",
"prepack": "yarn g:all prepack '-vip --topological-dev'",
"postinstall": "ts-patch install"
"postinstall": "ts-patch install",
"g:readme:copy": "yarn g:pkgroot $INIT_CWD tsconfig.json 'copy README' 'cp ../README.md .'",
"g:readme:urls": "yarn g:pkgroot $INIT_CWD tsconfig.json 'resolve README URLs' 'sed -i.bak \"s#\\(\\[\\`@proti-iac/[^\\`]*\\`\\](\\)\\./proti-\\([^/]*/)\\)#\\1https://www.npmjs.com/package/@proti-iac/\\2#g\" README.md && sed -i.bak \"s#\\(\\](\\)\\.\\{1,2\\}/\\([^)]*)\\)#\\1https://github.com/proti-iac/proti/blob/main/\\2#g\" README.md && rm README.md.bak'"
},
"devDependencies": {
"@proti-iac/runner": "workspace:^",
Expand Down
2 changes: 1 addition & 1 deletion proti-core/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
"build": "yarn g:build:tspc",
"lint": "yarn g:lint",
"test": "yarn g:test",
"prepack": "yarn g:prepack && cp ../README.md . && cp ../LICENSE .",
"prepack": "yarn g:prepack && yarn g:license:copy && yarn g:readme:copy && yarn g:readme:urls",
"postpack": "rm README.md LICENSE",
"postinstall": "ts-patch install"
},
Expand Down
13 changes: 8 additions & 5 deletions proti-plugins-demo/README.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,24 @@
# @proti-iac/plugins-demo

![CI workflow](https://github.com/proti-iac/proti/actions/workflows/ci.yaml/badge.svg)
[![GitHub version](https://badge.fury.io/gh/proti-iac%2Fproti.svg)](https://badge.fury.io/gh/proti-iac%2Fproti)
[![NPM version](https://badge.fury.io/js/%40proti-iac%2Fplugins-demo.svg)](https://npmjs.com/package/@proti-iac/plugins-demo)
[![License](https://img.shields.io/github/license/proti-iac/proti)](LICENSE)
[![License](https://img.shields.io/github/license/proti-iac/proti)](../LICENSE)
[![DOI](https://zenodo.org/badge/706779109.svg)](https://zenodo.org/doi/10.5281/zenodo.10028479)

This package demonstrates the implementation of a [ProTI](https://proti-iac.github.io/) generator and a [ProTI](https://proti-iac.github.io/) oracle plugin. The [demo generator](./src/demo-arbitrary.ts) simply generates an empty output configuration for all resources. The [demo oracle](./src/demo-oracle.ts) checks whether all resource URNs in the IaC program are unique. Both plugins and the [config](./src/config.ts) demonstrate how ProTI plugins can leverage ProTI's interface for ProTI plugin user configuration through Jest. Use this package as a starter blueprint for your own ProTI plugin implementations.
This package demonstrates the implementation of a [ProTI](https://proti-iac.github.io/) generator and a [ProTI](https://proti-iac.github.io/) oracle plugin. The [demo generator](../proti-plugins-demo/src/demo-arbitrary.ts) simply generates an empty output configuration for all resources. The [demo oracle](../proti-plugins-demo/src/demo-oracle.ts) checks whether all resource URNs in the IaC program are unique. Both plugins and the [config](../proti-plugins-demo/src/config.ts) demonstrate how ProTI plugins can leverage ProTI's interface for ProTI plugin user configuration through Jest. Use this package as a starter blueprint for your own ProTI plugin implementations.

For development, we provide some general instructions under the [Developers Guide](../README.md#developers-guide) in the main README of this repository.

This package implements a rudimentary end-to-end test using one of the example Pulumi TypeScript programs in this project. You can run it through jest by running `yarn test`.

## Implementing Generator Plugins

[`src/demo-arbitrary.ts`](./src/demo-arbitrary.ts) is an example ProTI generator plugin. Generator modules must have a default export that implements `Arbitrary<Generator>`. `Arbitrary` is the arbitrary abstraction from fast-check, and [`Generator`](../proti-core/src/generator.ts) the test generator abstraction of ProTI. ProTI draws one generator from the arbitrary per test run. The generator itself is then invoked throughout the test run to incrementally provide the input for the test run, practically unfolding the test case.
[`src/demo-arbitrary.ts`](../proti-plugins-demo/src/demo-arbitrary.ts) is an example ProTI generator plugin. Generator modules must have a default export that implements `Arbitrary<Generator>`. `Arbitrary` is the arbitrary abstraction from fast-check, and [`Generator`](../proti-core/src/generator.ts) the test generator abstraction of ProTI. ProTI draws one generator from the arbitrary per test run. The generator itself is then invoked throughout the test run to incrementally provide the input for the test run, practically unfolding the test case.

## Implementing Oracle Plugins

[`src/demo-oracle.ts`](./src/demo-oracle.ts) is an example ProTI oracle plugin. Oracle modules must have a default export that implements one of [ProTI's oracle interfaces](../proti-core/src/oracle.ts). The following oracle interfaces exist:
[`src/demo-oracle.ts`](../proti-plugins-demo/src/demo-oracle.ts) is an example ProTI oracle plugin. Oracle modules must have a default export that implements one of [ProTI's oracle interfaces](../proti-core/src/oracle.ts). The following oracle interfaces exist:

* `ResourceOracle` is for oracles that are synchronously invoked for every resource on its definition.
* `AsyncResourceOracle` is like `ResourceOracle` but asynchronous.
Expand Down Expand Up @@ -58,4 +61,4 @@ const config = {
module.exports = config;
```

This package is configurable. The options are defined by the [exported `Config` type](./src/config.ts). This configuration can be defined under `globals.proti.plugins.plugins-demo`.
This package is configurable. The options are defined by the [exported `Config` type](../proti-plugins-demo/src/config.ts). This configuration can be defined under `globals.proti.plugins.plugins-demo`.
4 changes: 2 additions & 2 deletions proti-plugins-demo/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@
"dev": "yarn g:build:tspc && yarn jest -c \"{ \\\"preset\\\": \\\"../proti-test-runner/jest-preset.js\\\", \\\"globals\\\": { \\\"proti\\\": { \\\"testCoordinator\\\": { \\\"arbitrary\\\": \\\"$(pwd)/bin/demo-arbitrary\\\", \\\"oracles\\\": [ \\\"$(pwd)/bin/demo-oracle\\\" ] }, \\\"plugins\\\": { \\\"plugins-demo\\\": {} } } } }\" --roots",
"lint": "yarn g:lint",
"test": "yarn g:test",
"prepack": "yarn g:prepack && cp ../LICENSE .",
"postpack": "rm LICENSE",
"prepack": "yarn g:prepack && yarn g:license:copy && cp README.md READMEbu.md && yarn g:readme:urls",
"postpack": "rm README.md LICENSE && mv READMEbu.md README.md",
"postinstall": "ts-patch install"
},
"dependencies": {
Expand Down
7 changes: 5 additions & 2 deletions proti-pulumi-packages-schema/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
# @proti-iac/pulumi-packages-schema

![CI workflow](https://github.com/proti-iac/proti/actions/workflows/ci.yaml/badge.svg)
[![GitHub version](https://badge.fury.io/gh/proti-iac%2Fproti.svg)](https://badge.fury.io/gh/proti-iac%2Fproti)
[![NPM version](https://badge.fury.io/js/%40proti-iac%2Fpulumi-packages-schema.svg)](https://npmjs.com/package/@proti-iac/pulumi-packages-schema)
[![License](https://img.shields.io/github/license/proti-iac/proti)](LICENSE)
[![License](https://img.shields.io/github/license/proti-iac/proti)](../LICENSE)
[![DOI](https://zenodo.org/badge/706779109.svg)](https://zenodo.org/doi/10.5281/zenodo.10028479)

This package implements a [ProTI](https://proti-iac.github.io/) generator and a [ProTI](https://proti-iac.github.io/) oracle plugin. They are type-based, leveraging input and output configuration type metadata from [Pulumi package schemas](https://www.pulumi.com/docs/using-pulumi/pulumi-packages/schema/). By design, these are available for all resource types distributed in [Pulumi packages](https://www.pulumi.com/product/packages/). The generator plugin composes primitive fast-check arbitraries to a complex arbitrary of the shape of the resource type's output configuration type and draws random output configuration values from such arbitrary. The oracle plugin checks all received input configurations for type compliance with the resource type's input configuration type.

Expand Down Expand Up @@ -35,7 +38,7 @@ const config = {
module.exports = config;
```

This package is configurable. The options are defined by the [exported `Config` type](./src/config.ts). This configuration can be defined under `globals.proti.plugins.pulumi-packages-schema`. For instance, to disable the schema registry's schema cache you can set `globals.proti.plugins.pulumi-packages-schema.registry.cacheDownloadedSchemas` of the `config` object in your project's `jest.config.js` file to `false`.
This package is configurable. The options are defined by the [exported `Config` type](../proti-pulumi-packages-schema/src/config.ts). This configuration can be defined under `globals.proti.plugins.pulumi-packages-schema`. For instance, to disable the schema registry's schema cache you can set `globals.proti.plugins.pulumi-packages-schema.registry.cacheDownloadedSchemas` of the `config` object in your project's `jest.config.js` file to `false`.

## Schema Registry

Expand Down
4 changes: 2 additions & 2 deletions proti-pulumi-packages-schema/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@
"dev": "yarn g:build:tspc && yarn jest -c \"{ \\\"preset\\\": \\\"../proti-test-runner/jest-preset.js\\\", \\\"globals\\\": { \\\"proti\\\": { \\\"testCoordinator\\\": { \\\"arbitrary\\\": \\\"$(pwd)/bin/arbitrary\\\", \\\"oracles\\\": [ \\\"$(pwd)/bin/oracle\\\" ] }, \\\"plugins\\\": { \\\"pulumi-packages-schema\\\": {} } } } }\" --roots",
"lint": "yarn g:lint",
"test": "yarn g:test",
"prepack": "yarn g:prepack && cp ../LICENSE .",
"postpack": "rm LICENSE",
"prepack": "yarn g:prepack && yarn g:license:copy && cp README.md READMEbu.md && yarn g:readme:urls",
"postpack": "rm README.md LICENSE && mv READMEbu.md README.md",
"update-schema": "curl https://raw.githubusercontent.com/pulumi/pulumi/master/pkg/codegen/schema/pulumi.json | yarn json2ts > src/pulumi-generated.d.ts",
"postinstall": "ts-patch install"
},
Expand Down
2 changes: 1 addition & 1 deletion proti-reporter/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"build": "yarn g:build:tspc",
"lint": "yarn g:lint",
"test": "yarn g:test",
"prepack": "yarn g:prepack && cp ../README.md . && cp ../LICENSE .",
"prepack": "yarn g:prepack && yarn g:license:copy && yarn g:readme:copy && yarn g:readme:urls",
"postpack": "rm README.md LICENSE",
"postinstall": "ts-patch install"
},
Expand Down
2 changes: 1 addition & 1 deletion proti-runner/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"build": "yarn g:build",
"lint": "yarn g:lint",
"test": "yarn g:test",
"prepack": "yarn g:prepack && cp ../README.md . && cp ../LICENSE .",
"prepack": "yarn g:prepack && yarn g:license:copy && yarn g:readme:copy && yarn g:readme:urls",
"postpack": "rm README.md LICENSE"
},
"dependencies": {
Expand Down
2 changes: 1 addition & 1 deletion proti-spec/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"build": "yarn g:build",
"lint": "yarn g:lint",
"test": "yarn g:test",
"prepack": "yarn g:prepack && cp ../README.md . && cp ../LICENSE .",
"prepack": "yarn g:prepack && yarn g:license:copy && yarn g:readme:copy && yarn g:readme:urls",
"postpack": "rm README.md LICENSE"
},
"dependencies": {
Expand Down
2 changes: 1 addition & 1 deletion proti-test-runner/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
"dev": "yarn g:build && yarn jest -c jest-preset.js --roots",
"lint": "yarn g:lint",
"test": "yarn g:test",
"prepack": "yarn g:prepack && cp ../README.md . && cp ../LICENSE .",
"prepack": "yarn g:prepack && yarn g:license:copy && yarn g:readme:copy && yarn g:readme:urls",
"postpack": "rm README.md LICENSE"
},
"dependencies": {
Expand Down

0 comments on commit 95b8ff5

Please sign in to comment.