Skip to content

Commit

Permalink
Merge pull request #404 from OpenFn/engine
Browse files Browse the repository at this point in the history
Runtime Engine
  • Loading branch information
josephjclark authored Oct 31, 2023
2 parents 1a0e3a9 + dc4a197 commit c4a8b84
Show file tree
Hide file tree
Showing 120 changed files with 5,505 additions and 1,758 deletions.
5 changes: 5 additions & 0 deletions .changeset/cyan-worms-clap.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@openfn/logger': patch
---

Export SanitizePolicies type
5 changes: 5 additions & 0 deletions .changeset/short-pens-punch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@openfn/runtime': patch
---

Support a cacheKey to bust cached modules in long-running processes
5 changes: 5 additions & 0 deletions .changeset/tough-coats-unite.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@openfn/runtime': patch
---

Trigger callbacks on job start, complete and init
2 changes: 1 addition & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ jobs:
# Specify the execution environment. You can specify an image from Dockerhub or use one of our Convenience Images from CircleCI's Developer Hub.
# See: https://circleci.com/docs/2.0/configuration-reference/#docker-machine-macos-windows-executor
docker:
- image: cimg/node:18.12
- image: cimg/node:18.18
resource_class: medium
# Add steps to the job
# See: https://circleci.com/docs/2.0/configuration-reference/#steps
Expand Down
2 changes: 1 addition & 1 deletion .tool-versions
Original file line number Diff line number Diff line change
@@ -1 +1 @@
nodejs 18.12.1
nodejs 18.18.2
4 changes: 2 additions & 2 deletions integration-tests/cli/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@openfn/integration-tests",
"name": "@openfn/integration-tests-cli",
"private": true,
"version": "0.0.1",
"version": "1.0.0",
"description": "CLI integration tests",
"author": "Open Function Group <admin@openfn.org>",
"license": "ISC",
Expand Down
2 changes: 2 additions & 0 deletions integration-tests/worker/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
node_modules
repo
15 changes: 15 additions & 0 deletions integration-tests/worker/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# This dockerfile allows us to run tests against the build in isolation
# (I'm not really sure we need it but it's been a useful dev tool)
FROM node:18.12.0

ENV NODE_ENV=production

WORKDIR /app

COPY . .

COPY ./package.json.container ./package.json

RUN npm install --production

CMD ["npm", "test"]
13 changes: 13 additions & 0 deletions integration-tests/worker/ava.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
export default {
extensions: {
ts: 'module',
},

environmentVariables: {
TS_NODE_TRANSPILE_ONLY: 'true',
},

nodeArguments: ['--loader=ts-node/esm', '--no-warnings'],

files: ['test/**/*test.ts'],
};

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions integration-tests/worker/dummy-repo/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"name": "test-repo",
"private": true,
"version": "1.0.0",
"dependencies": {
"@openfn/stateful-test_1.0.0": "@npm:@openfn/stateful-test@1.0.0"
}
}
35 changes: 35 additions & 0 deletions integration-tests/worker/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{
"name": "@openfn/integration-tests-worker",
"private": true,
"version": "1.0.0",
"description": "Lightning WOrker integration tests",
"author": "Open Function Group <admin@openfn.org>",
"license": "ISC",
"type": "module",
"scripts": {
"clean": "rimraf dist tmp/repo/*",
"build:pack": "pnpm clean && cd ../.. && pnpm pack:local integration-tests/worker/dist --no-version",
"build": "pnpm build:pack && docker build --tag worker-integration-tests .",
"start": "docker run worker-integration-tests",
"test": "pnpm clean && npx ava -s --timeout 2m"
},
"dependencies": {
"@openfn/engine-multi": "workspace:^",
"@openfn/lightning-mock": "workspace:^",
"@openfn/logger": "workspace:^",
"@openfn/ws-worker": "workspace:^",
"@types/node": "^18.15.13",
"@types/rimraf": "^3.0.2",
"ava": "5.3.1",
"date-fns": "^2.30.0",
"koa": "^2.13.4",
"rimraf": "^3.0.2",
"ts-node": "10.8.1",
"tslib": "^2.4.0",
"typescript": "^5.1.6"
},
"files": [
"dist",
"README.md"
]
}
34 changes: 34 additions & 0 deletions integration-tests/worker/package.json.container
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
{
"name": "@openfn/integration-tests-worker",
"private": true,
"version": "1.0.0",
"description": "Lightning WOrker integration tests",
"author": "Open Function Group <admin@openfn.org>",
"license": "ISC",
"type": "module",
"scripts": {
"clean": "rimraf dist tmp/repo/*",
"build:pack": "npm run clean && cd ../.. && npm run pack:local integration-tests/worker/dist --no-version",
"build": "npm run build:pack && docker build --tag worker-integration-tests .",
"start": "docker run worker-integration-tests",
"test": "npm run clean && npx ava -s --timeout 2m"
},
"dependencies": {
"@openfn/engine-multi": "./dist/openfn-engine-multi.tgz",
"@openfn/lightning-mock": "./dist/openfn-lightning-mock.tgz",
"@openfn/logger": "./dist/openfn-logger.tgz",
"@openfn/ws-worker": "./dist/openfn-ws-worker.tgz",
"@types/node": "^18.15.13",
"ava": "5.3.1",
"date-fns": "^2.30.0",
"rimraf": "^3.0.2",
"ts-node": "10.8.1",
"tslib": "^2.4.0",
"typescript": "^5.1.6",
"@types/rimraf": "^3.0.2"
},
"files": [
"dist",
"README.md"
]
}
1 change: 1 addition & 0 deletions integration-tests/worker/readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Integration tests for Lightning Worker architecture.
Loading

0 comments on commit c4a8b84

Please sign in to comment.