diff --git a/docs/topics/dependencies.md b/docs/topics/dependencies.md new file mode 100644 index 000000000..0cf5022c9 --- /dev/null +++ b/docs/topics/dependencies.md @@ -0,0 +1,123 @@ +# Import dependencies in your `brigade.js` file + +A Brigade worker is responsible for executing your `brigade.js` file. By default, Brigade comes with a general purpose worker which does not have any external dependency that is not critical to controlling the flow of your pipeline. + +If you want to have other dependencies available in your worker execution environment (and available in `brigade.js`), there are multiple approaches: + +- by creating a custom worker container image, which has your dependencies. This approach is [described in detail in this document](workers.md). In a nutshell, use this approach if you have the same dependency for multiple projects, or if your dependencies take a long time to pull. + +- without creating a custom container image: + - by adding a `brigade.json` file in your repository (similar to a `package.json` file) +that contains the dependencies and that are specific on every Brigade project. + - by directly using local dependencies located in your project repository. + +This document describes the last two approaches. + +> Here you can find a [repository that exemplifies both approaches here](https://github.com/radu-matei/brigade-javascript-deps). + +## Add custom dependencies using a `brigade.json` file + +If you need different dependencies for every Brigade project, this can be easily achieved +using a `brigade.json` file placed side-by-side the `brigade.js` file. This file contains +the dependency name and version, and has the following structure: + +``` +{ + "dependencies": { + "is-thirteen": "2.0.0" + } +} +``` +Before starting to execute the `brigade.js` script, the worker will install the +dependencies using `yarn`, adding them to the `node_modules` folder. + +Then, in the `brigade.js` file, the new dependency can be used just like any +other NodeJS dependency: + +``` +const { events } = require("brigadier") +const is = require("is-thirteen"); + +events.on("exec", function (e, p) { + console.log("is 13 thirteen? " + is(13).thirteen()); +}) +``` + +Now if we run a build for this project, we see the `is-thirteen` dependency added, +as well as the console log resulted from using it: + +``` +$ brig run brigade-86959b08a89af5b6b83f0ace6d9030f1fdca7ed8ea0a296e27d72e +Event created. Waiting for worker pod named "brigade-worker-01cexrvrs08shcev26961cwd6n". +Started build 01cexrvrs08shcev26961cwd6n as "brigade-worker-01cexrvrs08shcev26961cwd6n" +installing is-thirteen@2.0.0 +prestart: src/brigade.js written +[brigade] brigade-worker version: 0.14.0 +[brigade:k8s] Creating PVC named brigade-worker-01cexrvrs08shcev26961cwd6n +is 13 thirteen? true +[brigade:app] after: default event handler fired +[brigade:app] beforeExit(2): destroying storage +[brigade:k8s] Destroying PVC named brigade-worker-01cexrvrs08shcev26961cwd6n +``` + +Notes: + +- the dependencies _must_ point to the exact version (and not use the tilde ~ and caret ^ to indicate semver compatible versions) + +- when adding a custom dependency using `brigade.json`, `yarn` will add it side-by-side with [the worker's +dependencies](../../brigade-worker/package.json) - this means that the process will fail if a dependency that conflicts with one of the +worker's dependencies is added. However, already existing dependencies (such as `@kubernetes/client-node`, `ulid` or `chai`) +can be used from `brigade.js` without adding them to `brigade.json`. + +However, the only issue is trying to add a different version of an already existing dependency of the worker. + +- as the Brigade worker is capable of modifying the state of the cluster, be mindful +of any external dependencies added through `brigade.json` + +- for now, the only part of `brigade.json` used is the `dependencies` object - it means the rest of the file +can be used to pass additional information to `brigade.js`, such as environment data - however, as the `brigade.json` +file is passed in the source control repository, information passed there will be available to anyone with access to the repository. + +- all dependencies added in `brigade.json` are dynamically installed on every Brigade build - this means if the dependencies added +are large, and the build frequency is high for a particular project, it might make sense to make a pre-built Docker image that +already contains the dependencies, [as described in this document](workers.md). + +## Using local dependencies from the project repository + +*Note: this approach is currently experimental, and only for projects whose `brigade.js` is located at the root of the repository.* + +This approach works great for using dependencies that are not intended to be external packages, and which are located in the project repository. + +Let's consider the following scenario: we have a JavaScript file located in `/local-deps/circle.js`, where `local-deps` is a directory at the root of our git repository. In our `brigade.js` file, we can use any exported method or variable from that package by simply using a `require` statement, just like in any other JavaScript project. + +```javascript +// file /local-deps/circle.js +var PI = 3.14; +exports.area = function (r) { + return PI * r * r; +}; +exports.circumference = function (r) { + return 2 * PI * r; +}; +``` + +Then, in our `brigade.js` we can import that file and use it: + +```javascript +const { events } = require("@azure/brigadier") +const circle = require("./local-deps/circle"); + +events.on("exec", function (e, p) { + console.log("area of a circle with radius 3 " + circle.area(3)); +}); +``` + +## Best Practices + +As we have seen, it is easy to add new functionality to the Brigade worker. But +it is important to keep in mind that the Worker is intended to do one thing: +execute Brigade chains. + +To that end, it is best to fight the temptation to put too much logic into the +Brigade worker. Where possible, use Jobs to perform specific tasks within their +own containers, and use workers to control the execution of a series of Jobs. \ No newline at end of file diff --git a/docs/topics/index.md b/docs/topics/index.md index 6c3f968bc..a3292294a 100644 --- a/docs/topics/index.md +++ b/docs/topics/index.md @@ -12,6 +12,7 @@ If you don't see a topic guide here and have a reasonable level of knowledge on - [Scripting Guide](scripting.md): How to write JavaScript for `brigade.js` files. - [Brigade.js Reference](javascript.md): The API for brigade.js files. - [Scripting Guide - Advanced](scripting_advanced.md): Advanced examples for `brigade.js` files. + - [Adding dependencies to `brigade.js`](dependencies.md): How to add local dependencies and NPM packages to your `brigade.js` files. - [GitHub Integration](github.md): A guide for configuring GitHub integration. - [Container Registry Integration](dockerhub.md): A guide for configuring integration with DockerHub or Azure Container Registry. - [Generic Gateway](genericgateway.md): How to use Brigade's Generic Gateway functionality. diff --git a/docs/topics/workers.md b/docs/topics/workers.md index 4cb72ffb9..8f1a2d7eb 100644 --- a/docs/topics/workers.md +++ b/docs/topics/workers.md @@ -9,84 +9,15 @@ Sometimes, though, it is desirable to include additional libraries -- perhaps ev custom libraries -- to your workers. There are two ways of adding custom dependencies to a Brigade worker: -- by adding a `brigade.json` file in your repository (similar to a `package.json` file) -that contains the dependencies and that are specific on every Brigade project - - by creating a custom Docker image for the worker that already contains the dependencies -and which will be used for all Brigade projects +and which will be used for all Brigade projects. -This guide explains both processes and describes the cases where each method is most suitable. +- without creating a custom container image - check [the dependencies document](dependencies.md) for a detailed description for this approach. **Note:** This area of Brigade is still evolving. If you have ideas for improving it, feel free to [file an issue](https://github.com/Azure/brigade/issues) explaining your idea. -## Add custom dependencies using a `brigade.json` file - -If you need different dependencies for every Brigade project, this can be easily achieved -using a `brigade.json` file placed side-by-side the `brigade.js` file. This file contains -the dependency name and version, and has the following structure: - -``` -{ - "dependencies": { - "is-thirteen": "2.0.0" - } -} -``` -Before starting to execute the `brigade.js` script, the worker will install the -dependencies using `yarn`, adding them to the `node_modules` folder. - -Then, in the `brigade.js` file, the new dependency can be used just like any -other NodeJS dependency: - -``` -const { events } = require("brigadier") -const is = require("is-thirteen"); - -events.on("exec", function (e, p) { - console.log("is 13 thirteen? " + is(13).thirteen()); -}) -``` - -Now if we run a build for this project, we see the `is-thirteen` dependency added, -as well as the console log resulted from using it: - -``` -$ brig run brigade-86959b08a89af5b6b83f0ace6d9030f1fdca7ed8ea0a296e27d72e -Event created. Waiting for worker pod named "brigade-worker-01cexrvrs08shcev26961cwd6n". -Started build 01cexrvrs08shcev26961cwd6n as "brigade-worker-01cexrvrs08shcev26961cwd6n" -installing is-thirteen@2.0.0 -prestart: src/brigade.js written -[brigade] brigade-worker version: 0.14.0 -[brigade:k8s] Creating PVC named brigade-worker-01cexrvrs08shcev26961cwd6n -is 13 thirteen? true -[brigade:app] after: default event handler fired -[brigade:app] beforeExit(2): destroying storage -[brigade:k8s] Destroying PVC named brigade-worker-01cexrvrs08shcev26961cwd6n -``` - -Notes: - -- the dependencies should point the exact version (and not use the tilde ~ and caret ^ to indicate semver compatible versions) - -- when adding a custom dependency using `brigade.json`, `yarn` will add it side-by-side with [the worker's -dependencies](../../brigade-worker/package.json) - this means that the process will fail if a dependency that conflicts with one of the -worker's dependency is added. However, already existing dependencies (such as `@kubernetes/client-node`, `ulid` or `chai`) -can be used from `brigade.js` without adding them to `brigade.json`. - -However, the only issue is trying to add a different version of an already existing dependency of the worker. - -- as the Brigade worker is capable of modifying the state of the cluster, be mindful -of any external dependencies added through `brigade.json` - -- for now, the only part of `brigade.json` used is the `dependencies` object - it means the rest of the file -can be used to pass additional information to `brigade.js`, such as environment data - however, as the `brigade.json` -file is passed in the source control repository, information passed there will be available to anyone with access to the repository. - -- all dependencies added in `brigade.json` are dynamically installed on every Brigade build - this means if the dependencies added -are large, and the build frequency is high for a particular project, it might make sense to make a pre-built Docker image that -already contains the dependencies, as described in the sections below. ## Workers and Docker Images @@ -284,15 +215,7 @@ Now we've added a few new lines to the script. We import `alpineJob` from our `./mylib` module at the top. Then, late in the script, we call `alpineJob` to create a new job for us. -## Best Practices - -As we have seen, it is easy to add new functionality to the Brigade worker. But -it is important to keep in mind that the Worker is intended to do one thing: -execute Brigade chains. - -To that end, it is best to fight the temptation to put too much logic into the -Brigade worker. Where possible, use Jobs to perform specific tasks within their -own containers, and use workers to control the execution of a series of Jobs. +## Best practices We strongly discourage attempting to turn a worker into a long-running server. This violates the design assumptions of Brigade, and can result in unintended