Merge pull request #54 from godaddy/charts

Charts

CHANGELOG.md

@@ -1,5 +1,10 @@
 # CHANGELOG
 
+- [#54] Generate diagrams
+  - Include `canihaz` to optionally install `puppeteer`.
+  - Provide script `bin/diagrams` to render `mermaid` diagrams and store snapshots to file
+  - Provide npm command to execute above script
+  - Reference diagrams in documentation.
 - [#53] Give credits for Github templates
 - [#52] Default documenation
   - Add: `CONTRUBUTING.md`, `CHANGELOG.md`, `SECURITY.md`
@@ -8,3 +13,4 @@
 
 [#52]: https://github.com/godaddy/warehouse.ai/pull/52
 [#53]: https://github.com/godaddy/warehouse.ai/pull/53
+[#54]: https://github.com/godaddy/warehouse.ai/pull/54

README.md

@@ -12,8 +12,6 @@ The goal of the Warehouse is to support modular UI development by:
 
 In other words the Warehouse is designed to give as many programmatic guarantees that it is safe to "always be on latest" and make rolling back as painless as possible when issues arise.
 
-![](assets/workflow.png)
-
 - [Developer Experience](#developer-experience)
   - [Releasing code](#releasing-code)
   - [Rolling back to previous versions](#rolling-back-to-previous-versions)
@@ -53,11 +51,19 @@ The release process for any module using the Warehouse is:
 cd /path/to/my/front-end/module
 npm publish
 ```
+![Diagram of npm publish workflow to Warehouse.ai](assets/publish.png)
+
 3. Perform any manual QA in your DEV environment.
 4. Promote the `module@version` to production using `npm dist-tag add`
 ```
 npm dist-tag add module@version prod
 ```
+![Diagram of npm dist-tag promotion of modules](assets/promote.png)
+
+Warehouse.ai builds are an interaction between multiple smaller microservices to garantuee high
+concurrency and stability.
+
+![Diagram of build workflow of a single package in Warehouse.ai](assets/build.png)
 
 **NOTE** In order to publish to warehouse you must add the following to your
 `.npmrc`. Authorization information is stubbed to let the `npm` client itself
@@ -76,6 +82,18 @@ actually make the publish request and not just throw an error before it even tri
 npm c set strict-ssl false
 ```
 
+### Automatically build dependents
+
+> Stability: 2 – **Stable**
+
+After a package build completes Warehouse.ai will query the database for any dependant modules. Builds for
+dependents will automatically be queued. The system will walk all the way up the dependency tree until it
+reaches a "top-level" package.
+
+**NOTE** Dependent builds are only performed if the dependent has opted for building with Warehouse.ai.
+
+![Diagram of build scheduling of dependent packages](assets/dependent.png)
+
 ### Rolling back to previous versions
 
 > Stability: 2 – **Stable**
@@ -135,7 +153,7 @@ private-dep-3@2.9.8
 
 Then the build the Warehouse returns for your module _will include those dependencies._
 
-![](assets/package-versions.png)
+![Depedency resolution](assets/package-versions.png)
 
 ## API documentation
 
@@ -176,6 +194,11 @@ POST /builds/:pkg                     # Ad-hoc build
 POST /builds/compose                  # Trigger multiple builds
 ```
 
+To use the fingerprinted assets from the CDN the build information can be fetched through the above API endpoints
+or by using [the warehouse.ai-client][client].
+
+![Diagram of web services retrieving CDN metadata of builds](assets/assets.png)
+
 ### Packages API
 ```scala
 GET  /packages/                       # Get information about all packages
@@ -204,7 +227,7 @@ GET /env/:pkg  # Install a package against a specified "environment" (i.e. `dist
 
 The purpose of this section is to document important internals, conventions and patterns used by the Warehouse.
 
-![](assets/component-model.png)
+![Internals and conventions](assets/component-model.png)
 
 ### Data Models
 
@@ -270,3 +293,4 @@ npm test
 
 [carpenterd]: https://github.com/godaddy/carpenterd
 [carpenterd-identify]: https://github.com/godaddy/carpenterd#identification-of-build-system-type
+[client]: https://github.com/warehouseai/warehouse.ai-api-client

assets/diagrams/assets.mmd

@@ -0,0 +1,9 @@
+graph LR
+A(fa:fa-server warehouse.ai)
+B{fa:fa-database No-SQL Database}
+C(fa:fa-lightbulb web service)
+D[fa:fa-laptop visitor browser]
+
+C --> |"serve response with CDN assets"| D
+B --> |"fetch metadata"| A
+C --> |"fetch build CDN metadata - wrhs.ai-client"| A

assets/diagrams/build.mmd

@@ -0,0 +1,15 @@
+graph LR
+A>BUILD]
+B(fa:fa-toolbox carpenterd)
+C{fa:fa-broadcast-tower NSQ}
+D(fa:fa-spinner carpenterd-worker)
+E{fa:fa-box-open S3 compatible CDN}
+F{fa:fa-database No-SQL Database}
+
+A -.-> |"<small>{ NODE_ENV: env, LOCALE: locale }</small>"| B
+B --> |"enqueue build"|C
+D --> |"subscribe"| C
+D --> |"store build output"| E
+D --> |"build metadata"| F
+
+style A fill:#ccf,stroke:#f66,stroke-width:2px,stroke-dasharray: 5, 5

assets/diagrams/dependent.mmd

@@ -0,0 +1,12 @@
+graph LR
+B(fa:fa-server warehouse.ai)
+C{fa:fa-database No-SQL Database}
+D{fa:fa-database npm-like registry}
+E>BUILD]
+
+E -.-> |"trigger dependent builds"| B
+C --> |"fetch dependent metadata"| B
+D --> |"fetch dependent tarball"| B
+B -.-> |"provide { tarball, metadata }"| E
+
+style E fill:#ccf,stroke:#f66,stroke-width:2px,stroke-dasharray: 5, 5

assets/diagrams/promote.mmd

@@ -0,0 +1,12 @@
+graph LR
+A(npm dist-tag add)
+B(fa:fa-server warehouse.ai)
+C{fa:fa-database npm-like registry}
+D>BUILD]
+
+B -.-> |"{NODE_ENV=env, ...metadata}"| D
+C --> |"fetch @version tarball"| B
+B --> |"store dist-tag"| C
+A --> |"module@version env"| B
+
+style D fill:#ccf,stroke:#f66,stroke-width:2px,stroke-dasharray: 5, 5

assets/diagrams/publish.mmd

@@ -0,0 +1,11 @@
+graph LR
+A(npm publish)
+B(fa:fa-server warehouse.ai)
+C{fa:fa-database npm-like registry}
+D>BUILD]
+
+A --> |"publish tarball"| B
+B --> |"store tarball"| C
+B --> |"duplicate tarball"| D
+
+style D fill:#ccf,stroke:#f66,stroke-width:2px,stroke-dasharray: 5, 5

bin/diagrams

@@ -0,0 +1,105 @@
+#!/usr/bin/env node
+
+const canihaz = require('canihaz');
+const path = require('path');
+const fs = require('fs').promises;
+let browser;
+
+const source = path.join(__dirname, '..', 'assets', 'diagrams');
+const template = chart => `
+<!doctype html>
+<html>
+  <head>
+    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.8.1/css/all.min.css">
+    <style>
+      html, * {
+        font-family: 'monaco';
+      }
+
+      .node {
+        text-align: center;
+      }
+      
+      i.fa {
+        display: block;
+        margin: 5px;
+      }
+
+      .edgeLabel:not(:empty) {
+        padding: 0px 8px;
+      }
+    </style>
+  </head>
+  <body>
+    <script src="https://unpkg.com/mermaid/dist/mermaid.min.js" charset="utf-8"></script>
+    <script>
+      const diagram = document.createElement('div');
+
+      window.mermaid.mermaidAPI.initialize({ theme: 'forest' });
+      window.mermaid.mermaidAPI.render('container', \`${chart}\`, svg => {
+        diagram.innerHTML = svg;
+        diagram.id = 'container';
+
+        document.body.appendChild(diagram);
+      });
+    </script>
+  </body>
+</html>
+`;
+
+async function snapshot(browser, file) {
+  try {
+    const chart = await fs.readFile(path.join(source, file), 'utf-8');
+    const output = path.join(source, '..', `${ path.basename(file, '.mmd') }.png`);
+    const page = await browser.newPage();
+
+    console.log(`Rendering ${ file }`);
+
+    await page.setViewport({ width: 1920, height: 1080 });
+    await page.setContent(template(chart), { waitUntil: ['networkidle0'] });
+
+    const clip = await page.$eval('svg', svg => {
+      const box = svg.getBoundingClientRect();
+      return { x: box.left, y: box.top, width: box.width, height: box.height };
+    });
+
+    await page.screenshot({ path: output, clip, omitBackground: true });
+  } catch (error) {
+    throw new Error(`Snapshot of diagram failed: ${ error.message }`);
+  }
+}
+
+console.log('Requesting availability of puppeteer, it will be installed if unavailable.'); // eslint-disable-line
+
+canihaz({
+  key: 'cliDependencies'
+}).puppeteer(async function puppetmaster(error, puppeteer) {
+  if (error) throw error;
+  const input = process.argv.slice(2);
+
+  try {
+    browser = await puppeteer.launch();
+
+    const diagrams = input.length
+      ? input.map(file => path.extname(file) === '.mmd' ? file : `${ file }.mmd`)
+      : await fs.readdir(source);
+
+    for (const file of diagrams) {
+      if (path.extname(file) === '.mmd') await snapshot(browser, file);
+    }
+
+    await browser.close();
+
+    console.log('Chart images generated.'); // eslint-disable-line
+  } catch (error) {
+    throw new Error(`Puppeteer failed: ${ error.message }`);
+  }
+});
+
+//
+// Ensure Chromium process is closed.
+//
+process.once('SIGINT', async function kill() {
+  if (!browser) process.exit(); // eslint-disable-line
+  await browser.close();
+});