Skip to content
Permalink
Browse files

Merge pull request #54 from godaddy/charts

Charts
  • Loading branch information...
Swaagie committed May 9, 2019
2 parents 262eca5 + 77670a9 commit 5823169bdbadb73d422e90dde081c0e285ee32dd
@@ -1,5 +1,10 @@
# CHANGELOG

- [#54] Generate diagrams
- Include `canihaz` to optionally install `puppeteer`.
- Provide script `bin/diagrams` to render `mermaid` diagrams and store snapshots to file
- Provide npm command to execute above script
- Reference diagrams in documentation.
- [#53] Give credits for Github templates
- [#52] Default documenation
- Add: `CONTRUBUTING.md`, `CHANGELOG.md`, `SECURITY.md`
@@ -8,3 +13,4 @@

[#52]: https://github.com/godaddy/warehouse.ai/pull/52
[#53]: https://github.com/godaddy/warehouse.ai/pull/53
[#54]: https://github.com/godaddy/warehouse.ai/pull/54
@@ -12,8 +12,6 @@ The goal of the Warehouse is to support modular UI development by:

In other words the Warehouse is designed to give as many programmatic guarantees that it is safe to "always be on latest" and make rolling back as painless as possible when issues arise.

![](assets/workflow.png)

- [Developer Experience](#developer-experience)
- [Releasing code](#releasing-code)
- [Rolling back to previous versions](#rolling-back-to-previous-versions)
@@ -53,11 +51,19 @@ The release process for any module using the Warehouse is:
cd /path/to/my/front-end/module
npm publish
```
![Diagram of npm publish workflow to Warehouse.ai](assets/publish.png)

3. Perform any manual QA in your DEV environment.
4. Promote the `module@version` to production using `npm dist-tag add`
```
npm dist-tag add module@version prod
```
![Diagram of npm dist-tag promotion of modules](assets/promote.png)

Warehouse.ai builds are an interaction between multiple smaller microservices to garantuee high
concurrency and stability.

![Diagram of build workflow of a single package in Warehouse.ai](assets/build.png)

**NOTE** In order to publish to warehouse you must add the following to your
`.npmrc`. Authorization information is stubbed to let the `npm` client itself
@@ -76,6 +82,18 @@ actually make the publish request and not just throw an error before it even tri
npm c set strict-ssl false
```

### Automatically build dependents

> Stability: 2 – **Stable**
After a package build completes Warehouse.ai will query the database for any dependant modules. Builds for
dependents will automatically be queued. The system will walk all the way up the dependency tree until it
reaches a "top-level" package.

**NOTE** Dependent builds are only performed if the dependent has opted for building with Warehouse.ai.

![Diagram of build scheduling of dependent packages](assets/dependent.png)

### Rolling back to previous versions

> Stability: 2 – **Stable**
@@ -135,7 +153,7 @@ private-dep-3@2.9.8

Then the build the Warehouse returns for your module _will include those dependencies._

![](assets/package-versions.png)
![Depedency resolution](assets/package-versions.png)

## API documentation

@@ -176,6 +194,11 @@ POST /builds/:pkg # Ad-hoc build
POST /builds/compose # Trigger multiple builds
```

To use the fingerprinted assets from the CDN the build information can be fetched through the above API endpoints
or by using [the warehouse.ai-client][client].

![Diagram of web services retrieving CDN metadata of builds](assets/assets.png)

### Packages API
```scala
GET /packages/ # Get information about all packages
@@ -204,7 +227,7 @@ GET /env/:pkg # Install a package against a specified "environment" (i.e. `dist

The purpose of this section is to document important internals, conventions and patterns used by the Warehouse.

![](assets/component-model.png)
![Internals and conventions](assets/component-model.png)

### Data Models

@@ -270,3 +293,4 @@ npm test

[carpenterd]: https://github.com/godaddy/carpenterd
[carpenterd-identify]: https://github.com/godaddy/carpenterd#identification-of-build-system-type
[client]: https://github.com/warehouseai/warehouse.ai-api-client
BIN +35.8 KB assets/assets.png
Binary file not shown.
BIN +51.1 KB assets/build.png
Binary file not shown.
BIN +42.5 KB assets/dependent.png
Binary file not shown.
@@ -0,0 +1,9 @@
graph LR
A(fa:fa-server warehouse.ai)
B{fa:fa-database No-SQL Database}
C(fa:fa-lightbulb web service)
D[fa:fa-laptop visitor browser]

C --> |"serve response with CDN assets"| D
B --> |"fetch metadata"| A
C --> |"fetch build CDN metadata - wrhs.ai-client"| A
@@ -0,0 +1,15 @@
graph LR
A>BUILD]
B(fa:fa-toolbox carpenterd)
C{fa:fa-broadcast-tower NSQ}
D(fa:fa-spinner carpenterd-worker)
E{fa:fa-box-open S3 compatible CDN}
F{fa:fa-database No-SQL Database}

A -.-> |"<small>{ NODE_ENV: env, LOCALE: locale }</small>"| B
B --> |"enqueue build"|C
D --> |"subscribe"| C
D --> |"store build output"| E
D --> |"build metadata"| F

style A fill:#ccf,stroke:#f66,stroke-width:2px,stroke-dasharray: 5, 5
@@ -0,0 +1,12 @@
graph LR
B(fa:fa-server warehouse.ai)
C{fa:fa-database No-SQL Database}
D{fa:fa-database npm-like registry}
E>BUILD]

E -.-> |"trigger dependent builds"| B
C --> |"fetch dependent metadata"| B
D --> |"fetch dependent tarball"| B
B -.-> |"provide { tarball, metadata }"| E

style E fill:#ccf,stroke:#f66,stroke-width:2px,stroke-dasharray: 5, 5
@@ -0,0 +1,12 @@
graph LR
A(npm dist-tag add)
B(fa:fa-server warehouse.ai)
C{fa:fa-database npm-like registry}
D>BUILD]

B -.-> |"{NODE_ENV=env, ...metadata}"| D
C --> |"fetch @version tarball"| B
B --> |"store dist-tag"| C
A --> |"module@version env"| B

style D fill:#ccf,stroke:#f66,stroke-width:2px,stroke-dasharray: 5, 5
@@ -0,0 +1,11 @@
graph LR
A(npm publish)
B(fa:fa-server warehouse.ai)
C{fa:fa-database npm-like registry}
D>BUILD]

A --> |"publish tarball"| B
B --> |"store tarball"| C
B --> |"duplicate tarball"| D

style D fill:#ccf,stroke:#f66,stroke-width:2px,stroke-dasharray: 5, 5
BIN +35.5 KB assets/promote.png
Binary file not shown.
BIN +24.9 KB assets/publish.png
Binary file not shown.
BIN -104 KB assets/workflow.png
Binary file not shown.
@@ -0,0 +1,105 @@
#!/usr/bin/env node

const canihaz = require('canihaz');
const path = require('path');
const fs = require('fs').promises;
let browser;

const source = path.join(__dirname, '..', 'assets', 'diagrams');
const template = chart => `
<!doctype html>
<html>
<head>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.8.1/css/all.min.css">
<style>
html, * {
font-family: 'monaco';
}
.node {
text-align: center;
}
i.fa {
display: block;
margin: 5px;
}
.edgeLabel:not(:empty) {
padding: 0px 8px;
}
</style>
</head>
<body>
<script src="https://unpkg.com/mermaid/dist/mermaid.min.js" charset="utf-8"></script>
<script>
const diagram = document.createElement('div');
window.mermaid.mermaidAPI.initialize({ theme: 'forest' });
window.mermaid.mermaidAPI.render('container', \`${chart}\`, svg => {
diagram.innerHTML = svg;
diagram.id = 'container';
document.body.appendChild(diagram);
});
</script>
</body>
</html>
`;

async function snapshot(browser, file) {
try {
const chart = await fs.readFile(path.join(source, file), 'utf-8');
const output = path.join(source, '..', `${ path.basename(file, '.mmd') }.png`);
const page = await browser.newPage();

console.log(`Rendering ${ file }`);

await page.setViewport({ width: 1920, height: 1080 });
await page.setContent(template(chart), { waitUntil: ['networkidle0'] });

const clip = await page.$eval('svg', svg => {
const box = svg.getBoundingClientRect();
return { x: box.left, y: box.top, width: box.width, height: box.height };
});

await page.screenshot({ path: output, clip, omitBackground: true });
} catch (error) {
throw new Error(`Snapshot of diagram failed: ${ error.message }`);
}
}

console.log('Requesting availability of puppeteer, it will be installed if unavailable.'); // eslint-disable-line

canihaz({
key: 'cliDependencies'
}).puppeteer(async function puppetmaster(error, puppeteer) {
if (error) throw error;
const input = process.argv.slice(2);

try {
browser = await puppeteer.launch();

const diagrams = input.length
? input.map(file => path.extname(file) === '.mmd' ? file : `${ file }.mmd`)
: await fs.readdir(source);

for (const file of diagrams) {
if (path.extname(file) === '.mmd') await snapshot(browser, file);
}

await browser.close();

console.log('Chart images generated.'); // eslint-disable-line
} catch (error) {
throw new Error(`Puppeteer failed: ${ error.message }`);
}
});

//
// Ensure Chromium process is closed.
//
process.once('SIGINT', async function kill() {
if (!browser) process.exit(); // eslint-disable-line
await browser.close();
});
Oops, something went wrong.

0 comments on commit 5823169

Please sign in to comment.
You can’t perform that action at this time.