Skip to content

Commit

Permalink
feat: add x-origin property (#147)
Browse files Browse the repository at this point in the history
  • Loading branch information
@aeworxet
aeworxet committed Apr 19, 2024
1 parent 1b41aad commit e47ee49
Show file tree
Hide file tree
Showing 21 changed files with 5,134 additions and 10,802 deletions.
45 changes: 24 additions & 21 deletions API.md
View file
Expand Up @@ -75,25 +75,26 @@ console.log(document.string()); // get JSON string

| Param | Type | Description |
| --- | --- | --- |
| files | <code>Array.&lt;string&gt;</code> | <p>Array of stringified AsyncAPI documents in YAML format, that are to be bundled (or array of filepaths, resolved and passed via <code>Array.map()</code> and <code>fs.readFileSync</code>, which is the same, see <code>README.md</code>).</p> |
| files | <code>string</code> \| <code>Array.&lt;string&gt;</code> | <p>One or more relative/absolute paths to AsyncAPI Documents that should be bundled.</p> |
| [options] | <code>Object</code> | |
| [options.base] | <code>string</code> \| <code>object</code> | <p>Base object whose properties will be retained.</p> |
| [options.referenceIntoComponents] | <code>boolean</code> | <p>Pass <code>true</code> to resolve external references to components.</p> |
| [options.baseDir] | <code>string</code> | <p>Pass folder path to</p> |
| [options.base] | <code>string</code> | <p>One relative/absolute path to base object whose properties will be retained.</p> |
| [options.baseDir] | <code>string</code> | <p>One relative/absolute path to directory relative to which paths to AsyncAPI Documents that should be bundled will be resolved.</p> |
| [options.xOrigin] | <code>boolean</code> | <p>Pass <code>true</code> to generate properties <code>x-origin</code> that will contain historical values of dereferenced <code>$ref</code>s.</p> |

**Example**
**TypeScript**
```ts
import { readFileSync, writeFileSync } from 'fs';
import { writeFileSync } from 'fs';
import bundle from '@asyncapi/bundler';

async function main() {
const document = await bundle([readFileSync('./main.yaml', 'utf-8')], {
referenceIntoComponents: true,
});
const document = await bundle(['social-media/comments-service/main.yaml'], {
baseDir: 'example-data',
xOrigin: true,
});

console.log(document.yml()); // the complete bundled AsyncAPI document
writeFileSync('asyncapi.yaml', document.yml()); // the complete bundled AsyncAPI document
console.log(document.yml()); // the complete bundled AsyncAPI document
writeFileSync('asyncapi.yaml', document.yml()); // the complete bundled AsyncAPI document
}

main().catch(e => console.error(e));
Expand All @@ -103,14 +104,15 @@ main().catch(e => console.error(e));
```js
'use strict';

const { readFileSync, writeFileSync } = require('fs');
const { writeFileSync } = require('fs');
const bundle = require('@asyncapi/bundler');

async function main() {
const document = await bundle([readFileSync('./main.yaml', 'utf-8')], {
referenceIntoComponents: true,
});
writeFileSync('asyncapi.yaml', document.yml());
const document = await bundle(['social-media/comments-service/main.yaml'], {
baseDir: 'example-data',
xOrigin: true,
});
writeFileSync('asyncapi.yaml', document.yml());
}

main().catch(e => console.error(e));
Expand All @@ -120,15 +122,16 @@ main().catch(e => console.error(e));
```js
'use strict';

import { readFileSync, writeFileSync } from 'fs';
import { writeFileSync } from 'fs';
import bundle from '@asyncapi/bundler';

async function main() {
const document = await bundle([readFileSync('./main.yaml', 'utf-8')], {
referenceIntoComponents: true,
});
writeFileSync('asyncapi.yaml', document.yml());
const document = await bundle(['social-media/comments-service/main.yaml'], {
baseDir: 'example-data',
xOrigin: true,
});
writeFileSync('asyncapi.yaml', document.yml());
}

main().catch(e => console.error(e));
main().catch(e => console.error(e));
```
215 changes: 106 additions & 109 deletions README.md
View file
Expand Up @@ -11,14 +11,20 @@
- [Overview](#overview)
- [Installation](#installation)
- [Usage](#usage)
* [Resolving external references into components](#resolving-external-references-into-components)
* [Dereference of the external references](#dereference-of-the-external-references)
* [Option `baseDir`](#option-basedir)
* [Property `x-origin`](#property-x-origin)
* [Movement of components to `components`](#movement-of-components-to-components)
* [Code examples](#code-examples)
- [bundle(files, [options])](#bundlefiles-options)
- [Contributors](#contributors)

<!-- tocstop -->

## Overview
An official library that lets you bundle/merge your specification files into one. AsyncAPI Bundler can help you if:
An official library that lets you bundle/dereference or merge into one your AsyncAPI Documents.

AsyncAPI Bundler can help you if:

<details>
<summary>your specification file is divided into different smaller files and is using JSON `$ref` property to reference components </summary>
Expand Down Expand Up @@ -166,121 +172,105 @@ AsyncAPI Bundler can be easily used within your JavaScript projects as a Node.js
```js
'use strict';

const { readFileSync, writeFileSync } = require('fs');
const { writeFileSync } = require('fs');
const bundle = require('@asyncapi/bundler');

async function main() {
const filePaths = ['./camera.yml','./audio.yml'];
const document = await bundle(
filePaths.map(filePath => readFileSync(filePath, 'utf-8')), {
base: readFileSync('./base.yml', 'utf-8'),
}
);

console.log(document.yml()); // the complete bundled AsyncAPI document
writeFileSync('asyncapi.yaml', document.yml()); // the complete bundled AsyncAPI document
const document = await bundle(['social-media/comments-service/main.yaml'], {
baseDir: 'example-data',
xOrigin: true,
});
if (document.yml()) {
console.log(document.yml()); // the complete bundled AsyncAPI document
writeFileSync('asyncapi.yaml', document.yml()); // the complete bundled AsyncAPI document
}
}

main().catch(e => console.error(e));
```

### Resolving external references into components
You can resolve external references by moving them to Messages Object, under `components/messages`.
### Dereference of the external references

<details>
<summary>For example</summary>
`Bundler` dereferences the provided AsyncAPI Document to the maximum possible extent, leaving intact only those internal references that MUST be `Reference Object`s according to the AsyncAPI Specification (thus, should never be dereferenced):

```yml
# main.yaml
asyncapi: 2.5.0
info:
title: Account Service
version: 1.0.0
description: This service is in charge of processing user signups
channels:
user/signedup:
subscribe:
message:
$ref: './messages.yaml#/messages/UserSignedUp'
test:
subscribe:
message:
$ref: '#/components/messages/TestMessage'
components:
messages:
TestMessage:
payload:
type: string
- AsyncAPI Specification v2.6.0

There are no internal references that MUST be `Reference Object`s.

# messages.yaml
messages:
UserSignedUp:
payload:
type: object
properties:
displayName:
type: string
description: Name of the user
email:
type: string
format: email
description: Email of the user
UserLoggedIn:
payload:
type: object
properties:
id: string
- AsyncAPI Specification v3.0.0

# After combining
# asyncapi.yaml
asyncapi: 2.5.0
info:
title: Account Service
version: 1.0.0
description: This service is in charge of processing user signups
channels:
user/signedup:
subscribe:
message:
$ref: '#/components/messages/UserSignedUp'
test:
subscribe:
message:
$ref: '#/components/messages/TestMessage'
components:
messages:
TestMessage:
payload:
type: string
UserSignedUp:
payload:
type: object
properties:
displayName:
type: string
description: Name of the user
email:
type: string
format: email
description: Email of the user
Regexes of internal references that MUST be `Reference Object`s:

```
</details>
<br />
/#\/channels\/,*\/servers/
/#\/operations\/.*\/channel/
/#\/operations\/.*\/messages/
/#\/operations\/.*\/reply\/channel/
/#\/operations\/,*\/reply\/messages/
/#\/components\/channels\/.*\/servers/
/#\/components\/operations\/.*\/channel/
/#\/components\/operations\/.*\/messages/
/#\/components\/operations\/.*\/reply\/channel/
/#\/components\/operations\/.*\/reply\/messages/
```


### Option `baseDir`

Option `baseDir` represents the main working directory of the program, "root directory," relatively to which will be resolved all paths of AsyncAPI Documents passed to the `Bundler`.

Starting from `Bundler` v0.5.0, option `baseDir` is reimplemented with changed logic, and `Bundler` accepts only **paths** of AsyncAPI Documents, which will be read with `readFileSync()` internally.

In a nutshell, the process looks like this:

- Paths of AsyncAPI Documents are passed as `'main.yaml'` | `'./main.yaml'` | `'../main.yaml'` | `['./main.yaml']` | `['main.yaml', 'audio.yaml']`, etc.

- Path/paths are assured to have an `Array` type with `Array.from()` to make them iterable.

- Working directory of the program is changed to the `baseDir` with `process.chdir()`.

- **And only then** are the paths of the AsyncAPI Documents starting to be read from the array the are currently in, one by one, resolving paths and `$ref`s relatively to the `baseDir`.

Take a look at `./example/bundle-cjs.cjs`, which demonstrates working with `baseDir` and `$ref`s of different levels of nesting.


### Property `x-origin`

Property `x-origin` is used for origin tracing in `Bundler` and component naming in `Optimizer`.

It originated from [this comment](https://github.com/asyncapi/bundler/issues/97#issuecomment-1330501758) in a year-long discussion:

> The $ref usually also carries a semantical meaning to understand easier what it is (example "$ref : financial-system.yaml#/components/schemas/bankAccountIdentifier"). If the bundling just resolves this ref inline, the semantical meaning of the $ref pointer gets lost and cannot be recovered in later steps. The optimizer would need to invent an artificial component name for the "bankAccountIdentifier" when moving it to the components section.
Thus, property `x-origin` contains historical values of dereferenced `$ref`s, which are also used by `Optimizer` to give meaningful names to components it moves through the AsyncAPI Document.

However, if a user doesn't need / doesn't want `x-origin` properties to be present in the structure of the AsyncAPI Document (values of the `x-origin` property may leak internal details about how the system described by the AsyncAPI Document is structured,) they can pass `{ xOrigin: false }` (or omit passing `xOrigin` at all) to the `Bundler` in the options object.


### Movement of components to `components`

The movement of all AsyncAPI Specification-valid components to the `components` section of the AsyncAPI Document starting from `Bundler` v0.5.0 is done by the [`Optimizer`](https://github.com/asyncapi/optimizer) v1.0.0+.

To get in CI/code an AsyncAPI Document, that is dereferenced [to its maximum possible extent](#dereference-of-the-external-references) with all of its components moved to the `components` section, the original AsyncAPI Document must be run through chain `Bundler -> Optimizer`.

If `Optimizer` is not able to find `x-origin` properties during optimization of the provided AsyncAPI Document, the existing names of components are used as a fallback mechanism, but keep in mind that components' names may lack semantic meaning in this case.


### Code examples

**TypeScript**
```ts
import { readFileSync, writeFileSync } from 'fs';
import { writeFileSync } from 'fs';
import bundle from '@asyncapi/bundler';

async function main() {
const document = await bundle([readFileSync('./main.yaml', 'utf-8')], {
referenceIntoComponents: true,
const document = await bundle(['social-media/comments-service/main.yaml'], {
baseDir: 'example-data',
xOrigin: true,
});

console.log(document.yml()); // the complete bundled AsyncAPI document
writeFileSync('asyncapi.yaml', document.yml()); // the complete bundled AsyncAPI document
}
if (document.yml()) {
writeFileSync('asyncapi.yaml', document.yml());
}

main().catch(e => console.error(e));
```
Expand All @@ -289,15 +279,17 @@ main().catch(e => console.error(e));
```js
'use strict';

const { readFileSync, writeFileSync } = require('fs');
const { writeFileSync } = require('fs');
const bundle = require('@asyncapi/bundler');

async function main() {
const document = await bundle([readFileSync('./main.yaml', 'utf-8')], {
referenceIntoComponents: true,
const document = await bundle(['social-media/comments-service/main.yaml'], {
baseDir: 'example-data',
xOrigin: true,
});
writeFileSync('asyncapi.yaml', document.yml());
}
if (document.yml()) {
writeFileSync('asyncapi.yaml', document.yml());
}

main().catch(e => console.error(e));
```
Expand All @@ -306,31 +298,36 @@ main().catch(e => console.error(e));
```js
'use strict';

import { readFileSync, writeFileSync } from 'fs';
import { writeFileSync } from 'fs';
import bundle from '@asyncapi/bundler';

async function main() {
const document = await bundle([readFileSync('./main.yaml', 'utf-8')], {
referenceIntoComponents: true,
const document = await bundle(['social-media/comments-service/main.yaml'], {
baseDir: 'example-data',
xOrigin: true,
});
writeFileSync('asyncapi.yaml', document.yml());
}
if (document.yml()) {
writeFileSync('asyncapi.yaml', document.yml());
}

main().catch(e => console.error(e));

```
<a name="bundle"></a>
## bundle(files, [options])
**Kind**: global function
| Param | Type | Description |
| --- | --- | --- |
| files | <code>Array.&lt;string&gt; | Array of stringified AsyncAPI documents in YAML format, that are to be bundled (or array of filepaths, resolved and passed via `Array.map()` and `fs.readFileSync`, which is the same). |
| files | <code>string</code> \| <code>Array.&lt;string&gt;</code> | <p>One or more relative/absolute paths to AsyncAPI Documents that should be bundled.</p> |
| [options] | <code>Object</code> | |
| [options.base] | <code>string</code> \| <code>object</code> | Base object whose properties will be retained. |
| [options.referenceIntoComponents] | <code>boolean<code> | Pass `true` to resolve external references to components. |
| [options.base] | <code>string</code> | <p>One relative/absolute path to base object whose properties will be retained.</p> |
| [options.baseDir] | <code>string</code> | <p>One relative/absolute path to directory relative to which paths to AsyncAPI Documents that should be bundled will be resolved.</p> |
| [options.xOrigin] | <code>boolean</code> | <p>Pass <code>true</code> to generate properties <code>x-origin</code> that will contain historical values of dereferenced <code>$ref</code>s.</p> |
## Contributors
Expand Down
11 changes: 7 additions & 4 deletions example/bundle-cjs.cjs
View file
Expand Up @@ -5,14 +5,17 @@

'use strict';

const { readFileSync, writeFileSync } = require('fs');
const { writeFileSync } = require('fs');
const bundle = require('@asyncapi/bundler');

async function main() {
const document = await bundle([readFileSync('./main.yaml', 'utf-8')], {
referenceIntoComponents: true,
const document = await bundle(['./social-media/comments-service/main.yaml', '../main.yaml'], {
baseDir: 'example-data',
xOrigin: true,
});
writeFileSync('asyncapi.yaml', document.yml());
if (document.yml()) {
writeFileSync('asyncapi.yaml', document.yml());
}
}

main().catch(e => console.error(e));

0 comments on commit e47ee49

Please sign in to comment.