Skip to content

Commit

Permalink
Updated input schema and README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
msudgh committed Mar 2, 2024
1 parent 24a1f79 commit 0631a72
Show file tree
Hide file tree
Showing 4 changed files with 185 additions and 14 deletions.
89 changes: 84 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,27 @@
[![Codecov Status](https://codecov.io/gh/msudgh/sync-cloud-storage/branch/main/graph/badge.svg?token=2BY6063VOY)](https://codecov.io/gh/msudgh/sync-cloud-storage)
[![License](https://img.shields.io/github/license/msudgh/sync-cloud-storage)](LICENSE)

Synchronize files and directories between a remote computer and multiple Serverless cloud providers' storage.
Synchronize files and directories between a remote computer and multiple Serverless cloud providers' storages.

Supported cloud providers:

- [x] AWS S3

## Features

- Uses the latest official cloud provider's SDK
- AWS S3: [`aws-sdk@3.x`](https://www.npmjs.com/package/aws-sdk)
- Sync multiple storages at once by just defining patterns of [`glob`](<https://en.wikipedia.org/wiki/Glob_(programming)>) strings to include or exclude files and directories
- Set the following for each synced file:
- Modern and uses the latest official cloud provider's SDK
- AWS S3: [`aws-sdk@3.x`](https://www.npmjs.com/package/@aws-sdk/client-s3)
- Sync multiple storages at once and flexible file matching (single or multiple file/dir sync) by just defining patterns of [`glob`](<https://en.wikipedia.org/wiki/Glob_(programming)>) to include or exclude
- Set the following configs for each synced file:
- prefix
- access control list (ACL)
- tags
- metadata
- Select a list of specific sync actions for each storage:
- upload
- delete
- Written in **TypeScript** with strong type checking
- Integration test with cloud providers

## Installation

Expand Down Expand Up @@ -56,3 +58,80 @@ custom:
foo: bar
bar: foo
```

## Configuration Reference

This section provides a detailed reference for all configuration options.

- `name`:

- Type: `string`
- Required: `true`
- Minimum length: 1
- Example: `name: assets`

- `patterns`: File patterns to include or exclude during synchronization.

- Type: `array` of `string`
- Required: `true`
- Minimum items: 1
- Example:
```yaml
patterns:
- 'assets/*'
- '!assets/temp/*'
```

- `actions`: List of actions to perform during synchronization.

- Type: `array` of `string`
- Required: `true`
- Valid Values: `upload`, `delete`
- Example:
```yaml
actions:
- upload
- delete
```

- `prefix`: The prefix to apply to all synced files in the bucket.

- Type: `string`
- Required: `false`
- Default: `""`
- Example: `prefix: assets`

- `enabled`: Whether to enable the sync for the storage.

- Type: `boolean`
- Required: `false`
- Default: `true`

- `acl`: Access control list setting for the synced files.

- Type: `string`
- Required: `false`
- Valid Values: `private`, `public-read`, `public-read-write`, `authenticated-read`
- Reference: [AWS S3 Canned ACL](https://docs.aws.amazon.com/AmazonS3/latest/userguide/acl-overview.html#canned-acl)
- Example: `acl: public-read`

- `metadata`: Custom metadata to apply to the synced files.

- Type: `object`
- Required: `false`
- Example:
```yaml
metadata:
foo: bar
bar: foo
```

- `tags`: Custom tags to apply to the synced files.
- Type: `object`
- Required: `false`
- Default: `{}`
- Example:
```yaml
tags:
environment: production
```
106 changes: 99 additions & 7 deletions docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -48,20 +48,20 @@ <h3> </h3>
<a href="./.github/workflows/ci.yml"><img src="https://github.com/msudgh/sync-cloud-storage/actions/workflows/ci.yml/badge.svg?branch=main" alt="Pipeline Status"></a>
<a href="https://codecov.io/gh/msudgh/sync-cloud-storage"><img src="https://codecov.io/gh/msudgh/sync-cloud-storage/branch/main/graph/badge.svg?token=2BY6063VOY" alt="Codecov Status"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/msudgh/sync-cloud-storage" alt="License"></a></p>
<p>Synchronize files and directories between a remote computer and multiple Serverless cloud providers' storage.</p>
<p>Synchronize files and directories between a remote computer and multiple Serverless cloud providers' storages.</p>
<p>Supported cloud providers:</p>
<ul>
<li>[x] AWS S3</li>
</ul>
<h2>Features</h2>
<ul>
<li>Uses the latest official cloud provider's SDK
<li>Modern and uses the latest official cloud provider's SDK
<ul>
<li>AWS S3: <a href="https://www.npmjs.com/package/aws-sdk"><code>aws-sdk@3.x</code></a></li>
<li>AWS S3: <a href="https://www.npmjs.com/package/@aws-sdk/client-s3"><code>aws-sdk@3.x</code></a></li>
</ul>
</li>
<li>Sync multiple storages at once by just defining patterns of <a href="https://en.wikipedia.org/wiki/Glob_(programming)"><code>glob</code></a> strings to include or exclude files and directories</li>
<li>Set the following for each synced file:
<li>Sync multiple storages at once and flexible file matching (single or multiple file/dir sync) by just defining patterns of <a href="https://en.wikipedia.org/wiki/Glob_(programming)"><code>glob</code></a> to include or exclude</li>
<li>Set the following configs for each synced file:
<ul>
<li>prefix</li>
<li>access control list (ACL)</li>
Expand All @@ -75,6 +75,8 @@ <h2>Features</h2>
<li>delete</li>
</ul>
</li>
<li>Written in <strong>TypeScript</strong> with strong type checking</li>
<li>Integration test with cloud providers</li>
</ul>
<h2>Installation</h2>
<ol>
Expand Down Expand Up @@ -102,7 +104,97 @@ <h4>AWS S3</h4>
metadata:
foo: bar
bar: foo
</code></pre></article>
</code></pre>
<h2>Configuration Reference</h2>
<p>This section provides a detailed reference for all configuration options.</p>
<ul>
<li>
<p><code>name</code>:</p>
<ul>
<li>Type: <code>string</code></li>
<li>Required: <code>true</code></li>
<li>Minimum length: 1</li>
<li>Example: <code>name: assets</code></li>
</ul>
</li>
<li>
<p><code>patterns</code>: File patterns to include or exclude during synchronization.</p>
<ul>
<li>Type: <code>array</code> of <code>string</code></li>
<li>Required: <code>true</code></li>
<li>Minimum items: 1</li>
<li>Example:<pre class="prettyprint source lang-yaml"><code>patterns:
- &quot;assets/*&quot;
- &quot;!assets/temp/*&quot;
</code></pre>
</li>
</ul>
</li>
<li>
<p><code>actions</code>: List of actions to perform during synchronization.</p>
<ul>
<li>Type: <code>array</code> of <code>string</code></li>
<li>Required: <code>true</code></li>
<li>Valid Values: <code>upload</code>, <code>delete</code></li>
<li>Example:<pre class="prettyprint source lang-yaml"><code>actions:
- upload
- delete
</code></pre>
</li>
</ul>
</li>
<li>
<p><code>prefix</code>: The prefix to apply to all synced files in the bucket.</p>
<ul>
<li>Type: <code>string</code></li>
<li>Required: <code>false</code></li>
<li>Default: <code>&quot;&quot;</code></li>
<li>Example: <code>prefix: assets</code></li>
</ul>
</li>
<li>
<p><code>enabled</code>: Whether to enable the sync for the storage.</p>
<ul>
<li>Type: <code>boolean</code></li>
<li>Required: <code>false</code></li>
<li>Default: <code>true</code></li>
</ul>
</li>
<li>
<p><code>acl</code>: Access control list setting for the synced files.</p>
<ul>
<li>Type: <code>string</code></li>
<li>Required: <code>false</code></li>
<li>Valid Values: <code>private</code>, <code>public-read</code>, <code>public-read-write</code>, <code>authenticated-read</code></li>
<li>Reference: <a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/acl-overview.html#canned-acl">AWS S3 Canned ACL</a></li>
<li>Example: <code>acl: public-read</code></li>
</ul>
</li>
<li>
<p><code>metadata</code>: Custom metadata to apply to the synced files.</p>
<ul>
<li>Type: <code>object</code></li>
<li>Required: <code>false</code></li>
<li>Example:<pre class="prettyprint source lang-yaml"><code>metadata:
foo: bar
bar: foo
</code></pre>
</li>
</ul>
</li>
<li>
<p><code>tags</code>: Custom tags to apply to the synced files.</p>
<ul>
<li>Type: <code>object</code></li>
<li>Required: <code>false</code></li>
<li>Default: <code>{}</code></li>
<li>Example:<pre class="prettyprint source lang-yaml"><code>tags:
environment: production
</code></pre>
</li>
</ul>
</li>
</ul></article>
</section>


Expand All @@ -119,7 +211,7 @@ <h2><a href="index.html">Home</a></h2>
<br class="clear">

<footer>
Documentation generated by <a href="https://github.com/jsdoc/jsdoc">JSDoc 4.0.2</a> on Sat Mar 02 2024 22:44:16 GMT+0100 (Central European Standard Time)
Documentation generated by <a href="https://github.com/jsdoc/jsdoc">JSDoc 4.0.2</a> on Sat Mar 02 2024 23:22:23 GMT+0100 (Central European Standard Time)
</footer>

<script> prettyPrint(); </script>
Expand Down
2 changes: 1 addition & 1 deletion src/providers/s3/buckets.ts
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,7 @@ export const syncTags = async (
}
}

const mergedTagSet = mergeTags(existingTagSet.TagSet, storage.tags ?? {})
const mergedTagSet = mergeTags(existingTagSet.TagSet, storage.tags)

await client.send(
new PutBucketTaggingCommand({
Expand Down
2 changes: 1 addition & 1 deletion src/schemas/input.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ const objectCannedACLs = Object.values(ObjectCannedACL).map(
const tags = z.record(z.string(), z.string())

const storage = z.object({
name: z.string(),
name: z.string().min(1),
patterns: z.array(z.string()).min(1),
actions: z.array(z.string()).default(['upload', 'delete']),
prefix: z.string().default(''),
Expand Down

0 comments on commit 0631a72

Please sign in to comment.