Skip to content
This repository has been archived by the owner on Oct 26, 2023. It is now read-only.

Commit

Permalink
📖 improve readme
Browse files Browse the repository at this point in the history
  • Loading branch information
markysoft committed May 29, 2018
1 parent 07b8769 commit 828d940
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 1 deletion.
23 changes: 22 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,30 @@

## ETL Store

The `etlStore` manages the state of the ETL including IDs, loaded records, and a list of errored IDs.
The `EtlStore` manages the state of the ETL including IDs, loaded records, and a list of errored IDs.
The store can persist state to the local file system during queue processing enabling an ETL to continue after interruption.

The `EtlStore` is a class and can be constructed as below:
```
const etlStore = new EtlStore({ idKey, log, outputFile);
```
Where `idKey` is the property on the object by which it is uniquely identified, i.e. `id`;
`log` is a logging framework that has `info` and `error` functions.
`outputFile` is the name of the file the EltStore will write to, i.e. `pharmacy-data`;

Optional config settings are:

`outputDir`: local output folder. Defaults to `output`

`cacheIdFilename`: local ID cache file. Defaults to `cache-ids.json`

`cacheDataFilename`: local data cache file. Defaults to `cache-data.json`

`summaryFilename`: summary file name. Defaults to `summary.json`

When the `saveRecords` function is called, items in the store will be saved to `{outputFile}.json`, and IDs will be saved to
`{outputFile}-seed-ids.json`.

## Environment variables

Environment variables are expected to be managed by the environment in which
Expand Down
1 change: 1 addition & 0 deletions lib/etlStore.js
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ class EtlStore {
deleteRecord(id) {
delete this.cache[id];
}

getRecords() {
return Object.values(this.cache);
}
Expand Down

0 comments on commit 828d940

Please sign in to comment.