meta-documentation (docs for docs & doc generators)

.bookignore

@@ -1,4 +1,4 @@
-bin/
+bin/*.js
 src/
 web/
 

README.md

@@ -4,13 +4,70 @@
 
 Documentation for Tidepool's data models.
 
+Jump to...
+
+ - [diabetes device data](./device-data/README.md 'Diabetes device data')
+
+## meta-documentation (documentation for this documentation toolset)
+
 > (This and all Markdown documents in [this repository](https://github.com/tidepool-org/data-model 'Tidepool on GitHub: data-model') are used to generate [the Tidepool data model documentation website](http://developer.tidepool.io/data-model/ 'Tidepool data model documentation') via [GitBook](https://github.com/GitbookIO/gitbook 'GitHub: GitBook').
 
-> If you want to serve yourself the documentation locally, after cloning this repository, you can:
+The developer microsite has [more information on Tidepool's use of GitBook for documentation and recommended workflows](http://developer.tidepool.io/docs/ 'Tidepool Developer Portal: Docs').
 
-> 1. install the GitBook CLI with the command `npm install -g gitbook-cli`
-> 1. then run `gitbook serve` to serve the docs at [http://localhost:4000](http://localhost:4000).)
+After downloading or cloning this repository, if you want to serve yourself the documentation locally, first install the required dependencies with `npm install` (assuming you have [Node.js](https://nodejs.org/en/ 'Node.js')) and [the node package manager (`npm`)](https://www.npmjs.com/ 'npm homepage') installed already). After installing the required dependencies, you may use either `npm start` or `npm run serve-docs` to serve the docs at [http://localhost:4000](http://localhost:4000).
 
-Jump to...
+### editing documents
 
- - [diabetes device data](./device-data/README.md 'Diabetes device data')
+Only certain sections of most of the Markdown files in this repository should be edited *directly*. Most of the content for the data model documentation here is generated through the JavaScript code found in the `src/` directory.
+
+Specifically, each .md document or directory of .md documents under `device-data/` is generated automatically unless marked with `<!-- non-generated document! all areas editable -->` at the top.
+
+In the automatically-generated docs, the following three areas should **not** be edited, as any changes will be steamrolled the next time that the doc is regenerated using the [document template generator](./bin/docTemplateGenerator.md).
+
+1) The title, note about required fields, and links to jump to example JSON. For example:
+
+```
+## Coffee (coffee)
+
+**NB:** All fields are *required* unless otherwise noted.
+
+
+> Jump to example JSON:
+
+>  - [client example](#example-client)
+>  - [ingestion example](#example-ingestion)
+>  - [storage example](#example-storage)
+```
+
+2) The sub-header for each field through to the starting comment for "color commentary," including the "quick summary". For example:
+
+```
+### type
+
+[ingestion, storage, client] The string `coffee`.
+
+  QUICK SUMMARY
+  Required:
+    jellyfish: yes
+    platform: yes
+
+<!-- start type -->
+```
+
+3) The JSON examples at the bottom of the document, starting with the first header for these—`### example (client)`.
+
+*The **only** area that can (and should!) be edited by hand is the "color commentary" space between the `<!-- start $FIELD -->` and `<!-- end $FIELD -->` comments within the section for each field. On initial generation of the documentation template, this space will contain a `<!-- TODO -->` comment that **must** be deleted in order for other changes in this space to be preserved. (The lack of the `<!-- TODO -->` comment is what signals the script to look for and preserve the content between the `<!-- start $FIELD -->` and `<!-- end $FIELD -->` comments.)*
+
+For data types with sub-types, the README in the directory for the type is also an automatically generated document. Only the space between the header and the `<!-- end intro -->` comment should be edited by hand. See [the introduction to the `bolus` type](https://raw.githubusercontent.com/tidepool-org/data-model/master/device-data/types/bolus/README.md) for an example.
+
+### (re)generating documents
+
+There is a bash script in this repository to generate new and regenerate old or regenerate (after updates to the source) all of the data model documents.
+
+The script is intended to be run from the root of this repository, and it requires no arguments, simply:
+
+```bash
+$ ./regenerate-all-docs.sh
+```
+
+When you regenerate docs that have been generated before, there will *always* be changes in the resulting Markdown document because the JSON examples in various formats for each data type are regenerated on every run. This is *intentional*: the doc generation script cannot tell if anything has changed in the source code that generates the examples, and we want to ensure that the examples get updated if something has! (If a field or value has been renamed, to give a simple example.) It is recommended to use a workflow like `git add -p` (to review "chunks" of the `git diff` and stage them for commitment interactively) after regenerating all of the docs in order to discard unneeded updates (like updates to the JSON examples if you're *absolutely* certain they are unnecessary) while keeping those that are needed. After committing, use `git checkout -- path/to/docs/` to clear the changes you felt were superfluous.

SUMMARY.md

@@ -31,3 +31,6 @@
     * [smbg](device-data/types/smbg.md)
     * [upload](device-data/types/upload.md)
     * [wizard](device-data/types/wizard.md)
+* [tools](bin/README.md)
+  * [doc template generator](bin/docTemplateGenerator.md)
+  * [sample datum generator](bin/sampleDatumGenerator.md)

bin/README.md

@@ -0,0 +1,7 @@
+# data model tools
+
+This repository includes two command-line tools: a document template generator and a sample datum generator. The first tool is only relevant and useful internal to Tidepool: it is only for those of us adding or editing new data model docs. The sample datum generator, on the other hand, may also be useful for third parties who are learning about Tidepool's data model(s). The sample generator can generate single example datums for any type in Tidepool's documented data model(s) in both the format necessary for `ingestion` of data (useful for third parties wishing to build apps that upload data directly into the Tidepool cloud) and in the format that `client` applications will see after requesting data from the Tidepool cloud. (In addition, it is possible to generate a sample datum in `storage` format, but that is again mostly only of interest to Tidepool's own engineering team(s).)
+
+For more information about each of these tools, see the following pages:
+- [the doc template generator](./docTemplateGenerator.md)
+- [the sample datum generator](./sampleDatumGenerator.md)

bin/docTemplateGenerator.md

@@ -0,0 +1,34 @@
+# the doc template generator
+
+This command-line tool generates or regenerates Markdown files for types or sub-types in the Tidepool data model(s). At present, it is designed to be run from within the `bin/` directory, although it is on @jebeck's list of things to do to make it more flexible. As with any (good...) command-line tool, you can pass it the argument `--help` to get information about the arguments it takes and their syntax. The tool itself is a node script, but it is executable, so it can be run just like any bash script:
+
+```bash
+$ ./docTemplateGenerator.js --help
+```
+
+The present usage (although always use `--help` in case this meta-doc hasn't been updated!) is:
+```
+  Usage: docTemplateGenerator [options] <type>
+
+  Options:
+
+    -h, --help                    output usage information
+    -p, --path <value>            path to docs directory
+    --subtype, --subType <value>  subType for this datum
+```
+
+The bash script `regenerate-all-docs.sh` employs the doc template generator to regenerate all the Markdown files for all data types. Tidepool developers are encouraged to use this script and discard any unnecessary changes to ensure that *everything* that should be updated gets updated. See the section on [regenerating documents](../README.md#regenerating-documents) in the introduction for more details.
+
+## examples
+
+To regenerate the Markdown file for the `wizard` data type:
+
+```bash
+$ ./docTemplateGenerator.js wizard
+```
+
+To regenerate just the Markdown file documenting a `basal` with a `deliveryType` of `temp`:
+
+```bash
+$ ./docTemplateGenerator.js --subtype temp basal
+```

bin/sampleDatumGenerator.js

@@ -8,7 +8,6 @@
     -c, --clipboard               copy resulting sample JSON to clipboard
     -d, --datetime <value>        ISO-formatted datetime for datum
     -f, --format <value>          data format of client, ingestion, storage
-    -m, --manufacturer <value>    manufacturer of diabetes device
     --subtype, --subType <value>  subType for this datum
 */
 
@@ -42,7 +41,6 @@ commander.usage('[options] <type>')
   .option('-c, --clipboard', 'copy resulting sample JSON to clipboard')
   .option('-d, --datetime <value>', 'ISO-formatted datetime for datum', new Date().toISOString())
   .option('-f, --format <value>', 'data format of client, ingestion, storage', /^(client|ingestion|storage)$/i, 'client')
-  .option('-m, --manufacturer <value>', 'manufacturer of diabetes device', null)
   .option('--subtype, --subType <value>', 'subType for this datum', null)
   .parse(process.argv);
 
@@ -100,9 +98,6 @@ console.log(chalk.gray('Type:'), type);
 if (commander.subType) {
   console.log(chalk.gray('Subtype:'), commander.subType);
 }
-if (commander.manufacturer) {
-  console.log(chalk.gray('Manufacturer:'), commander.manufacturer);
-}
 console.log(chalk.gray('Datetime:'), commander.datetime);
 console.log(chalk.gray('Data model format:'), commander.format);
 console.log();

bin/sampleDatumGenerator.md

@@ -0,0 +1,63 @@
+# the sample datum generator
+
+This command-line tool generates sample JSON datums in all the types of the Tidepool data model(s) in a variety of formats, at the user's choice. As with any (good...) command-line tool, you can pass it the argument `--help` to get information about the arguments it takes and their syntax. The tool itself is a node script, but it is executable, so it can be run just like any bash script:
+
+```bash
+$ ./sampleDatumGenerator.js --help
+```
+
+The present usage (although always use `--help` in case this meta-doc hasn't been updated!) is:
+```
+  Usage: sampleDatumGenerator [options] <type>
+
+  Options:
+
+    -h, --help                    output usage information
+    -c, --clipboard               copy resulting sample JSON to clipboard
+    -d, --datetime <value>        ISO-formatted datetime for datum
+    -f, --format <value>          data format of client, ingestion, storage
+    --subtype, --subType <value>  subType for this datum
+```
+
+Or, less concisely...
+
+The only required argument is the `type` of the Tidepool data type, from those listed on the [index page for diabetes data types](../device-data/types/README.md).
+
+Along with this mandatory argument, any of the following optional arguments or flags may appear in any order:
+
+- `-c` or `--clipboard` without an argument to copy the requested sample JSON to the system clipboard
+- `-d` or `--datetime` plus an [ISO-formatted](https://en.wikipedia.org/wiki/ISO_8601 'Wikipedia: ISO 8601') datetime to produce the sample datum at the given datetime
+- `-f` or `--format` plus one of `client`, `ingestion`, or `storage` to specify a format other than the default `client` format
+- `--subtype` or `--subType` plus an appropriate `subType`, `deliveryType` or manufacturer (from `animas`, `insulet`, `medtronic`, or `tandem`), depending on the data type of the sample datum under request
+
+## examples
+
+To generate a sample `smbg` datum (defaults to `client` format):
+
+```bash
+$ ./sampleDatumGenerator.js smbg
+```
+
+To generate a sample `smbg` datum at noon, April Fool's Day, 2016, UTC time:
+
+```
+$ ./sampleDatumGenerator.js smbg -d 2016-04-01T12:00:00+00:00
+```
+
+To do the same *and* copy the result to your operating system's clipboard:
+
+```
+$ ./sampleDatumGenerator.js smbg -d 2016-04-01T12:00:00+00:00 -c
+```
+
+To generate an extended ("square") `bolus`:
+
+```
+./sampleDatumGenerator.js bolus --subType square
+```
+
+To generate `pumpSettings` in the `tandem` format:
+
+```
+./sampleDatumGenerator.js --subtype tandem pumpSettings
+```

device-data/README.md

@@ -1,3 +1,5 @@
+<!-- non-generated document! all areas editable -->
+
 This section documents the diabetes device data types that the Tidepool data ingestion API ([jellyfish](https://github.com/tidepool-org/jellyfish 'Tidepool on GitHub: jellyfish')) reads and stores. All events read and stored by the Tidepool platform use the [JSON](http://www.json.org/ 'JSON.org') data interchange format. All events have a `type` field identifying the subcategory of event. The semantics of the other fields in each subcategory are generally defined individually per subcategory, but there are some common fields.
 
 The page describing these [common fields](./common.md) that apply to all (or at least most) data types is a good place to start.

device-data/out-of-range.md

@@ -1,3 +1,5 @@
+<!-- non-generated document! all areas editable -->
+
 Coming soon!
 
 <!-- TODO: document on out-of-range values, relevant especially to cbg, smbg, and bloodKetone records -->

device-data/types/README.md

@@ -1,12 +1,14 @@
+<!-- non-generated document! all areas editable -->
+
 ## Diabetes Data Types
 
- - [basal insulin](./basal/README.md)
- - [blood ketones](./bloodKetone.md)
- - [bolus insulin](./bolus/README.md)
- - [(bolus) calculator/"wizard" records](./wizard.md)
- - [continuous blood glucose (cbg)](./cbg.md)
- - [CGM settings](./cgmSettings.md)
- - [device events](./deviceEvent/README.md)
- - [insulin pump settings](./pumpSettings.md)
- - [self-monitored blood glucose (smbg)](./smbg.md)
- - [upload metadata](./upload.md)
+ - [`basal` for basal insulin](./basal/README.md)
+ - [`bloodKetone` for blood ketones](./bloodKetone.md)
+ - [`bolus` for bolus insulin](./bolus/README.md)
+ - [`wizard` for (bolus) calculator/"wizard" records](./wizard.md)
+ - [`cbg` for continuous blood glucose (cbg)](./cbg.md)
+ - [`cgmSettings` for CGM settings](./cgmSettings.md)
+ - [`deviceEvent` for miscellaneous other device events](./deviceEvent/README.md)
+ - [`pumpSettings` for insulin pump settings](./pumpSettings.md)
+ - [`smbg` for self-monitored blood glucose (smbg)](./smbg.md)
+ - [`upload` for upload metadata](./upload.md)

device-data/types/basal/previous.md

@@ -1,3 +1,5 @@
+<!-- non-generated document! all areas editable -->
+
 ## Uploading sequences of `basal` events via jellyfish's `previous` field
 
 The legacy jellyfish data ingestion API was designed for *real-time* ingestion of diabetes device data. This posed a problem for ingesting events like `basal` events that are *not* point-in-time but rather represent intervals of time. There are a couple of obvious solutions to this problem:

device-data/types/basal/suppressed.md

@@ -1,3 +1,5 @@
+<!-- non-generated document! all areas editable -->
+
 Coming soon!
 
 <!-- TODO: document how `suppressed` works when a `temp` or `suspend` basal crosses > 1 segment of a basal schedule -->

device-data/types/deviceEvent/previous.md

@@ -1,3 +1,5 @@
+<!-- non-generated document! all areas editable -->
+
 Coming soon!
 
 <!-- TODO: document status.previous! -->

device-data/units.md

@@ -1,3 +1,5 @@
+<!-- non-generated document! all areas editable -->
+
 ## Units
 
 In general, the Tidepool platform does not mutate data. Rather, data is read from diabetes devices (or other sources of diabetes device data) and stored in a form as close as possible to the original form while still achieving the translation into the standardized Tidepool data model(s).