From 377987b7e28ae8ac1330851655dd0f6f8c293034 Mon Sep 17 00:00:00 2001 From: Oscar Polanco Date: Thu, 14 Feb 2019 11:20:07 -0400 Subject: [PATCH] Add basic doc structure using docusaurus (#125) --- .gitignore | 12 + docs/cli.md | 884 ++ website/README.md | 67 + website/core/Footer.js | 69 + website/package-lock.json | 10357 ++++++++++++++++ website/package.json | 16 + website/pages/en/versions.js | 94 + website/sidebars.json | 3 + website/siteConfig.js | 98 + website/static/css/custom.css | 16 + website/static/img/docusaurus.svg | 1 + website/static/img/favicon.png | Bin 0 -> 984 bytes website/static/img/favicon/favicon.ico | Bin 0 -> 9662 bytes website/static/img/oss_logo.png | Bin 0 -> 4370 bytes website/static/index.html | 14 + website/versioned_docs/version-3.9.1/cli.md | 884 ++ .../version-3.9.1-sidebars.json | 3 + website/versions.json | 3 + 18 files changed, 12521 insertions(+) create mode 100644 docs/cli.md create mode 100644 website/README.md create mode 100644 website/core/Footer.js create mode 100644 website/package-lock.json create mode 100644 website/package.json create mode 100644 website/pages/en/versions.js create mode 100644 website/sidebars.json create mode 100644 website/siteConfig.js create mode 100644 website/static/css/custom.css create mode 100644 website/static/img/docusaurus.svg create mode 100644 website/static/img/favicon.png create mode 100644 website/static/img/favicon/favicon.ico create mode 100644 website/static/img/oss_logo.png create mode 100644 website/static/index.html create mode 100644 website/versioned_docs/version-3.9.1/cli.md create mode 100644 website/versioned_sidebars/version-3.9.1-sidebars.json create mode 100644 website/versions.json diff --git a/.gitignore b/.gitignore index 5b9f297..a4e323d 100644 --- a/.gitignore +++ b/.gitignore @@ -38,3 +38,15 @@ jspm_packages # ide .idea + +# docusaurus +.DS_Store + +lib/core/metadata.js +lib/core/MetadataBlog.js + +website/translated_docs +website/build/ +website/yarn.lock +website/node_modules +website/i18n/* diff --git a/docs/cli.md b/docs/cli.md new file mode 100644 index 0000000..5d6111d --- /dev/null +++ b/docs/cli.md @@ -0,0 +1,884 @@ +--- +id: version-3.9.1-cli +title: Claycli +original_id: cli +--- +This is the `Clay` command line utility. Using `claycli` you will have some useful options for your site. Here we will provide you with documentation and examples that will help you to run all the functionalities that you need. + +## Installation + +To install `claycli` you just need to type the following command: +```bash +$ npm install -g claycli +``` +> Note: You will need to install node or yarns >= 8.10 + +## Usage + +If installed globally, call `clay` from the command line with the [command](#commands) and `options` that you need. +```bash +$ clay [options] +``` + +Much like `git`; `claycli` is configured using a [dotfile](https://medium.com/@webprolific/getting-started-with-dotfiles-43c3602fd789) (`.clayconfig`) in your home folder. In it, you may specify references to api keys and `urls / site prefixes` that you use frequently. For `urls` and `site prefixes`, it will assume `http://` and port `80` unless you specify otherwise. + +Note that a `_site prefix_` is everything before the api route, e.g. `http://domain.com/site1` in `http://domain.com/site1/_components/article`. + +``` +[keys] + local = ha8yds9a8shdf98asdf + qa = 8quwqwer09ewr0w9uer + prod = bj34b6345k634jnk63n4 +[urls] + local-site1 = https://localhost.site1.com:3001 + local-site2 = site2.com/site-2 # http and port 80 +``` + +For smaller `Clay` installations (or, ironically, for very large teams where devs spend most of their time on individual sites), you may specify a default api key and `url / site prefix` by using the `CLAYCLI_DEFAULT_KEY` and `CLAYCLI_DEFAULT_URL` environment variables. + +## Commands + +* [`config`](#config) +* [`lint`](#lint) +* [`import`](#import) +* [`export`](#export) +* [`compile`](#compile) + +## Common Arguments + +`claycli` uses some common arguments across many commands. + +* `-v, --version` will print the `claycli` version and exit +* `-h, --help` will print helpful info about `claycli` and exit +* `-r, --reporter` allows specifying how results should be logged +* `-c, --concurrency` allows setting the concurrency of api calls (defaults to 10) +* `-k, --key` allows specifying an api key or alias + +### Logging + +When running `claycli` programmatically (i.e., `import { someMethod } from 'claycli'`), most commands will return a stream of objects with `{ type, message, details }`. The `type` may be: + * `success` Signals that an operation succeeded + * `error` Signals that an operation was unsuccessful + * `warning` Shows a potentially undesired situation + * `info` Gives the progress information of the current operation + * `debug` Shows information of the current operation + + As you can see, most of those correspond directly to log levels. Here you can check how to programmatically get the `claycli` methods on the [Programmatic API](#programmatic-api) section. + +When running `claycli` from the command line, you may specify a `reporter` argument to output logs in different formats. The default is `dots`, which will print out green and red dots showing operation success / failure. There is also `pretty` (which prints more detailed messages on each line), `json` (which prints newline-separated json logs in a format that can be passed to ELK), and `nyan` (which is mostly just for fun). + +```bash +$ clay lint --reporter pretty domain.com/_components/article +``` + +You may also specify which reporter to use by setting the `CLAYCLI_REPORTER` environment variable. If you add a `reporter` argument, it will be used instead of the env variable. + +``` +export CLAYCLI_REPORTER=json +``` + +`claycli` pipes to `stderr`. If you want to pipe the logs to a file, you may use `2>`. + +```bash +$ clay lint --reporter json domain.com/_components/article 2> article-log.json +``` + +## Handling Files + +### Dispatch + +Many `claycli` commands allow you to pipe in the contents of files to `stdin` or pipe data out from `stdout`. The format that `claycli` uses to represent data (similar to a database dump) is called a _dispatch_, and it consists of newline-separated JSON without site prefixes. + +``` +{"/_components/article/instances/123":{"title":"My Article","content":[{"_ref":"/_components/paragraph/instances/234","text":"Four score and seven years ago..."}]}} +{"/_components/meta-title/instances/345":{"title":"My Article","ogTitle":"My Longer Titled Article","twitterTitle":"Article"}} +``` + +Each line of a _dispatch_ contains [composed data for a component](https://github.com/clay/amphora/blob/master/lib/routes/readme.md#component-data) (or page, user, list, etc), including any data for its child components. This means that each line is able to be sent as a [cascading PUT](https://github.com/clay/amphora/pull/73) to the Clay server, which is a highly efficient way of importing large amounts of data. Note that a _dispatch_ is not meant to be human-readable, and manually editing it is a very easy way to introduce data errors. + +A _dispatch_ may be piped into or out of commands such as `clay import` and `clay export`. Because _dispatches_ are a special format (rather than regular JSON files), the convention is to use the `.clay` extension, but this isn't required. + +```bash +$ clay export domain.com > article_dump.clay +$ clay import domain.com < article_dump.clay +$ clay export domain.com | clay import localhost +``` + +### Bootstrap + +For working with human-readable data, we use a format called a _bootstrap_. These are human-readable [yaml](http://docs.ansible.com/ansible/latest/YAMLSyntax.html) files that divide components (pages, users, lists, etc) by type. [This is the same format that is used by the `bootstrap.yml` files in your Clay install](http://clay.github.io/amphora/docs/lifecycle/startup/bootstrap.html). + +```yaml +_components: + article: + instances: + 123: + title: My Article + content: + - _ref: /_components/paragraph/instances/234 + paragraph: + instances: + 234: + text: Four score and seven years ago... + meta-title: + instances: + 345: + title: My Article + ogTitle: My Longer Titled Article + twitterTitle: Article +``` + +A _bootstrap_ may be piped into and out of any `claycli` commands that accept _dispatches_. To tell `claycli` that you're dealing with _bootstraps_, please use the `--yaml` argument. + +```bash +$ clay export --yaml domain.com > article_dump.yml +$ clay import --yaml domain.com < article_dump.yml +``` + +If you're a backend developer or database architect, it may be helpful to think of _dispatches_ and _bootstraps_ as [denormalized and normalized data](https://medium.com/@katedoesdev/normalized-vs-denormalized-databases-210e1d67927d). You'll notice that the two examples above contain the same data. The denormalized _dispatches_ allow a single API call per line and use less memory because they're streamable, while the normalized _bootstraps_ are better for hand-coding data because components are not duplicated if referenced multiple times. Generally speaking, use _dispatches_ for transporting and storing data and _bootstraps_ for hand-coding. + +## Config + +```bash +$ clay config --key [value] +$ clay config --url [value] +``` + +Show or set configuration options. These are saved to `~/.clayconfig`. As specified above, sites will assume `http` and port `80` if you do not write the protocol and port. + +### Arguments + +* `-k, --key` allows viewing or saving an api key +* `-u, --url` allows viewing or saving a url / site prefix +* `-r, --reporter` allows specifying how results should be logged (note: all reporters except `json` report `clay config` the same) + +### Examples + +```bash +# view all configuration options +$ clay config + +# view 'local' api key +$ clay config --key local + +# set 'local' api key +$ clay config --key local ab27s9d + +# view 'qa' site prefix +$ clay config --url qa + +# set 'qa' site prefix +$ clay config --url qa https://qa.domain.com:3001 + +# set a specific url +$ clay config --url my-cool-article domain.com/_components/article/instances/123 +``` + +## Lint + +```bash +clay lint [--concurrency ] [url] +``` + +Verify Clay data against standardized conventions and make sure all child components exist. + +Linting a page, component, or user url will verify that the data for that url exists, and (for pages and components) will (recursively) verify that all references to child components exist. The url must be a raw url, an alias specified via `clay config`, or omitted in favor of `CLAYCLI_DEFAULT_URL`. Linting a public url (or a page/component url that has a `.html` extension) will attempt to render that url with the extension and, if that fails, try to figure out which component isn't rendering correctly. You may lint other renderers by providing their extensions, e.g. `.amp` or `.rss`. + +Instead of linting a url, you may pipe in a component's `schema.yml` to lint. It will go through the schema and verify that it conforms to [Kiln's schema rules](https://claycms.gitbooks.io/kiln/editing-components.html). + +### Arguments + +* `-r, --reporter` allows specifying how results should be logged +* `-c, --concurrency` allows setting the concurrency of api calls + +### Examples + +```bash +# lint all components on a page +$ clay lint domain.com/_pages/123 + +# lint a page via public url +$ clay lint domain.com/2018/02/some-slug.html + +# lint a component and its html +$ clay lint domain.com/_components/article/instances/abc.html + +# lint a component specified via config alias +$ clay lint my-cool-article + +# lint single schema +$ clay lint < components/article/schema.yml +``` + +## Import + +```bash +clay import [--key ] [--concurrency ] [--publish] [--yaml] [site prefix] +``` + +Imports data into Clay from `stdin`. Data may be in _dispatch_ or _bootstrap_ format. Site prefix must be a raw url, an alias specified via `clay config`, or omitted in favor of `CLAYCLI_DEFAULT_URL`. Key must be an alias specified via `clay config`, or omitted in favor of `CLAYCLI_DEFAULT_KEY`. + +The `publish` argument will trigger a publish of the pages and/or components you're importing. Note that the generated url of an imported page might be different than its original url, depending on your Clay url generation / publishing logic. + +### Arguments + +* `-k, --key` allows specifying an api key or alias +* `-r, --reporter` allows specifying how results should be logged +* `-c, --concurrency` allows setting the concurrency of api calls +* `-p, --publish` triggers publishing of imported pages +* `-y, --yaml` specifies that input is _bootstrap_ format + +### Examples + +```bash +# import a dispatch +$ clay import --key local localhost:3001 < db_dump.clay + +# import and publish pages in a bootstrap +$ clay import --key qa --publish --yaml < bootstrap.yml + +# pipe from 3rd party exporter +$ wordpress-export domain.com/blog | clay import --key local localhost.domain.com + +# pipe from clay exporter +$ clay export --key prod domain.com/_components/article/instances/123 | clay import --key local localhost.domain.com + +# import multiple dispatches +$ cat *.clay | clay import --key local localhost:3001 + +# import multiple bootstraps +$ tail -n +1 *.yml | clay import --key local --yaml localhost:3001 + +# recursively import multiple bootstraps +$ find . -name '*.yml' -exec cat "{}" \; | clay import --key local --yaml localhost:3001 + +# recursively import multiple bootstraps (bash v4+ & zsh) +$ cat **/*.yml | clay import --key local --yaml localhost:3001 +``` + +## Export + +```bash +clay export [--key ] [--concurrency ] [--size ] [--layout] [--yaml] [url] +``` + +Exports data from Clay to `stdout`. Data may be in _dispatch_ or _bootstrap_ format. The url must be a raw url, an alias specified via `clay config`, or omitted in favor of `CLAYCLI_DEFAULT_URL`. + +If the url points to a site prefix (i.e. it does not point to a specific type of data (a specific page, public url, component, user, list, etc)), `claycli` will query the built-in `pages` index to pull the latest 10 pages from the site. When querying the `pages` index, you must specify a `key` or have the `CLAYCLI_DEFAULT_KEY` set. The api key is only required when exporting multiple pages (by querying the `pages` index or by running custom queries, below). + +Instead of fetching the latest pages, you may pipe in a yaml-formatted [elasticsearch query](https://www.elastic.co/guide/en/elasticsearch/reference/6.6/query-dsl.html). Use this to set custom offsets (for batching and chunking exports), export non-page content from other indices or filter exported data via certain properties. Note that if you pipe in a query that includes `size`, it will take precedence over the CLI `size` argument. + +```yaml +index: pages +size: 100 +body: + sort: + updateTime: + order: desc # sort by latest updated + query: + bool: + must: + - + terms: + siteSlug: + - intelligencer # show only pages for a specific site + - + match: + published: true # show only published pages + +``` + +You may also query other elastic indices, but please make sure that each document returned has a clay uri (e.g. `domain.com/_components/foo/instances/bar` or `domain.com/_pages/foo`) as its `_id`. + +```yaml +index: published-products +size: 5 +from: 10 +sort: + - price +body: + query: + match_all: {} +``` + +By default, layouts are not exported when exporting pages. This allows you to easily copy individual pages between sites and environments. To trigger layout exporting, please use the `layout` argument. + +### Arguments + +* `-k, --key` allows specifying an api key or alias +* `-r, --reporter` allows specifying how results should be logged +* `-c, --concurrency` allows setting the concurrency of api calls +* `-s, --size` specifies the number of pages to export (defaults to 10) +* `-l, --layout` triggers exporting of layouts +* `-y, --yaml` specifies that output is _bootstrap_ format + +### Examples + +```bash +# export individual component +$ clay export domain.com/_components/article/instances/123 > article_dump.clay + +# export individual page +$ clay export --yaml domain.com/_pages/123 > page_bootstrap.yml + +# export page with layout +$ clay export --layout --yaml domain.com/_pages/123 > page_bootstrap.yml + +# copy page to local environment +$ clay export domain.com/_pages/123 | clay import --key local local.domain.com + +# export latest updated page +$ clay export --key prod --size 1 domain.com > recent_page.clay + +# export custom query to dispatch +$ cat query.yml | clay export --key prod domain.com > db_dump.clay + +# export custom query to bootstrap +$ clay export --yaml --key prod domain.com/sub-site < query.yml > pages.yml + +# note that 'cat query.yml | clay export' and 'clay export < query.yml' are equivalent ways +# to pipe from a file into claycli in most operating systems + +# +# other things you may export +# + +# export single user +$ clay export domain.com/_users/abs8a7s8d --yaml > my_user.yml + +# export all users +$ clay export domain.com/_users --yaml > users.yml + +# export single list +$ clay export domain.com/_lists/tags > tags.clay + +# export all lists +$ clay export domain.com/_lists > lists.clay + +# export published page via public url +$ clay export domain.com/2017/02/some-slug.html + +# export built-in 'New Page Templates' list (page uris will be unprefixed) +$ clay export domnain.com/_lists/new-pages +``` + +## Compile + +```bash +clay compile [--watch] [--minify] [--inlined] [--linked] [--plugins ] [--globs ] [--reporter ] +``` + +Compile assets based on standardized Clay conventions. Assets are compiled to a `public` folder at the root of your Clay install (the directory where you run the `clay compile` command), with scripts (including templates), styles (including fonts), and media output to the `js`, `css`, and `media` folders. You may run `clay compile` to compile _all_ assets or run any of its subcommands (`media`, `fonts`, `styles`, `templates`, `scripts`) to compile a specific type of asset. + +Specifying `--watch` on `claycli compile` or any of its subcommands will compile assets once, then watch source files (and their dependencies) for changes. Specifying `--minify` (or setting `CLAYCLI_COMPILE_MINIFIED`) will run assets through minification and bundling if applicable. The `CLAYCLI_COMPILE_ASSET_HOST` and `CLAYCLI_COMPILE_ASSET_PATH` variables are used by the `styles` and `fonts` subcommands to generate links to media and font files in the compiled CSS. + +A project specific clay config file is also supported, [read more here](#project-specific-config-file). + +#### Arguments + +* `-w, --watch` enables watching of source files after compilation +* `-m, --minify` enables minification and bundling of source files +* `-i, --inlined` enables the generation of base64 inlined font CSS +* `-l, --linked` enables the generation of linked font CSS +* `-p, --plugins` allows running additional postcss plugins when compiling styles +* `-g, --globs` allows compiling additional JavaScript to `public/js/_global.js` +* `-r, --reporter` allows specifying how results should be logged + +#### Examples + +```bash +# compile all assets once +$ clay compile + +# compile and watch all assets +$ clay compile --watch + +# compile all assets once for production environments +$ clay compile --minify + +# compile all assets, creating both inlined and linked font CSS +$ clay compile --inlined --linked +``` + +### Media + +```bash +clay compile media [--watch] [--reporter ] +``` + +Copy component, layout, style guide, and site-specific media files from their source folders to the `public` directory. Media files are images (`jpg`, `jpeg`, `png`, `gif`), `svgs`, and favicons (`ico`). + +* `components//media/` are referenced by component templates and get copied to `public/media/components//` +* `layouts//media/` are referenced by layout templates and get copied to `public/media/layouts//` +* `styleguides//media/` are referenced by that styleguide's CSS and get copied to `public/media/stylesguides//` +* `sites//media/` are favicons and other site-specific icons that are referenced by particular components in the `` of pages. They get copied to `public/media/sites//` + +#### Arguments + +* `-w, --watch` enables watching of source files after compilation +* `-r, --reporter` allows specifying how results should be logged + +#### Examples + +```bash +# compile media files once +$ clay compile media + +# compile and watch media files +$ clay compile media --watch +``` + +### Fonts + +```bash +clay compile fonts [--watch] [--minify] [--inlined] [--linked] [--reporter ] +``` + +Compile fonts from `styleguides//fonts/` to the `public` directory. By default (and if `--linked` is specified or `CLAYCLI_COMPILE_LINKED_FONTS` is set), this will generate a `public/css/_linked-fonts..css` file with `@font-face` declarations and copy the original font file to `public/fonts/`. Note that naming collisions are possible when using fonts of the same filename across different styleguides. If `--inlined` is specified (or `CLAYCLI_COMPILE_INLINED_FONTS` is set), this will generate a `public/css/_inlined-fonts..css` file with `@font-face` declarations that include a base64-encoded copy of the font. + +`@font-face` declarations are generated based on the filename of the original font file, with a simple convention to support various weights and styles. + +* `.` font with normal weight and style +* `-.` or `-