Merge fc3eec3 into 511e6e4

CONTRIBUTING.md

@@ -83,7 +83,10 @@ The main documentation is written in [Markdown][] using [MkDocs][]. Dredd uses
 - https://dredd.readthedocs.org/ - preferred long URL
 - http://dredd.rtfd.org/ - preferred short URL
 
-Source of the documentation can be found in the [docs][] directory. To contribute to Dredd's documentation, you will need to follow the [MkDocs installation instructions](http://www.mkdocs.org/#installation). Once installed, you may use `mkdocs serve` from the Dredd directory to run the local server for the documentation.
+Source of the documentation can be found in the [docs][] directory. To contribute to Dredd's documentation, you will need to follow the [MkDocs installation instructions](http://www.mkdocs.org/#installation). Once installed, you may use following commands:
+
+- `npm run docs:build` - Builds the documentation.
+- `npm run docs:serve` - Runs live preview of the documentation.
 
 ### Coverage
 
@@ -104,6 +107,18 @@ coverage:
 If a Pull Request introduces drop in coverage, it won't be accepted unless
 the author or reviewer provides a good reason why an exception should be made.
 
+### Hacking Apiary Reporter
+
+If you want to build something on top of the Apiary Reporter, note that it uses a public API described in following documents:
+
+- [Apiary Tests API for anonymous test reports][]
+- [Apiary Tests API for authenticated test reports][]
+
+There are also some environment variables you could find useful:
+
+- `APIARY_API_URL='https://api.apiary.io'` - Allows to override host of the Apiary Tests API.
+- `DREDD_REST_DEBUG=true` - Turns on some additional logging. Useful for debugging.
+
 
 [Apiary]: https://apiary.io/
 [API Blueprint]: http://apiblueprint.org/
@@ -127,3 +142,6 @@ the author or reviewer provides a good reason why an exception should be made.
 
 [upstream repository]: https://github.com/apiaryio/dredd
 [issues]: https://github.com/apiaryio/dredd/issues
+
+[Apiary Tests API for anonymous test reports]: https://github.com/apiaryio/dredd/blob/master/ApiaryReportingApiAnonymous.apib
+[Apiary Tests API for authenticated test reports]: https://github.com/apiaryio/dredd/blob/master/ApiaryReportingApi.apib

README.md

@@ -20,7 +20,7 @@ Dredd reads your API description and step by step validates whether your API
 implementation replies with responses as they are described in the
 documentation.
 
-### Supported Hook Languages
+### Supported Hooks Languages
 
 Dredd supports writing [hooks](http://dredd.readthedocs.org/en/latest/hooks/)
 — a glue code for each test setup and teardown. Following languages are supported:

docs/execution-lifecycle.md

@@ -0,0 +1,36 @@
+# Execution Lifecycle
+
+This section provides the order in which Dredd executes to give a better understanding of how Dredd works.
+
+1. Load and parse blueprints
+    - Report parsing warnings
+2. Pre-run blueprint check
+    - Missing example values for URI template parameters
+    - Required parameters present in URI
+    - Report non-parseable JSON bodies
+    - Report invalid URI parameters
+    - Report Invalid URI templates
+3. Compile HTTP transactions from blueprints
+    - Inherit headers
+    - Inherit parameters
+    - Expand URI templates with parameters
+4. Load hooks
+5. Test run
+    - Report test run `start`
+    - Run `beforeAll` hooks
+    - For each compiled transaction:
+        - Report `test start`
+        - Run `beforeEach` hook
+        - Run `before` hook
+        - Send HTTP request
+        - Receive HTTP response
+        - Run `beforeEachValidation` hook
+        - Run `beforeValidation` hook
+        - Perform [Gavel][] validation
+        - Run `after` hook
+        - Run `afterEach` hook
+        - Report `test end` with result for in-progress reporting
+    - Run `afterAll` hooks
+6. Report test run `end` with result statistics
+
+[Gavel]: http://blog.apiary.io/2013/07/24/Bam-this-is-Gavel/

docs/hooks-go.md

@@ -44,7 +44,7 @@ The `Runner` type has the following callback methods.
 3. `BeforeAll`, `AfterAll`
   - accepts a function as a first argument passing a Slice of [Transaction objects](hooks.md#transaction-object-structure) as a first argument
 
-Refer to [Dredd execution lifecycle](usage.md#dredd-execution-lifecycle) to find when each hook callback is executed.
+Refer to [Dredd execution lifecycle](execution-lifecycle.md) to find when each hook callback is executed.
 
 ### Using the Go API
 

docs/hooks-new-language.md

@@ -19,7 +19,7 @@ If you want to write a hook handler for your language you will have to implement
 - CLI Command runnning TCP socket server
   - [Must return message `Starting` to stdout](https://github.com/apiaryio/dredd-hooks-template/blob/master/features/tcp_server.feature#L5)
 
-- Hooks API in your language for registering code being executed during the [Dredd lifecycle](usage.md#dredd-execution-lifecycle):
+- Hooks API in your language for registering code being executed during the [Dredd lifecycle](execution-lifecycle.md):
   - before all transactions
   - before each transaction
   - before transaction
@@ -73,16 +73,14 @@ If you want to write a hook handler for your language you will have to implement
 
 There are several configuration options, which can help you during development:
 
-- `--hooks-worker-timeout`: `5000` - How long to wait for hooks worker to start. [ms]
-- `--hooks-worker-connect-timeout`: `1500` - How long to wait for hooks worker to acknowledge connection. [ms]
-- `--hooks-worker-connect-retry`: `500` - How long to wait between attempts to connect to hooks worker. [ms]
-- `--hooks-worker-after-connect-wait`: `100` - How long to wait between connecting to hooks worker and start of testing. [ms]
-- `--hooks-worker-term-timeout`: `5000` - How long to wait between trying to terminate hooks worker and killing it. [ms]
-- `--hooks-worker-term-retry`: `500` - How long to wait between attempts to terminate hooks worker. [ms]
-- `--hooks-worker-handler-host`: `localhost` - Host of the hook worker.
-- `--hooks-worker-handler-port`: `61321` - Port of the hook worker.
-
-See [usage](usage.md) for further details.
+- [--hooks-worker-timeout](usage-cli.md#-hooks-worker-timeout)
+- [--hooks-worker-connect-timeout](usage-cli.md#-hooks-worker-connect-timeout)
+- [--hooks-worker-connect-retry](usage-cli.md#-hooks-worker-connect-retry)
+- [--hooks-worker-after-connect-wait](usage-cli.md#-hooks-worker-after-connect-wait)
+- [--hooks-worker-term-timeout](usage-cli.md#-hooks-worker-term-timeout)
+- [--hooks-worker-term-retry](usage-cli.md#-hooks-worker-term-retry)
+- [--hooks-worker-handler-host](usage-cli.md#-hooks-worker-handler-host)
+- [--hooks-worker-handler-port](usage-cli.md#-hooks-worker-handler-port)
 
 ## Need help? No problem!
 

docs/hooks-perl.md

@@ -32,7 +32,7 @@ Module `Dredd::Hooks::Methods` imports following decorators:
 3. `beforeAll`, `afterAll`
   - wraps a function and passes an Array of [Transaction objects](hooks.md#transaction-object-structure) as a first argument to it
 
-Refer to [Dredd execution life-cycle](usage.md#dredd-execution-lifecycle) to find when is each hook function executed.
+Refer to [Dredd execution life-cycle](execution-lifecycle.md) to find when is each hook function executed.
 
 ### Using Perl API
 

docs/hooks-php.md

@@ -38,7 +38,7 @@ The `Dredd\Hooks` class provides the static methods listed below to create hooks
   - accepts a block as a first argument passing an Array of [Transaction objects](hooks.md#transaction-object-structure) as a first argument
 
 
-Refer to [Dredd execution lifecycle](usage.md#dredd-execution-lifecycle) to find when is each hook function executed.
+Refer to [Dredd execution lifecycle](execution-lifecycle.md) to find when is each hook function executed.
 
 ### Using PHP API
 

docs/hooks-python.md

@@ -34,7 +34,7 @@ Module `dredd_hooks` imports following decorators:
   - wraps a function and passes an Array of [Transaction objects](hooks.md#transaction-object-structure) as a first argument to it
 
 
-Refer to [Dredd execution life-cycle](usage.md#dredd-execution-lifecycle) to find when is each hook function executed.
+Refer to [Dredd execution life-cycle](execution-lifecycle.md) to find when is each hook function executed.
 
 ### Using Python API
 

docs/hooks-ruby.md

@@ -33,7 +33,7 @@ Including module `Dredd::Hooks:Methods` expands current scope with methods
   - accepts a block as a first argument passing an Array of [Transaction objects](hooks.md#transaction-object-structure) as a first argument
 
 
-Refer to [Dredd execution lifecycle](usage.md#dredd-execution-lifecycle) to find when is each hook function executed.
+Refer to [Dredd execution lifecycle](execution-lifecycle.md) to find when is each hook function executed.
 
 ### Using Ruby API
 

docs/hooks.md

@@ -1,7 +1,7 @@
 # Hook Scripts
 
 Similar to any other testing framework, Dredd supports executing code around each test step.
-Hooks are code blocks executed in defined stage of [execution lifecycle](usage.md#dredd-execution-lifecycle).
+Hooks are code blocks executed in defined stage of [execution lifecycle](execution-lifecycle.md).
 In the hooks code you have an access to compiled HTTP [transaction object](#transaction-object-structure) which you can modify.
 
 Hooks are usually used for:
@@ -89,7 +89,7 @@ Dredd supports following types of hooks:
 - `afterEach` called after each HTTP transaction
 - `afterAll` called after whole test run
 
-Refer to [Dredd execution lifecycle](usage.md#dredd-execution-lifecycle) when is each hook executed.
+Refer to [Dredd execution lifecycle](execution-lifecycle.md) when is each hook executed.
 
 ### Transaction Object Structure
 

docs/index.md

@@ -16,7 +16,7 @@ Dredd reads your API description and step by step validates whether your API
 implementation replies with responses as they are described in the
 documentation.
 
-### Supported Hook Languages
+### Supported Hooks Languages
 
 Dredd supports writing [hooks](http://dredd.readthedocs.org/en/latest/hooks/)
 — a glue code for each test setup and teardown. Following languages are supported:
@@ -38,17 +38,27 @@ Dredd supports writing [hooks](http://dredd.readthedocs.org/en/latest/hooks/)
 
 ## Documentation Reference
 
-1. [Overview](overview.md)
-2. [Quickstart](quickstart.md)
-3. [Usage](usage.md)
-4. [Hooks](hooks.md)
-5. [Ruby Hooks](hooks-ruby.md)
-6. [Python Hooks](hooks-python.md)
-7. [Node.js Hooks](hooks-nodejs.md)
-8. [PHP Hooks](hooks-php.md)
-9. [Sandboxed JavaScript Hooks](hooks-js-sandbox.md)
-10. [Hooks in new language](hooks-new-language.md)
-11. [Example application](example.md)
+- Dredd
+    - [About Dredd](index.md)
+    - [Quickstart](quickstart.md)
+    - [Execution Lifecycle](execution-lifecycle.md)
+    - [Overview](overview.md)
+- Usage
+    - [Command-line Interface](usage-cli.md)
+    - [From JavaScript](usage-js.md)
+- Hooks
+    - [About Hooks](hooks.md)
+    - Supported Languages
+        - [Go](hooks-go.md)
+        - [JavaScript (Sandboxed)](hooks-js-sandbox.md)
+        - [Node.js](hooks-nodejs.md)
+        - [Perl](hooks-perl.md)
+        - [PHP](hooks-php.md)
+        - [Python](hooks-python.md)
+        - [Ruby](hooks-ruby.md)
+    - [Other Languages](hooks-new-language.md)
+- Example
+    - [Full Example](example.md)
 
 ## Useful Links
 

docs/overview.md

@@ -23,16 +23,47 @@ Dredd automatically generates expectations on HTTP responses based on examples i
 - Plain text must match perfectly
 - If JSON Schema v4 or JSON Schema v3 is given in the blueprint, JSON response must be valid against this schema and JSON example is ignored.
 
-## Using Apiary Test Inspector
+## Using Apiary Reporter and Apiary Tests
 
-Command-line output of complex HTTP responses and expectation can be hard to read. Dredd can send test reports to Apiary and Apiary provides an interface for browsing them. To enable it, use argument `--reporter apiary`.
+Command-line output of complex HTTP responses and expectations can be hard to read. To tackle the problem, you can use Dredd to send test reports to [Apiary](https://apiary.io/). Apiary provides comfortable interface for browsing complex test reports:
 
-### Saving under your account in Apiary
+```
+$ dredd apiary.apib http://localhost --reporter=apiary
+info: Using apiary reporter.
+warn: Apiary reporter environment variable APIARY_API_KEY or APIARY_API_NAME not defined.
+info: Beginning Dredd testing...
+pass: DELETE /honey duration: 884ms
+complete: 1 passing, 0 failing, 0 errors, 0 skipped, 1 total
+complete: Tests took 1631ms
+complete: See results in Apiary at: https://app.apiary.io/public/tests/run/74d20a82-55c5-49bb-aac9-a3a5a7450f06
+```
+
+![Apiary Tests](https://raw.github.com/apiaryio/dredd/master/img/apiary-tests.png?raw=true)
+
+### Saving Test Reports under Your Account in Apiary
+
+As you can see on the screenshot, the test reports are anonymous by default and will expire after some time. However, if you provide Apiary credentials, your test reports will appear on _Tests_ page of your API Project. This is great especially for introspection of test reports from Continuous Integration.
+
+To get and setup credentials, just follow the tutorial in Apiary:
+
+![Apiary Tests Tutorial](https://raw.github.com/apiaryio/dredd/master/img/apiary-tests-tutorial.png?raw=true)
+
+As you can see, the parameters go like this:
+
+```
+$ dredd -c apiaryApiKey:<Apiary API Key> -c apiaryApiName:<API Project Subdomain>
+```
+
+Except of using parameters and `dredd.yml`, you can also use environment variables:
+
+- `APIARY_API_KEY=<Apiary API Key>` - Alternative way to pass credentials to Apiary Reporter.
+- `APIARY_API_NAME=<API Project Subdomain>` - Alternative way to pass credentials to Apiary Reporter.
 
-Reports are anonymous by default, but you can let Apiary save them under your API in Apiary by specifying Apiary Key and API Name with arguments
-`-c apiaryApiKey:yourApiKey -c apiaryApiName:yourapiname` This is great for introspecting reports from Continuous Integration.
+When sending test reports to Apiary, Dredd inspects the environment where it was executed and sends some information about it alongside test results. Those are used mainly for detection whether the environment is Continuous Integration and also, they help you to identify individual test reports on the _Tests_ page. You can use following variables to tell Dredd what to send:
 
-**TODO**: Screenshot image
+- agent (string) - `DREDD_AGENT` or current user in the OS
+- hostname (string) - `DREDD_HOSTNAME` or hostname of the OS
+- CI (boolean) - looks for `TRAVIS`, `CIRCLE`, `CI`, `DRONE`, `BUILD_ID`, ...
 
 ## Testing API Documentation
 
@@ -227,7 +258,7 @@ test:
     - dredd bluprint.md http://localhost:3000
 ```
 
-Example `travis.yml` configuration file for Travis CI:
+Example `travis.yml` configuration file for [Travis CI][]:
 
 ```
 before_install:
@@ -336,12 +367,6 @@ info: Resource > Update resource > Example 2
 ```
 
 [API Blueprint]: http://apiblueprint.org/
-[test coverage]: https://coveralls.io/r/apiaryio/dredd?branch=master
 [Travis CI]: https://travis-ci.org/
-[Jenkins]: http://jenkins-ci.org/
-[Gavel]: http://blog.apiary.io/2013/07/24/Bam-this-is-Gavel/
-[behavior specification]: https://www.relishapp.com/apiary/gavel/docs
-[vde]: https://github.com/apiaryio/dredd/blob/master/VirtualDevelopmentEnvironment.md
-[issues]: https://github.com/apiaryio/dredd/issues?state=open
-[Express.js]: http://expressjs.com/starter/hello-world.html
-[CoffeeScript]: http://coffeescript.org
+[Gavel.js]: https://github.com/apiaryio/gavel.js
+[Gavel]: https://www.relishapp.com/apiary/gavel/docs