Event forwarding

docs/api-reference/elasticsearch/index.md → docs/api-reference/loaders-storage-targets/elasticsearch/index.md

@@ -1,7 +1,7 @@
 ---
 title: "Elasticsearch Loader"
 date: "2020-11-25"
-sidebar_position: 50
+sidebar_position: 64
 ---
 
 ```mdx-code-block

docs/destinations/forwarding-events/snowbridge/3-X-X-upgrade-guide/index.md → docs/api-reference/snowbridge/3-X-X-upgrade-guide/index.md

@@ -9,7 +9,7 @@ The below breaking changes were made in version 3.0.0. All other functionality i
 
 ### Lua support removed
 
-Support for Lua transformations has been removed. If you are running a Lua transformation, you can port the logic to [Javascript](/docs/destinations/forwarding-events/snowbridge/configuration/transformations/custom-scripts/javascript-configuration/index.md) or [JQ](/docs/destinations/forwarding-events/snowbridge/configuration/transformations/builtin/jq.md).
+Support for Lua transformations has been removed. If you are running a Lua transformation, you can port the logic to [Javascript](/docs/api-reference/snowbridge/configuration/transformations/custom-scripts/javascript-configuration/index.md) or [JQ](/docs/api-reference/snowbridge/configuration/transformations/builtin/jq.md).
 
 ## HTTP target: non-JSON data no longer supported
 

docs/destinations/forwarding-events/snowbridge/concepts/failure-model/index.md → docs/api-reference/snowbridge/concepts/failure-model/index.md

@@ -10,7 +10,7 @@ sidebar_position: 600
 
 When Snowbridge hits an unrecoverable error — for example [oversized](#oversized-data) or [invalid](#invalid-data) data — it will emit a [failed event](/docs/fundamentals/failed-events/index.md#what-is-a-failed-event) to the configured failure target. A failure target is the same as a target, the only difference is that the configured destination will receive failed events.
 
-You can find more detail on setting up a failure target, in the [configuration section](/docs/destinations/forwarding-events/snowbridge/configuration/targets/index.md).
+You can find more detail on setting up a failure target, in the [configuration section](/docs/api-reference/snowbridge/configuration/targets/index.md).
 
 ## Failure cases
 
@@ -20,7 +20,7 @@ There are several different failures that Snowbridge may hit.
 
 This is where a request to the destination technology fails or is rejected - for example a HTTP 400 response is received.
 
-Retry behavior for target failures is determined by the retry configuration. You can find details of this in the [configuration section](/docs/destinations/forwarding-events/snowbridge/configuration/retries/index.md).
+Retry behavior for target failures is determined by the retry configuration. You can find details of this in the [configuration section](/docs/api-reference/snowbridge/configuration/retries/index.md).
 
 As of Snowbridge 2.4.2, the Kinesis target does not treat kinesis write throughput exceptions as this type of failure. Rather it has an in-built backoff and retry, which will persist until each event in the batch is either successful, or fails for a different reason.
 
@@ -38,7 +38,7 @@ Writes of oversized messages to the failure target will be recorded with 'Oversi
 
 In the unlikely event that Snowbridge encounters data which is invalid for the target destination (for example empty data is invalid for pubsub), it will create a [generic error failed event](/docs/fundamentals/failed-events/index.md#generic-error),  emit it to the failure target, and ack the original message.
 
-As of version 3.0.0, the HTTP target may produce 'invalid' type failures. This occurs when: the a POST request body cannot be formed; the templating feature's attempts to template data result in an error; or the response conforms to a response rules configuration which specifies that the failure is to be treated as invalid. You can find more details in the [configuration section](/docs/destinations/forwarding-events/snowbridge/configuration/targets/http/index.md).
+As of version 3.0.0, the HTTP target may produce 'invalid' type failures. This occurs when: the a POST request body cannot be formed; the templating feature's attempts to template data result in an error; or the response conforms to a response rules configuration which specifies that the failure is to be treated as invalid. You can find more details in the [configuration section](/docs/api-reference/snowbridge/configuration/targets/http/index.md).
 
 Transformation failures are also treated as invalid, as described below.
 

docs/destinations/forwarding-events/snowbridge/concepts/scaling/index.md → docs/api-reference/snowbridge/concepts/scaling/index.md

@@ -6,7 +6,7 @@ sidebar_position: 150
 Snowbridge is built to suit a **horizontal scaling** model, and you can safely deploy multiple instances of Snowbridge to consume the same input out-of-the-box. No addditional configuration or setup is required for the app to smoothly run across multiple instances/environments, compared to a single instance/environment.
 
 :::note
-If you are using the Kinesis source, you will need to manually create a few DynamoDB tables as described in [the Kinesis source configuration section](/docs/destinations/forwarding-events/snowbridge/configuration/sources/kinesis.md). Snowbridge uses these tables to coordinate multiple instances consuming from the same stream.
+If you are using the Kinesis source, you will need to manually create a few DynamoDB tables as described in [the Kinesis source configuration section](/docs/api-reference/snowbridge/configuration/sources/kinesis.md). Snowbridge uses these tables to coordinate multiple instances consuming from the same stream.
 :::
 
 How to configure scaling behavior will depend on the infrastructure you’re using, and the use case you have implemented. For example, if you choose to scale based on CPU usage, note that this metric will be affected by the size and shape of the data, by the transformations and filters used, and for script transformations, by the content of the scripts.

docs/destinations/forwarding-events/snowbridge/concepts/sources/index.md → docs/api-reference/snowbridge/concepts/sources/index.md

@@ -8,4 +8,4 @@ sidebar_position: 200
 
 Sources deal with retrieving data from the input stream, and forwarding it for processing — once messages are either filtered or successfully sent, they are then acked (if the source technology supports acking). Otherwise, messages will be retrieved again by the source. Sources also have a setting which controls concurrency for the instance — `concurrent_writes`.
 
-You can find more detail on setting up a source, in the [configuration section](/docs/destinations/forwarding-events/snowbridge/configuration/sources/index.md).
+You can find more detail on setting up a source, in the [configuration section](/docs/api-reference/snowbridge/configuration/sources/index.md).

docs/destinations/forwarding-events/snowbridge/concepts/targets/index.md → docs/api-reference/snowbridge/concepts/targets/index.md

@@ -6,4 +6,4 @@ sidebar_position: 400
 
 Targets check for [validity and size restrictions](../failure-model/index.md), [batch data](../batching-model/index.md) where appropriate and send data to the destination stream.
 
-You can find more detail on setting up a target, in the [configuration section](/docs/destinations/forwarding-events/snowbridge/configuration/targets/index.md).
+You can find more detail on setting up a target, in the [configuration section](/docs/api-reference/snowbridge/configuration/targets/index.md).

docs/destinations/forwarding-events/snowbridge/concepts/transformations/index.md → docs/api-reference/snowbridge/concepts/transformations/index.md

@@ -19,4 +19,4 @@ The latter use case, and further nuanced use cases can, however, be achieved usi
 
 Custom scripting transformations allow you to provide a script to transform the data, set the destination's partition key, or filter the data according to your own logic. For scripting, you can use Javascript. Snowbridge uses a runtime engine to run the script against the data. Scipts interface with the rest of the app via the `EngineProtocol` interface, which provides a means to pass data into the scripting layer, and return data from the scripting layer back to the app.
 
-You can find more detail on setting up custom scripts [in the configuration section](/docs/destinations/forwarding-events/snowbridge/configuration/transformations/index.md).
+You can find more detail on setting up custom scripts [in the configuration section](/docs/api-reference/snowbridge/configuration/transformations/index.md).

docs/destinations/forwarding-events/snowbridge/configuration/monitoring/index.md → docs/api-reference/snowbridge/configuration/monitoring/index.md

@@ -9,15 +9,13 @@ import {versions} from '@site/src/componentVersions';
 import CodeBlock from '@theme/CodeBlock';
 ```
 
-# Monitoring Configuration
-
 ## Stats and metrics
 
 Snowbridge comes with configurable logging, [pprof](https://github.com/google/pprof) profiling, [statsD](https://www.datadoghq.com/statsd-monitoring) statistics and [Sentry](https://sentry.io/welcome/) integrations to ensure that you know what's going on.
 
 ### Logging
 
-Use the log_level parameter to specify the log level.
+Use the `log_level` parameter to specify the log level.
 
 <CodeBlock language="hcl" reference>{`
 https://github.com/snowplow/snowbridge/blob/v${versions.snowbridge}/assets/docs/configuration/monitoring/log-level-example.hcl
@@ -43,6 +41,8 @@ Snowplow Enriched data only:
 https://github.com/snowplow/snowbridge/blob/v${versions.snowbridge}/assets/docs/configuration/metrics/e2e-latency-example.hcl
 `}</CodeBlock>
 
+## Metric definitions
+
 Snowbridge sends the following metrics to statsd:
 
 | Metric                   | Definitions                                                                                                                                             |

docs/destinations/forwarding-events/snowbridge/configuration/retries/index.md → docs/api-reference/snowbridge/configuration/retries/index.md

@@ -20,7 +20,7 @@ A **transient failure** is a failure which we expect to succeed again on retry.
 
 A **setup failure** is one we don't expect to be immediately resolved, for example an incorrect address, or an invalid API Key. Typically, you would configure a long backoff for this type of failure, under the assumption that the issue needs to be fixed with either a configuration change or a change to the target itself (e.g. permissions need to be granted). Setup errors will be retried 5 times before the app crashes.
 
-As of version 3.0.0, only the http target can be configured to return setup errors, via the response rules feature - see [the http target configuration section](/docs/destinations/forwarding-events/snowbridge/configuration/targets/http/index.md). For all other targets, all errors returned will be considered transient, and behavior can be configured using the `transient` block of the retry configuration.
+As of version 3.0.0, only the http target can be configured to return setup errors, via the response rules feature - see [the http target configuration section](/docs/api-reference/snowbridge/configuration/targets/http/index.md). For all other targets, all errors returned will be considered transient, and behavior can be configured using the `transient` block of the retry configuration.
 
 Retries will be attempted with an exponential backoff. In other words, on each subsequent failure, the backoff time will double. You can configure transient failures to be retried indefinitely by setting `max_attempts` to 0.
 

docs/destinations/forwarding-events/snowbridge/configuration/sources/kinesis.md → docs/api-reference/snowbridge/configuration/sources/kinesis.md

@@ -10,7 +10,7 @@ import CodeBlock from '@theme/CodeBlock';
 
 :::note
 
-To use this source, you need the AWS-specific version of Snowbridge that can only be run on AWS. See [the page on Snowbridge distributions](/docs/destinations/forwarding-events/snowbridge/getting-started/index.md) for more information.
+To use this source, you need the AWS-specific version of Snowbridge that can only be run on AWS. See [the page on Snowbridge distributions](/docs/api-reference/snowbridge/getting-started/index.md) for more information.
 
 :::
 

docs/destinations/forwarding-events/snowbridge/configuration/targets/eventhub.md → docs/api-reference/snowbridge/configuration/targets/eventhub.md

@@ -20,11 +20,11 @@ Here is an example of the minimum required configuration:
 https://github.com/snowplow/snowbridge/blob/v${versions.snowbridge}/assets/docs/configuration/targets/eventhub-minimal-example.hcl
 `}</CodeBlock>
 
-If you want to use this as a [failure target](/docs/destinations/forwarding-events/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.
+If you want to use this as a [failure target](/docs/api-reference/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.
 Here is an example of every configuration option:
 
 <CodeBlock language="hcl" reference>{`
 https://github.com/snowplow/snowbridge/blob/v${versions.snowbridge}/assets/docs/configuration/targets/eventhub-full-example.hcl
 `}</CodeBlock>
 
-If you want to use this as a [failure target](/docs/destinations/forwarding-events/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.
+If you want to use this as a [failure target](/docs/api-reference/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.

docs/destinations/forwarding-events/snowbridge/configuration/targets/http/google-tag-manager.md → docs/api-reference/snowbridge/configuration/targets/http/google-tag-manager.md

@@ -2,7 +2,7 @@
 
 You can use the HTTP target to send events to Google Tag Manager Server Side, where the [Snowplow Client tag](/docs/destinations/forwarding-events/google-tag-manager-server-side/snowplow-client-for-gtm-ss/index.md) is installed.
 
-To do this, you will need to include a [transformation](/docs/destinations/forwarding-events/snowbridge/concepts/transformations/index.md) that converts your events to JSON — [`spEnrichedToJson`](/docs/destinations/forwarding-events/snowbridge/configuration/transformations/builtin/spEnrichedToJson.md).
+To do this, you will need to include a [transformation](/docs/api-reference/snowbridge/concepts/transformations/index.md) that converts your events to JSON — [`spEnrichedToJson`](/docs/api-reference/snowbridge/configuration/transformations/builtin/spEnrichedToJson.md).
 
 Here’s an example configuration. Replace `<your-gtm-host>` with the hostname of your Google Tag Manager instance, and — optionally — `<preview-token>` with your preview mode token.
 

docs/destinations/forwarding-events/snowbridge/configuration/targets/http/index.md → docs/api-reference/snowbridge/configuration/targets/http/index.md

@@ -9,7 +9,7 @@ import CodeBlock from '@theme/CodeBlock';
 ```
 
 :::note
-Version 3.0.0 makes breaking changes to the HTTP target. Details on migrating can be found [in the migration guide](/docs/destinations/forwarding-events/snowbridge/3-X-X-upgrade-guide/index.md)
+Version 3.0.0 makes breaking changes to the HTTP target. Details on migrating can be found [in the migration guide](/docs/api-reference/snowbridge/3-X-X-upgrade-guide/index.md)
 :::
 
 ## Basic authentication
@@ -80,7 +80,7 @@ Data that matches an invalid response rule is sent to the failure target.
 
 `setup` means that this error is not retryable, but is something which can only be resolved by a change in configuration or a change to the target. An example of this is an authentication failure - retrying will fix the issue, the resolution is to grant the appropriate permissions, or provide the correct API key.
 
-Data that matches a setup response rule is handled by a retry as determined in the `setup` configuration block of [retry configuration](/docs/destinations/forwarding-events/snowbridge/configuration/retries/index.md).
+Data that matches a setup response rule is handled by a retry as determined in the `setup` configuration block of [retry configuration](/docs/api-reference/snowbridge/configuration/retries/index.md).
 
 `transient` errors are everything else - we assume that the issue is temporary and retrying will resolve the problem. An example of this is being throttled by an API because too much data is being sent at once. There is no explicit configuration for transient - rather, anything that is not configured as one of the other types is considered transient.
 
@@ -92,11 +92,11 @@ Here is an example of the minimum required configuration:
 https://github.com/snowplow/snowbridge/blob/v${versions.snowbridge}/assets/docs/configuration/targets/http-minimal-example.hcl
 `}</CodeBlock>
 
-If you want to use this as a [failure target](/docs/destinations/forwarding-events/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.
+If you want to use this as a [failure target](/docs/api-reference/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.
 Here is an example of every configuration option:
 
 <CodeBlock language="hcl" reference>{`
 https://github.com/snowplow/snowbridge/blob/v${versions.snowbridge}/assets/docs/configuration/targets/http-full-example.hcl
 `}</CodeBlock>
 
-If you want to use this as a [failure target](/docs/destinations/forwarding-events/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.
+If you want to use this as a [failure target](/docs/api-reference/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.

docs/destinations/forwarding-events/snowbridge/configuration/targets/index.md → docs/api-reference/snowbridge/configuration/targets/index.md

@@ -21,4 +21,4 @@ Here is an example of the configuration:
 https://github.com/snowplow/snowbridge/blob/v${versions.snowbridge}/assets/docs/configuration/targets/stdout-full-example.hcl
 `}</CodeBlock>
 
-If you want to use this as a [failure target](/docs/destinations/forwarding-events/snowbridge/concepts/failure-model/index.md#failure-targets), then use `failure_target` instead of `target`.
+If you want to use this as a [failure target](/docs/api-reference/snowbridge/concepts/failure-model/index.md#failure-targets), then use `failure_target` instead of `target`.

docs/destinations/forwarding-events/snowbridge/configuration/targets/kafka.md → docs/api-reference/snowbridge/configuration/targets/kafka.md

@@ -25,12 +25,12 @@ Here is an example of the minimum required configuration:
 https://github.com/snowplow/snowbridge/blob/v${versions.snowbridge}/assets/docs/configuration/targets/kafka-minimal-example.hcl
 `}</CodeBlock>
 
-If you want to use this as a [failure target](/docs/destinations/forwarding-events/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.
+If you want to use this as a [failure target](/docs/api-reference/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.
 
 Here is an example of every configuration option:
 
 <CodeBlock language="hcl" reference>{`
 https://github.com/snowplow/snowbridge/blob/v${versions.snowbridge}/assets/docs/configuration/targets/kafka-full-example.hcl
 `}</CodeBlock>
 
-If you want to use this as a [failure target](/docs/destinations/forwarding-events/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.
+If you want to use this as a [failure target](/docs/api-reference/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.

docs/destinations/forwarding-events/snowbridge/configuration/targets/kinesis.md → docs/api-reference/snowbridge/configuration/targets/kinesis.md

@@ -24,11 +24,11 @@ Here is an example of the minimum required configuration:
 https://github.com/snowplow/snowbridge/blob/v${versions.snowbridge}/assets/docs/configuration/targets/kinesis-minimal-example.hcl
 `}</CodeBlock>
 
-If you want to use this as a [failure target](/docs/destinations/forwarding-events/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.
+If you want to use this as a [failure target](/docs/api-reference/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.
 Here is an example of every configuration option:
 
 <CodeBlock language="hcl" reference>{`
 https://github.com/snowplow/snowbridge/blob/v${versions.snowbridge}/assets/docs/configuration/targets/kinesis-full-example.hcl
 `}</CodeBlock>
 
-If you want to use this as a [failure target](/docs/destinations/forwarding-events/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.
+If you want to use this as a [failure target](/docs/api-reference/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.

docs/destinations/forwarding-events/snowbridge/configuration/targets/pubsub.md → docs/api-reference/snowbridge/configuration/targets/pubsub.md

@@ -22,4 +22,4 @@ The PubSub Target has only two required options, and no optional ones.
 https://github.com/snowplow/snowbridge/blob/v${versions.snowbridge}/assets/docs/configuration/targets/pubsub-full-example.hcl
 `}</CodeBlock>
 
-If you want to use this as a [failure target](/docs/destinations/forwarding-events/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.
+If you want to use this as a [failure target](/docs/api-reference/snowbridge/concepts/failure-model/index.md#failure-targets), then use failure_target instead of target.