Skip to content

Schema Controls best practices #4435

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Mar 17, 2023
Merged
2 changes: 1 addition & 1 deletion src/protocols/enforce/forward-blocked-events.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Select the source you'll forward events to from the Blocked Events and Traits dr

Since forwarding happens server to server, Segment recommends creating a [HTTP Tracking API source](/docs/connections/sources/catalog/libraries/server/http-api/), though any server-side source will work.

![](../images/blocked_event_forwarding.png)
![A screenshot of the blocked events and traits section on the Schema Configuration settings page](../images/blocked_event_forwarding.png)

> note ""
> Only blocked events are forwarded to the source. Events with omitted traits are not forwarded. Instead, Segment inserts a `context.protocols` object into the event payload which contains the omitted properties or traits.
Expand Down
9 changes: 5 additions & 4 deletions src/protocols/enforce/schema-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,12 @@ redirect_from: '/protocols/enforce/'

The Schema Configuration settings for each source can be used to selectively block events, or omit properties and traits from `.track()`, `.identify()` and `.group()` calls. Segment can permanently drop events that are not included in your Tracking Plan, depending on the settings you select. Segment can also block events with invalid properties or invalid property values.

To enable blocking, go to the **Settings** tab for your source, and click on **Schema Controls**. See below for detailed descriptions for each of the configuration settings.
> warning "Blocked events not forwarded to a Source are discarded"
> Blocking is a serious step that you should only do after you have resolved any violations that appear when you first connect a Tracking Plan to a Source. Any blocked events that are not [forwarded to a separate Source](/docs/protocols/enforce/forward-blocked-events) are permanently discarded and cannot be recovered.

**IMPORTANT: It's critical that you feel confident about the quality of your data before you enable blocking. This is a serious step that you should only do after you have resolved any violations that appear when you first connect a Tracking Plan to a Source.**
To enable blocking, go to the **Settings** tab for your source and click on **Schema Configuration**. See below for detailed descriptions for each of the configuration settings.

![](../images/event_blocking.png)
![A screenshot showing the Unplanned Events, Properties and Values table on the Schema Configuration settings page.](../images/event_blocking.png)

> success ""
> You can [export your Source Schema](/docs/connections/destination-data-control/#export-your-source-schema) as a CSV file to quickly audit events from your Tracking Plan.
Expand Down Expand Up @@ -40,7 +41,7 @@ For example, if you include a single `subscription_id` property in the `Subscrip
**IMPORTANT: Unplanned property omission is ONLY supported in cloud-mode Destinations. Unplanned properties will not be omitted when sending to device-mode Destinations.**

## Track Calls - JSON Schema Violations
Setting this dropdown to Block Event will ensure that all events with JSON schema violations (i.e. missing required properties, incorrect property value data types, or invalid regex patterns) will be blocked. A less aggressive option is to select Omit from the dropdown which will simply remove the offending property from the event.
Setting this dropdown to Block Event will ensure that all events with JSON schema violations (for example, missing required properties, incorrect property value data types, or invalid regex patterns) will be blocked. A less aggressive option is to select Omit from the dropdown which will simply remove the offending property from the event.

This is an advanced feature that requires extensive testing and a squeaky clean data set + tracking plan to enable. To get a sense of which events will be blocked, or properties omitted, go to the Violations view for a source and note all events with a violation. For example, if you added a `subscription_id` required property to your `Subscription Cancelled` event in your Tracking Plan, the below track call would be blocked by Protocols, or property omitted, depending on your setting.

Expand Down
10 changes: 8 additions & 2 deletions src/protocols/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ The Tracking Plan to Source relationship is a one-to-many relationship. This mea

### Can I duplicate a Tracking Plan in the Segment UI?

You can duplicate Tracking Plans in the Segment web app by following the [instructions to copy a tracking plan](/docs/protocols/tracking-plan/create/#copy-a-tracking-plan). You can also use the [Tracking Plan API](/docs/protocols/apis-and-extensions/) to copy the underlying JSON schema from one Tracking Plan to another.
You can duplicate Tracking Plans in the Segment web app by following the [instructions to copy a tracking plan](/docs/protocols/tracking-plan/create/#copy-a-tracking-plan). You can also use the [Public API](/docs/protocols/apis-and-extensions/) to copy the underlying JSON schema from one Tracking Plan to another.

### How do I handle versioning with mobile apps?

Expand All @@ -66,6 +66,12 @@ Yes. [Tracking Plan Libraries](/docs/protocols/tracking-plan/libraries/) makes i

No. Unfortunately it's not yet possible to automatically transfer events from a Tracking Plan to Libraries. To import events into a new event library, import them directly from a source.

### Can I transfer a Tracking Plan between production and staging environments?

Yes. Using the [Public API](/docs/protocols/apis-and-extensions/), you can copy the underlying JSON schema from a Tracking Plan in one Workspace to a Tracking Plan in another Workspace.

If you [discarded events](/docs/protocols/enforce/schema-configuration) as a part of your original Tracking Plan, you must connect to the same Source and configure identical Schema Controls in your other Workspace so that blocked events behave as expected.

## Protocols Validation

### What is the difference between Violations Emails and the Violations page in the Segment UI?
Expand Down Expand Up @@ -131,7 +137,7 @@ Only workspace admins are allowed to create transformations.

All users with Protocols admin or read-only permissions can view transformations.

### Why can't we support transformations for device-mode destinations?
### Why can't Segment support transformations for device-mode destinations?

Transformations introduce advanced logic that at scale may impact performance of client-side libraries. If you are interested in testing new functionality which supports device-mode destination transformations in analytics.js, contact your account rep.

Expand Down