Skip to content

Functions updates #3645

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 13, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/_includes/content/functions/perms.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Functions have specific roles which can be used for [access management](/docs/se

Access to functions is controlled by two permissions [roles](/docs/segment-app/iam/roles/):

- **Functions Admin:** Create, edit and delete all functions, or a subset of specified functions.
- **Functions Admin:** Create, edit, and delete all functions, or a subset of specified functions.
- **Functions Read-only:** View all functions, or a subset of specified functions.

You also need additional **Source Admin** permissions to enable source functions, connect destination functions to a source, or to deploy changes to existing functions.
60 changes: 32 additions & 28 deletions src/_includes/content/functions/runtime.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,35 @@
Functions use Node.js 14.x.

Functions do not currently support importing dependencies, but you can [contact Segment Support](https://segment.com/help/contact/) to request that one be added.
Functions do not currently support importing dependencies, but you can [contact Segment Support](https://segment.com/help/contact/){:target="_blank"} to request that one be added.

The following dependencies are installed in the function environment by default.

- [`atob v2.1.2`](https://www.npmjs.com/package/atob) exposed as `atob`
- [`aws-sdk v2.488.0`](https://www.npmjs.com/package/aws-sdk) exposed as `AWS`
- [`btoa v1.2.1`](https://www.npmjs.com/package/btoa) exposed as `btoa`
- [`form-data v2.4.0`](https://www.npmjs.com/package/form-data) exposed as `FormData`
- [`@google-cloud/automl v2.2.0`](https://www.npmjs.com/package/@google-cloud/automl) exposed as `google.cloud.automl`
- [`@google-cloud/bigquery v5.3.0`](https://www.npmjs.com/package/@google-cloud/bigquery) exposed as `google.cloud.bigquery`
- [`@google-cloud/datastore v6.2.0`](https://www.npmjs.com/package/@google-cloud/datastore) exposed as `google.cloud.datastore`
- [`@google-cloud/firestore v4.4.0`](https://www.npmjs.com/package/@google-cloud/firestore) exposed as `google.cloud.firestore`
- [`@google-cloud/functions v1.1.0`](https://www.npmjs.com/package/@google-cloud/functions) exposed as `google.cloud.functions`
- [`@google-cloud/pubsub v2.6.0`](https://www.npmjs.com/package/@google-cloud/pubsub) exposed as `google.cloud.pubsub`
- [`@google-cloud/storage v5.3.0`](https://www.npmjs.com/package/@google-cloud/storage) exposed as `google.cloud.storage`
- [`jsforce v1.11.0`](https://www.npmjs.com/package/jsforce) exposed as `jsforce`
- [`jsonwebtoken v8.5.1`](https://www.npmjs.com/package/jsonwebtoken) exposed as `jsonwebtoken`
- [`lodash v4.17.15`](https://www.npmjs.com/package/lodash) exposed as `_`
- [`moment v2.26.0`](https://www.npmjs.com/package/moment/v/2.26.0) exposed as `moment`
- [`node-fetch v2.6.0`](https://www.npmjs.com/package/node-fetch) exposed as `fetch`
- [`oauth v0.9.15`](https://www.npmjs.com/package/oauth) exposed as `OAuth`
- [`@sendgrid/client v7.4.7`](https://www.npmjs.com/package/@sendgrid/client) exposed as `sendgrid.client`
- [`@sendgrid/mail v7.4.7`](https://www.npmjs.com/package/@sendgrid/mail) exposed as `sendgrid.mail`
- [`stripe v8.115.0`](https://www.npmjs.com/package/stripe) exposed as `stripe`
- [`twilio v3.68.0`](https://www.npmjs.com/package/twilio) exposed as `twilio`
- [`uuidv5 v1.0.0`](https://www.npmjs.com/package/uuidv5) exposed as `uuidv5`
- [`xml v1.0.1`](https://www.npmjs.com/package/xml) exposed as `xml`
- [`xml2js v0.4.23`](https://www.npmjs.com/package/xml2js) exposed as `xml2js`
- [`zlib v1.0.5`](https://www.npmjs.com/package/zlib) exposed as `zlib.zlib`
- [`atob v2.1.2`](https://www.npmjs.com/package/atob){:target="_blank"} exposed as `atob`
- [`aws-sdk v2.488.0`](https://www.npmjs.com/package/aws-sdk){:target="_blank"} exposed as `AWS`
- [`btoa v1.2.1`](https://www.npmjs.com/package/btoa){:target="_blank"} exposed as `btoa`
- [`form-data v2.4.0`](https://www.npmjs.com/package/form-data){:target="_blank"} exposed as `FormData`
- [`@google-cloud/automl v2.2.0`](https://www.npmjs.com/package/@google-cloud/automl){:target="_blank"} exposed as `google.cloud.automl`
- [`@google-cloud/bigquery v5.3.0`](https://www.npmjs.com/package/@google-cloud/bigquery){:target="_blank"} exposed as `google.cloud.bigquery`
- [`@google-cloud/datastore v6.2.0`](https://www.npmjs.com/package/@google-cloud/datastore){:target="_blank"} exposed as `google.cloud.datastore`
- [`@google-cloud/firestore v4.4.0`](https://www.npmjs.com/package/@google-cloud/firestore){:target="_blank"} exposed as `google.cloud.firestore`
- [`@google-cloud/functions v1.1.0`](https://www.npmjs.com/package/@google-cloud/functions){:target="_blank"} exposed as `google.cloud.functions`
- [`@google-cloud/pubsub v2.6.0`](https://www.npmjs.com/package/@google-cloud/pubsub){:target="_blank"} exposed as `google.cloud.pubsub`
- [`@google-cloud/storage v5.3.0`](https://www.npmjs.com/package/@google-cloud/storage){:target="_blank"} exposed as `google.cloud.storage`
- [`jsforce v1.11.0`](https://www.npmjs.com/package/jsforce){:target="_blank"} exposed as `jsforce`
- [`jsonwebtoken v8.5.1`](https://www.npmjs.com/package/jsonwebtoken){:target="_blank"} exposed as `jsonwebtoken`
- [`lodash v4.17.19`](https://www.npmjs.com/package/lodash){:target="\_blank"} exposed as `_`
- [`moment-timezone v0.5.31`](https://www.npmjs.com/package/moment-timezone/v/0.5.31){:target="_blank"} exposed as `moment`
- [`node-fetch v2.6.0`](https://www.npmjs.com/package/node-fetch){:target="_blank"} exposed as `fetch`
- [`oauth v0.9.15`](https://www.npmjs.com/package/oauth){:target="_blank"} exposed as `OAuth`
- [`@sendgrid/client v7.4.7`](https://www.npmjs.com/package/@sendgrid/client){:target="_blank"} exposed as `sendgrid.client`
- [`@sendgrid/mail v7.4.7`](https://www.npmjs.com/package/@sendgrid/mail){:target="_blank"} exposed as `sendgrid.mail`
- [`stripe v8.115.0`](https://www.npmjs.com/package/stripe){:target="_blank"} exposed as `stripe`
- [`twilio v3.68.0`](https://www.npmjs.com/package/twilio){:target="_blank"} exposed as `twilio`
- [`uuidv5 v1.0.0`](https://www.npmjs.com/package/uuidv5){:target="_blank"} exposed as `uuidv5`
- [`xml v1.0.1`](https://www.npmjs.com/package/xml){:target="_blank"} exposed as `xml`
- [`xml2js v0.4.23`](https://www.npmjs.com/package/xml2js){:target="_blank"} exposed as `xml2js`
- [`zlib v1.0.5`](https://www.npmjs.com/package/zlib){:target="_blank"} exposed as `zlib.zlib`

`zlib`'s asynchronous methods `inflate` and `deflate` must be used with `async` or `await`. For example:

```js
Expand All @@ -53,7 +53,11 @@ The following dependencies are installed in the function environment by default.
}
```

Only the [`crypto` Node.js module](https://nodejs.org/dist/latest-v10.x/docs/api/crypto.html ) is included (exposed as `crypto`). [Other built-in Node.js modules](https://nodejs.org/api/modules.html) are not available.
The following Node.js modules are available:
- [`crypto` Node.js module](https://nodejs.org/dist/latest-v10.x/docs/api/crypto.html ){:target="_blank"} exposed as `crypto`.
- [`https` Node.js module](https://nodejs.org/api/https.html){:target="_blank"} exposed as `https`.

[Other built-in Node.js modules](https://nodejs.org/api/modules.html){:target="_blank"} aren't available.

For more information on using the `aws-sdk` module, see how to [set up functions for calling AWS APIs](/docs/connections/functions/aws-apis/).

Expand Down
2 changes: 1 addition & 1 deletion src/connections/destinations/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ All mobile libraries handle retries by periodically attempting to flush their in

#### Retries between Segment and destinations

The destination endpoint APIs have fluctuations in availability due to a number of issues ranging from network failures to bugs to overload. Segment's internal systems retry failed destination API calls for 4 hours with a randomize exponential backoff after each attempt. This substantially improves delivery rates.
The destination endpoint APIs have fluctuations in availability due to a number of issues ranging from network failures to bugs to overload. Segment's internal systems retry failed destination API calls for four hours with a randomized exponential backoff after each attempt. This substantially improves delivery rates.

Here's an example destination that was only successfully accepting 93.36% of all API requests but was achieving a 99.28% final deliverability rate due to Segment's retry functionality.

Expand Down
15 changes: 9 additions & 6 deletions src/connections/functions/destination-functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,15 @@ All functions are scoped to your workspace, so members of other workspaces can't

## Create a destination function

1. From your workspace, go to the Catalog and click the [Functions tab](https://app.segment.com/goto-my-workspace/functions/catalog){:target="_blank"}.
1. From your workspace, go to **Connections > Catalog** and click the [Functions tab](https://app.segment.com/goto-my-workspace/functions/catalog){:target="_blank"}.
2. Click **New Function**.
3. Select **Destination Function** and click **Build**.
3. Select **Destination** as the function type and click **Build**.

After you click **Build**, a code editor appears. Use the editor to write the code for your function, configure settings, and test the function's behavior.

> success ""
> **Tip:** Want to see some example functions? Check out the templates available in the Functions UI, or in the open-source [Segment Functions Library](https://github.com/segmentio/functions-library){:target="_blank"}. (Contributions welcome!)

When you click **Build**, a code editor appears. Use the editor to write the code for your function, configure settings, and test the function's behavior.

## Code the destination function

Expand Down Expand Up @@ -352,7 +353,7 @@ A function can throw errors, or Segment might encounter errors while invoking yo
- **Bad Request** - Any error thrown by the function code that is not covered by the other errors.
- **Invalid Settings** - A configuration error prevented Segment from executing your code. If this error persists for more than an hour, [contact Segment Support](https://segment.com/help/contact/){:target="_blank"}.
- **Message Rejected** - Your code threw `InvalidEventPayload` or `ValidationError` due to invalid input.
- **Unsupported Event Type** - Your code doesn't implement a specific event type (for example, `onTrack()`) or threw a `EventNotSupported` error.
- **Unsupported Event Type** - Your code doesn't implement a specific event type (for example, `onTrack()`) or threw an `EventNotSupported` error.
- **Retry** - Your code threw `RetryError` indicating that the function should be retried.

Segment only attempts to send the event to your destination function again if a **Retry** error occurs.
Expand Down Expand Up @@ -442,7 +443,7 @@ In addition to using [Destination Filters](/docs/connections/destinations/destin
...
```

In the example above, the integrations object directly references and enables the `My Destination Function`. Be sure to include the workspace name in which the Destination Function is created, as shown. Like all items in the integration object, Destination Functions (and workspace names) are case sensitive.
In the example above, the integrations object directly references and enables the `My Destination Function`. Be sure to include the workspace name in which the destination function is created, as shown. Like all items in the integration object, destination functions (and workspace names) are case sensitive.

## Destination functions FAQs

Expand All @@ -452,7 +453,9 @@ Yes, Functions access is logged in the [Audit Trail](/docs/segment-app/iam/audit

##### Does Segment retry failed function invocations?

Segment retries invocations that throw RetryError or Timeout errors for up to four hours. Segment does not retry if your function throws a [non-recoverable error](#errors-and-error-handling).
Yes, Segment retries invocations that throw RetryError or Timeout errors (temporary errors only). Segment's internal system retries failed functions API calls for four hours with a randomized exponential backoff after each attempt. This substantially improves delivery rates.

[Retries](/docs/connections/destinations/#retries-between-segment-and-destinations) work the same for both functions and cloud-mode destinations in Segment.

##### Are events guaranteed to send data in order?

Expand Down
8 changes: 4 additions & 4 deletions src/connections/functions/environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Segment Functions create reusable code that can be run in your Segment workspace
When you create a function, write code for it, and save it, the function appears in the Catalog in your workspace _only_. You can then deploy that function in your workspace just as you would a conventional source or destination.

> note ""
> Access to Functions is controlled by specific [access management roles](#functions-permissions). You may need additional access to create, and deploy functions.
> Access to Functions is controlled by specific [access management roles](#functions-permissions). You may need additional access to create and deploy functions.


## Creating functions
Expand All @@ -29,7 +29,7 @@ After you click **Create Function**, the function appears on the [Functions cata

## Editing a function

If you are a **Workspace Owner** or **Functions Admin**, you can manage your function from the [Functions catalog page](https://app.segment.com/goto-my-workspace/functions/catalog/).
If you are a **Workspace Owner** or **Functions Admin**, you can manage your function from the [Functions catalog page](https://app.segment.com/goto-my-workspace/functions/catalog/){:target="_blank"}.

If you're editing an existing function, you can **Save** changes without changing the behavior of existing instances of the function.

Expand Down Expand Up @@ -58,10 +58,10 @@ When you deploy your destination function in your workspace, you fill out the se

{% include content/functions/perms.md%}

## ️Settings and Secrets
## ️Settings and secrets

{% include content/functions/settings.md %}

## Runtime and Dependencies
## Runtime and dependencies

{% include content/functions/runtime.md %}
12 changes: 6 additions & 6 deletions src/connections/functions/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,15 @@ title: Functions Overview
Functions let you create your own sources and destinations directly within your workspace to bring new types of data into Segment and send data to new tools with just a few lines of JavaScript - no additional infrastructure required.

> info ""
> Functions is available to all customer plan types with a free allotment of usage hours. Read more about [Functions usage limits](/docs/connections/functions/usage/), or see [your workspace's Functions usage stats](https://app.segment.com/goto-my-workspace/settings/usage?metric=functions).
> Functions is available to all customer plan types with a free allotment of usage hours. Read more about [Functions usage limits](/docs/connections/functions/usage/), or see [your workspace's Functions usage stats](https://app.segment.com/goto-my-workspace/settings/usage?metric=functions){:target="_blank"}.

![An image illustrating source functions and destination functions in a Segment workspace](images/functions_overview.png)

## What can you do with Functions?
Functions can help you bring external data into Segment ([Source Functions](/docs/connections/functions/source-functions)) and send data in Segment out to external destinations ([Destination Functions](/docs/connections/functions/destination-functions)). Functions are scoped to your specific workspace. If you're a technology partner and want to build a new integration and publish it in Segment's catalog, see the [Developer Center documentation](/docs/partners/).
Functions can help you bring external data into Segment ([Source functions](/docs/connections/functions/source-functions)) and send data in Segment out to external destinations ([Destination functions](/docs/connections/functions/destination-functions)). Functions are scoped to your specific workspace. If you're a technology partner and want to build a new integration and publish it in Segment's catalog, see the [Developer Center documentation](/docs/partners/).

#### Source Functions
Source Functions receive external data from a webhook and can create Segment events, objects, or both. Source Functions have access to the full power of JavaScript so you can validate and transform the incoming data and even make external API requests to annotate your data.
#### Source functions
Source functions receive external data from a webhook and can create Segment events, objects, or both. Source functions have access to the full power of JavaScript so you can validate and transform the incoming data and even make external API requests to annotate your data.

Use cases:
- Ingest data into Segment from a source that's unavailable in the catalog
Expand All @@ -22,8 +22,8 @@ Use cases:

Learn more about [source functions](/docs/connections/functions/source-functions).

#### Destination Functions
Destination Functions can take events from a Segment source, transform the events, and deliver them to external APIs. Destination Functions can make arbitrary requests to annotate data, as well.
#### Destination functions
Destination functions can take events from a Segment source, transform the events, and deliver them to external APIs. Destination functions can make arbitrary requests to annotate data, as well.

Use cases:
- Send data from Segment to a service that's unavailable in the catalog
Expand Down
Loading