Release 22.11.2

src/_data/sidenav/main.yml

@@ -219,6 +219,8 @@ sections:
         title: Redshift Cluster and Redshift Connector Limitations
       - path: /connections/storage/warehouses/redshift-tuning
         title: Speeding Up Redshift Queries
+      - path: /connections/storage/warehouses/redshift-useful-sql
+        title: Useful SQL Queries for Redshift
   - path: /connections/test-connections
     title: Testing Connections
   - path: /connections/data-export-options

src/_includes/content/how-a-sync-works.md

@@ -0,0 +1,3 @@
+When Segment loads data into your warehouse, each sync goes through two steps:
+1. **Ping:** Segment servers connect to your warehouse. For Redshift warehouses, Segment also runs a query to determine how many slices a cluster has. Common reasons a sync might fail at this step include a blocked VPN or IP, a warehouse that isn't set to be publicly accessible, or an issue with user permissions or credentials. 
+2. **Load:** Segment de-duplicates the transformed data and loads it into your warehouse. If you have queries set up in your warehouse, they run after the data is loaded into your warehouse. 

src/_includes/content/spec-table-header.md

@@ -1,6 +1,8 @@
+<thead>
 <tr>
   <th>Field</th>
   <th></th>
   <th>Type</th>
   <th>Description</th>
 </tr>
+</thead>

src/_sass/components/_markdown.scss

@@ -308,10 +308,10 @@
 
 }
 
-a[target="_blank"]:not(.reference-button):after {
-  content: url("/docs/images/external-link-alt-solid.svg");
-  margin-left: 4px;
-}
+//a[target="_blank"]:not(.reference-button):after {
+  //content: url("/docs/images/external-link-alt-solid.svg");
+//  margin-left: 4px;
+//}
 
 a.no-icon[target="_blank"]:after {
   content: none
@@ -436,4 +436,4 @@ div.highlighter-rouge {
   color: black;
   font-weight: bold;
   margin-left: 3px
-}
+}

src/connections/sources/catalog/cloud-apps/sendgrid/index.md

@@ -1,34 +1,38 @@
 ---
-title: Sendgrid Source
+title: SendGrid Source
 id: jhr8dT2yHn
 ---
 {% include content/source-region-unsupported.md %}
 
-SendGrid is a trusted platform for transactional email and email marketing. [Visit Website](http://sendgrid.com)
+[SendGrid](http://sendgrid.com) is a trusted platform for transactional email and email marketing.
 
-Take your company's analysis to the next level by **adding Sendgrid as a Source to Segment.** Segment automatically  collects events like `Click` or `Delivered` and objects such as `Recipients` or `Campaigns` and load them into your data warehouse. 
+Take your company's analysis to the next level by **adding SendGrid as a Source to Segment.** Segment automatically  collects events like `Click` or `Delivered` and objects such as `Recipients` or `Campaigns` and loads them into your data warehouse. 
 
 ## Getting Started
 
-1. From the [Source catalog page](https://app.segment.com/goto-my-workspace/sources/catalog) in your Segment workspace, enter "Sendgrid" and select the Sendgrid source that appears.
-2. From the Sendgrid information panel that appears, click **Add source**.
+Adding SendGrid as a Source in Segment requires a SendGrid API key. If you don't yet have a SendGrid API key, first follow these steps within your SendGrid account:
 
-3. Give the Source a name and add any labels to help you organize and filter your sources. 
-   You can give the source any name, but Segment recommends a name that reflects the source itself, as this name autopopulates the schema name. For example, the source name  `Sendgrid` creates the schema `sendgrid`. You can add multiple instances if you have multiple SendGrid accounts.
+1.  Log in to your SendGrid account.
+2.  Navigate to **Settings > API Keys**, then click **General API Key**.
+3.  Name the key and, optionally, adjust its settings.
+4.  Copy the API Key, omitting all spaces.
 
-4. Provide your API Key.  In order to pull information about your contacts, we'll make requests to SendGrid's API with our [sync component](#sync).  You can create an API Key by navigating to **Settings > API Keys**, clicking **General API Key**.
+> info "SendGrid API Key Settings"
+> Segment recommends providing read permissions for **Email Activity** and **Marketing Activity**.
 
-  You will then be prompted to name that key and given the option to adjust the settings.  We recommend providing read permissions for **Email Activity** and **Marketing Activity**.
+To finish adding the SendGrid source, return to your Segment Workspace and follow these steps:
 
-6. Finally, copy the resulting API Key into the Segment interface, taking care to trim any errant trailing spaces from copying and pasting, and press connect.
+1. From the [Source catalog page](https://app.segment.com/goto-my-workspace/sources/catalog) in your Segment workspace, enter **SendGrid** and select the SendGrid source that appears.
+2. From the SendGrid information panel that appears, click **Add source**.
+3. Give the Source a name and add any labels to help you organize and filter your sources.
+   Segment recommends a name that reflects the source itself, as this name populates the schema name. For example, the source name `SendGrid` creates the schema `SendGrid`. You can add multiple instances if you have multiple SendGrid accounts.
+4. Paste the SendGrid API Key you copied above into the Segment interface. Click **Connect**.
+![](images/601347_Key.png)
 
-  ![](images/601347_Key.png)
+6. Copy the auto-generated Webhook URL and paste it into SendGrid's Event Notification settings pane under **Settings > Mail Settings**.
+![](images/694785_Webhook.png)
 
-7. Copy the auto-generated Webhook URL and paste it into SendGrid's Event Notification settings pane under **Settings > Mail Settings**.
-
-  ![](images/694785_Webhook.png)
-
-8. Once you enable the Event Notification, you're good to go! Press **Next**, and then **Finish** to wrap up the set up flow.
+7. Enable Event Notification in SendGrid. Select **Next** and then **Finish** to complete setup.
 
 ### Event URL
 
@@ -38,51 +42,54 @@ SendGrid has a single Event URL location. By using the SendGrid source, you will
 
 ### Sync
 
-SendGrid has a sync component, which means we'll make requests to their API on your behalf on a 3 hour interval to pull the latest data into Segment. In the initial sync, we'll grab all the SendGrid objects (and their corresponding properties) according to the [Collections Table](#collections) below. **Note**: If you don't use Sendgrid's marketing campaigns features, these collections will be empty in Sendgrid and you'll see "Zero data synced" in your runs. The webhook will still be processing activity data (but only activity data) for you though!
+Segment makes requests to the SendGrid API every three hours. In the initial sync, Segment pulls all SendGrid objects (and their corresponding properties) according to the [Collections Table](#collections) below. If you don't use SendGrid's marketing campaigns features, these collections will be empty in SendGrid and you'll see "Zero data synced" in your runs. The webhook still processes activity data.
 
-Our sync component gets resources from SendGrid and forwards them to Segment using an upsert API, so the dimensional data in your warehouse loaded will reflect the latest state of the corresponding resource in SendGrid.  For example, if `lists.recipient_count` goes from `100` to `200` between syncs, on its next flush to your warehouse, that tickets status will be  `200`.
+Segment's sync component pulls and forwards SendGrid resources to Segment using an upsert API. As a result, dimensional data loaded into your warehouse reflects the latest state of the corresponding resource in SendGrid.  For example, if `lists.recipient_count` goes from `100` to `200` between syncs, its status will be `200` on its next flush to your warehouse.
 
-The source syncs and warehouse syncs are independent processes. Source runs pull your data into the Segment Hub, and warehouse runs flush that data to your warehouse. Sources will sync with Segment every 3 hours. Depending on your Warehouses plan, we will push the Source data to your warehouse on the interval associated with your billing plan.
-
-At the moment, we don't support filtering which objects or properties get synced. If you're interested in this feature, [let us know](https://segment.com/help/contact/)!
+The source syncs and warehouse syncs are independent processes. Source runs pull your data into the Segment Hub, and warehouse runs flush that data to your warehouse. Sources sync with Segment every three hours. Depending on your Warehouses plan, Segment pushes the Source data to your warehouse on the interval associated with your billing plan.
 
+> info "SendGrid Syncs"
+> Segment syncs all objects and properties. [Reach out to support](https://segment.com/help/contact/) if you're interested in filtering objects or properties during syncs.
 
 ### Streaming
 
-The SendGrid source also has a streaming component which listens in real time for inbound webhooks from SendGrid's Event Notifications and batches the events to be uploaded on your next warehouse flush. **These events only append to your warehouse.**
+The SendGrid source's streaming component listens in real time for inbound webhooks from SendGrid's Event Notifications. The source batches these events for upload on your next warehouse flush. **These events only append to your warehouse.**
 
 > note ""
 > **NOTE:** If you don't use SendGrid's marketing features, this will be the only data that Segment receives from SendGrid. There isn't a way to retrieve email event history from SendGrid, so you will only have access to data that Segment collected after you successfully enable this component of the source destination.
 
 
 ## Collections
 
-Collections are the groupings of resources we pull from your source. In your warehouse, each collection gets its own table.
+Collections are the groupings of resources Segment pulls from your source. In your warehouse, each collection gets its own table.
 
 **Object** collections are updated with each sync. These are pulled using Segment's sync component.
 
-**Event** collections are append only, represent a user action or activity, and may be likened to fact tables in a traditional data warehouse. **Note:** Unlike traditional events captured by Segment, you can't forward these events to Destinations you've configured in your Segment workspace. You can only sync these events to a supported data warehouse.
+**Event** collections are append only, represent a user action or activity, and may be likened to fact tables in a traditional data warehouse. Unlike traditional events captured by Segment, you can't forward these events to Destinations you've configured in your Segment workspace. You can only sync these events to a supported data warehouse.
 
 
 |  Collection | Type | Description |
 |  ------ | ------ | ------ |
-|  activity | Event | The union of all SendGrid **event** tables. Useful for creating funnels |
-|  _open | Event | Recipient has opened the HTML message. You need to enable Open Tracking for getting this type of event. |
-|  click | Event | Recipient clicked on a link within the message. You need to enable Click Tracking for getting this type of event. |
+|  activity | Event | The union of all SendGrid **event** tables. Useful for creating funnels. |
+|  _open | Event | Recipient has opened the HTML message. Enable Open Tracking to get this type of event. |
+|  click | Event | Recipient clicked on a link within the message. Enable Click Tracking to get this type of event. |
 |  bounce | Event | Receiving server could not or would not accept message. |
 |  delivered | Event | Message has been successfully delivered to the receiving server. |
-|  processed | Event | Triggered when the email is processed |
+|  processed | Event | Triggered when the email is processed. |
 |  dropped | Event | You may see the following drop reasons: Invalid SMTPAPI header, Spam Content (if spam checker app enabled), Unsubscribed Address, Bounced Address, Spam Reporting Address, Invalid, Recipient List over Package Quota |
 |  deferred | Event | Recipient's email server temporarily rejected message. |
 |  unsubscribe | Event | Recipient clicked on message's subscription management link. You need to enable Subscription Tracking for getting this type of event. |
 |  spam_report | Event | Recipient marked message as spam. |
-|  lists | Object | [Groups of contacts](https://sendgrid.com/docs/API_Reference/Web_API_v3/Marketing_Campaigns/contactdb.html). **Will only return data if you're using Marketing Campaign features of SendGrid.** |
-|  segments | Object | [Slices of lists](https://sendgrid.com/docs/API_Reference/Web_API_v3/Marketing_Campaigns/contactdb.html). **Will only return data if you're using Marketing Campaign features of SendGrid.** |
-|  recipients | Object | All contacts who have received an email, with information about their last activities and custom activities. [More Info](https://sendgrid.com/docs/API_Reference/Web_API_v3/Marketing_Campaigns/contactdb.html).  **Will only return data if you're using Marketing Campaign features of SendGrid.** |
-|  campaigns | Object | All campaigns you've created in Sendgrid. [More Info](https://sendgrid.com/docs/API_Reference/Web_API_v3/Marketing_Campaigns/campaigns.html).  **Will only return data if you're using Marketing Campaign features of SendGrid.** |
+|  lists | Object | [Groups of contacts](https://sendgrid.com/docs/API_Reference/Web_API_v3/Marketing_Campaigns/contactdb.html). **Will only return data if you're using SendGrid's Marketing Campaign features.** |
+|  segments | Object | [Slices of lists](https://sendgrid.com/docs/API_Reference/Web_API_v3/Marketing_Campaigns/contactdb.html). **Will only return data if you're using SendGrid's Marketing Campaign features.** |
+|  recipients | Object | All contacts who have received an email, with information about their last activities and custom activities. [More Info](https://sendgrid.com/docs/API_Reference/Web_API_v3/Marketing_Campaigns/contactdb.html).  **Will only return data if you're using SendGrid's Marketing Campaign features.** |
+|  campaigns | Object | All campaigns you've created in SendGrid. [More Info](https://sendgrid.com/docs/API_Reference/Web_API_v3/Marketing_Campaigns/campaigns.html).  **Will only return data if you're using SendGrid's Marketing Campaign features.** |
+
+> info "SendGrid and Personas"
+> SendGrid data is not available in Personas.
 
-## Troubleshooting 
+## Troubleshooting
 
 If you're getting an "Invalid Credentials" error when setting up the SendGrid source, send a direct ping to the [SendGrid Marketing Campaigns API](https://sendgrid.com/docs/API_Reference/Web_API_v3/Marketing_Campaigns/campaigns.html) to test if you're using the correct credentials.
 
-Make sure you whitelist the Segment IP addresses on Sendgrid. [Contact Segment](https://segment.com/help/contact/) for the list of IP addresses to whitelist.
+Make sure you allowlist Segment IP addresses on SendGrid. [Contact Segment](https://segment.com/help/contact/) for the list of IP addresses to allowlist.

src/connections/sources/catalog/libraries/server/java/index.md

@@ -38,7 +38,7 @@ Add to `pom.xml`:
 or if you're using Gradle:
 
 ```bash
-compile 'com.segment.analytics.java:analytics:+'
+implementation 'com.segment.analytics.java:analytics:+'
 ```
 
 ### Initialize the SDK
@@ -74,14 +74,13 @@ We recommend calling `identify` a single time when the user's account is first c
 Example `identify` call:
 
 ```java
+Map<String, String> map = new HashMap();
+map.put("name", "Michael Bolton");
+map.put("email", "mbolton@example.com");
+
 analytics.enqueue(IdentifyMessage.builder()
-    .userId("f4ca124298")
-    .traits(ImmutableMap.builder()
-        .put("name", "Michael Bolton")
-        .put("email", "mbolton@example.com")
-        .build()
-    )
-);
+        .userId("f4ca124298")
+        .traits(map));
 ```
 
 This call is identifying  Michael by his unique User ID (the one you know him by in your database) and labeling him with `name` and `email` traits.

src/connections/sources/catalog/libraries/server/java/quickstart.md

@@ -40,7 +40,7 @@ Here's what it would look like with Maven:
 *or if you're using Gradle:*
 
 ```bash
-compile 'com.segment.analytics.java:analytics:+'
+implementation 'com.segment.analytics.java:analytics:+'
 ```
 
 ## Step 3: Initialize the SDK
@@ -71,14 +71,13 @@ The `identify` message is how you tell Segment who the current user is. It inclu
 Here's what a basic call to `identify` a user might look like:
 
 ```java
+Map<String, String> map = new HashMap();
+map.put("name", "Michael Bolton");
+map.put("email", "mbolton@example.com");
+
 analytics.enqueue(IdentifyMessage.builder()
-    .userId("f4ca124298")
-    .traits(ImmutableMap.builder()
-        .put("name", "Michael Bolton")
-        .put("email", "mbolton@example.com")
-        .build()
-    )
-);
+        .userId("f4ca124298")
+                .traits(map));
 ```
 
 **Note:** The enqueue method takes a `MessageBuilder` instance and not a `Message` instance directly. This is to allow you to use a `MessageTransformer` that applies to all incoming messages and transform or add data. <!-- LR: Can't find that we ever had a doc about this. Read more about it in the [transformer reference docs](/docs/connections/sources/catalog/libraries/server/java#transformer).-->

src/connections/sources/catalog/libraries/server/node/index.md

@@ -437,6 +437,84 @@ analytics.flush(function(err, batch){
 });
 ```
 
+## Long running process
+
+You should call `client.track(...)` and know that events will be queued and eventually sent to Segment. To prevent losing messages, be sure to capture any interruption (for example, a server restart) and call flush to know of and delay the process shutdown.
+
+```js
+import { randomUUID } from 'crypto';
+import Analytics from 'analytics-node'
+
+const WRITE_KEY = '...';
+
+const analytics = new Analytics(WRITE_KEY, { flushAt: 10 });
+
+analytics.track({
+  anonymousId: randomUUID(),
+  event: 'Test event',
+  properties: {
+    name: 'Test event',
+    timestamp: new Date()
+  }
+});
+
+const exitGracefully = async (code) => {
+  console.log('Flushing events');
+  await analytics.flush(function(err, batch) {
+    console.log('Flushed, and now this program can exit!');
+    process.exit(code);
+  });
+};
+
+[
+  'beforeExit', 'uncaughtException', 'unhandledRejection', 
+  'SIGHUP', 'SIGINT', 'SIGQUIT', 'SIGILL', 'SIGTRAP', 
+  'SIGABRT','SIGBUS', 'SIGFPE', 'SIGUSR1', 'SIGSEGV', 
+  'SIGUSR2', 'SIGTERM', 
+].forEach(evt => process.on(evt, exitGracefully));
+
+function logEvery2Seconds(i) {
+    setTimeout(() => {
+        console.log('Infinite Loop Test n:', i);
+        logEvery2Seconds(++i);
+    }, 2000);
+}
+
+logEvery2Seconds(0);
+```
+
+## Short lived process
+
+Short-lived functions have a predictably short and linear lifecycle, so use a queue big enough to hold all messages and then await flush to complete its work.
+
+
+```js
+import { randomUUID } from 'crypto';
+import Analytics from 'analytics-node'
+
+
+async function lambda()
+{
+  const WRITE_KEY = '...';
+  const analytics = new Analytics(WRITE_KEY, { flushAt: 20 });
+  analytics.flushed = true;
+
+  analytics.track({
+    anonymousId: randomUUID(),
+    event: 'Test event',
+    properties: {
+      name: 'Test event',
+      timestamp: new Date()
+    }
+  });
+  await analytics.flush(function(err, batch) {
+    console.log('Flushed, and now this program can exit!');
+  });
+}
+
+lambda();
+```
+
 
 ## Multiple Clients