-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
feat(docs): Private Links official documentation #3466
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
570b4e8
docs: add private networking user-facing documentation
0ski ee4842b
docs: add private networking troubleshooting page
0ski 3a1ab39
docs: remove unimplemented TRIGGER_PRIVATE_LINK env var injection fro…
0ski 2bd750a
docs: link private networking from Drizzle setup guide
0ski 8037c2e
docs: refine private networking overview per review feedback
0ski 19b4ba7
docs: reorder AWS console setup so target group is created before NLB
0ski a8b9289
docs: warn users to wait for NLB to be Active before creating VPC End…
0ski 23da2c0
docs: explain how to enable cross-region access on the VPC Endpoint S…
0ski 957806f
docs: move 'copy endpoint service name' step to after authorizing pri…
0ski 8747494
docs: add AWS console screenshots and remove DNS-exposure references …
0ski 360079f
Update docs/guides/frameworks/drizzle.mdx
0ski 7220ed6
docs: address PR review feedback (plan tier, NLB Lambda bullet, step …
0ski e71a999
docs: clarify that a private connection works across all deployed env…
0ski File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+250 KB
docs/images/priv-connections-network-load-balancer-add-target-group.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,274 @@ | ||
| --- | ||
| title: "Setting up PrivateLink in the AWS Console" | ||
| sidebarTitle: "AWS Console setup" | ||
| description: "Step-by-step guide for exposing a resource from your AWS account to Trigger.dev via PrivateLink." | ||
| --- | ||
|
|
||
| This guide walks through setting up the AWS side of a private connection: a Network Load Balancer (NLB), a target group pointing at the resource you want to expose, and a VPC Endpoint Service that authorizes Trigger.dev to consume it. | ||
|
|
||
| <Info> | ||
| Prefer Terraform? Open the "Add connection" page in the Trigger.dev dashboard and use the | ||
| Terraform wizard to generate a ready-to-apply script. The wizard creates everything described | ||
| below and pre-fills our AWS account ID for you. | ||
| </Info> | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Before you start you'll need: | ||
|
|
||
| - An **AWS account** with permission to create VPC, EC2, and ELB resources | ||
| - A **resource** in a VPC subnet that you want to expose (RDS instance, ElastiCache cluster, internal API, etc.) | ||
| - The **Trigger.dev AWS account ID** — find this on the "Add connection" page in your Trigger.dev dashboard, in the "I have my details" or "Step-by-step guide" cards | ||
| - A **VPC** that contains the resource, with at least one private subnet per Availability Zone you want to serve from | ||
|
|
||
| <Note> | ||
| PrivateLink connections are zonal. If your resource lives in a single AZ, your connection will | ||
| only be available from that AZ. For higher availability, ensure target groups can route to | ||
| multiple AZs. | ||
| </Note> | ||
|
|
||
| ## Step 1: Create a target group pointing at your resource | ||
|
|
||
| The target group is how the NLB will know where to forward traffic. AWS requires a target group when creating a load balancer, so we'll set this up first. | ||
|
|
||
| <Steps> | ||
| <Step title="Open the target groups page"> | ||
| Go to **EC2 → Target Groups → Create target group**. | ||
| </Step> | ||
| <Step title="Choose a target type"> | ||
| - **IP addresses** for RDS, ElastiCache, or any resource you can reach by IP | ||
| - **Instances** for EC2 instances you own | ||
| - **Application Load Balancer** if your resource sits behind an ALB | ||
|
|
||
| For most database use cases, **IP addresses** is correct. NLBs don't support Lambda targets | ||
| directly — if you need to expose a Lambda, put it behind an ALB and use the ALB target type. | ||
|
|
||
| </Step> | ||
| <Step title="Configure the target group"> | ||
| - **Name**: e.g. `trigger-postgres-tg` | ||
| - **Protocol**: TCP | ||
| - **Port**: the port your resource listens on (5432 for Postgres, 6379 for Redis, 3306 for MySQL, etc.) | ||
| - **VPC**: the VPC where your resource lives (this must match the VPC you'll use for the NLB) | ||
| - **Health check protocol**: TCP | ||
|
|
||
|  | ||
|
|
||
| </Step> | ||
| <Step title="Register your targets"> | ||
| Add the IP addresses of the resource. For RDS, look up the writer endpoint's IPs (`dig <endpoint>` from inside the VPC). | ||
| For ElastiCache, use the primary endpoint IPs. | ||
|
|
||
|  | ||
|
|
||
| <Warning> | ||
| RDS and ElastiCache endpoints' IP addresses can change after failover or maintenance. For long-lived | ||
| connections, consider running a small Lambda or sidecar that periodically resolves the DNS name and | ||
| updates the target group, or use a [DNS-resolved](https://aws.amazon.com/blogs/networking-and-content-delivery/hostname-as-target-for-network-load-balancers/) | ||
| target if your setup supports it. | ||
| </Warning> | ||
|
|
||
| </Step> | ||
| <Step title="Create the target group"> | ||
| Click **Create target group**. | ||
| </Step> | ||
| </Steps> | ||
|
|
||
| ## Step 2: Create an internal Network Load Balancer | ||
|
|
||
| The NLB is what PrivateLink exposes to Trigger.dev. It must be **internal** (not internet-facing). | ||
|
|
||
| <Steps> | ||
| <Step title="Open the EC2 console"> | ||
| Go to **EC2 → Load Balancers → Create load balancer** and choose **Network Load Balancer**. | ||
| </Step> | ||
| <Step title="Configure the basics"> | ||
| - **Name**: something descriptive, e.g. `trigger-postgres-nlb` | ||
| - **Scheme**: **Internal** | ||
| - **IP address type**: IPv4 | ||
|
|
||
|  | ||
|
|
||
| </Step> | ||
| <Step title="Choose VPC and subnets"> | ||
| Pick the same VPC as your target group. Select one private subnet per AZ that should serve traffic. | ||
| Each subnet you select adds an availability zone to the endpoint. | ||
|
|
||
|  | ||
|
|
||
| </Step> | ||
| <Step title="Add a TCP listener forwarding to your target group"> | ||
| Under **Listeners and routing**, configure: | ||
|
|
||
| - **Protocol**: TCP | ||
| - **Port**: same as your target group port (5432 for Postgres, 6379 for Redis, etc.) | ||
| - **Default action**: forward to the target group you created in Step 1 | ||
|
|
||
|  | ||
|
|
||
| </Step> | ||
| <Step title="Create the load balancer and wait until it's Active"> | ||
| Click **Create load balancer**. Provisioning takes 1–2 minutes — wait until the NLB's **State** | ||
| column shows **Active** before moving on. The endpoint service in the next step won't list the | ||
| NLB until it's fully active. | ||
| </Step> | ||
| </Steps> | ||
|
|
||
| <Tip> | ||
| Test connectivity from a bastion host or another instance in the same VPC before continuing — | ||
| e.g. `psql -h <nlb-dns-name> -p 5432 -U user -d db`. If the NLB can't reach your resource, the | ||
| PrivateLink connection won't either. | ||
| </Tip> | ||
|
|
||
| ## Step 3: Create a VPC Endpoint Service | ||
|
|
||
| This is the resource that PrivateLink consumers connect to. | ||
|
|
||
| <Note> | ||
| Confirm the NLB from Step 2 is in the **Active** state before starting this step. It won't appear | ||
| in the **Available load balancers** dropdown until it has finished provisioning. | ||
| </Note> | ||
|
|
||
| <Steps> | ||
| <Step title="Open the VPC console"> | ||
| Go to **VPC → Endpoint services → Create endpoint service**. | ||
| </Step> | ||
| <Step title="Configure the endpoint service"> | ||
| - **Name**: optional, but useful for identification, e.g. `trigger-postgres-endpoint` | ||
| - **Load balancer type**: Network | ||
| - **Available load balancers**: select the NLB you created | ||
| - **Require acceptance for endpoint**: **No** (recommended) | ||
|
|
||
|  | ||
|
|
||
| <Note> | ||
| If you set "Require acceptance" to **Yes**, every connection request from Trigger.dev will | ||
| sit in a pending state until you manually approve it. Setting it to **No** lets connections | ||
| come up automatically once the principal is allow-listed. | ||
| </Note> | ||
|
|
||
| </Step> | ||
| <Step title="Skip private DNS"> | ||
| Leave the "Private DNS name" option disabled. Trigger.dev tasks dial the endpoint by its | ||
| private IP, so private DNS isn't needed. | ||
| </Step> | ||
| <Step title="Configure cross-region access (optional)"> | ||
| If your Trigger.dev tasks run in a **different AWS region** from your endpoint service, expand | ||
| the **Supported Regions** section and add the region(s) where Trigger.dev should be allowed to | ||
| create the VPC Endpoint from (for example, add `eu-central-1` if your service is in | ||
| `us-east-1` but tasks run in `eu-central-1`). | ||
|
|
||
| If your tasks and resource are in the same region, you can skip this — same-region access is | ||
| enabled by default. | ||
|
|
||
| <Note> | ||
| Cross-region PrivateLink adds AWS data-transfer cost and ~10–30ms of latency depending on the | ||
| region pair. Prefer same-region when possible. | ||
| </Note> | ||
|
|
||
| </Step> | ||
| <Step title="Create the endpoint service"> | ||
| Click **Create**. The service is created immediately — you'll come back to copy its **Service | ||
| name** once you've authorized Trigger.dev in the next step. | ||
| </Step> | ||
| </Steps> | ||
|
|
||
| ## Step 4: Authorize the Trigger.dev AWS account | ||
|
|
||
| By default, no one can connect to your endpoint service. You need to explicitly allow Trigger.dev's AWS account. | ||
|
|
||
| <Steps> | ||
| <Step title="Open your endpoint service"> | ||
| Go to **VPC → Endpoint services**, select the service you just created. | ||
| </Step> | ||
| <Step title="Open the Allow principals tab"> | ||
| Click the **Allow principals** tab, then **Allow principals**. | ||
| </Step> | ||
| <Step title="Add Trigger.dev's account"> | ||
| Paste the principal ARN in this format, replacing `<account-id>` with the Trigger.dev AWS | ||
| account ID shown in your dashboard: | ||
|
|
||
| ```text | ||
| arn:aws:iam::<account-id>:root | ||
| ``` | ||
|
0ski marked this conversation as resolved.
|
||
|
|
||
|  | ||
|
|
||
| <Warning> | ||
| You will find the correct AWS account ID in the **Add connection** page of the Trigger.dev | ||
| dashboard. Do not assume an account ID — it differs between Trigger.dev environments. | ||
| </Warning> | ||
|
|
||
| </Step> | ||
| <Step title="Click Allow principals"> | ||
| The principal is now authorized to create a VPC Endpoint targeting your service. | ||
| </Step> | ||
| <Step title="Copy the endpoint service name"> | ||
| On the endpoint service detail page, copy the **Service name** value — it looks like | ||
| `com.amazonaws.vpce.us-east-1.vpce-svc-0123abcd...`. You'll paste this into the Trigger.dev | ||
| dashboard in the next step. | ||
|
|
||
|  | ||
|
|
||
| </Step> | ||
| </Steps> | ||
|
|
||
| ## Step 5: Add the connection in Trigger.dev | ||
|
|
||
| <Steps> | ||
| <Step title="Open the dashboard"> | ||
| In Trigger.dev, go to **Organization Settings → Private Connections** and click **Add | ||
| connection**. | ||
| </Step> | ||
| <Step title="Pick the I have my details card"> | ||
| Then fill in: | ||
|
|
||
| - **Friendly name**: a short, human-readable label for this connection. | ||
| - **VPC Endpoint Service name**: paste the `com.amazonaws.vpce.<region>.vpce-svc-...` value from Step 4. | ||
| - **Target region**: the AWS region your endpoint service lives in. | ||
|
|
||
| </Step> | ||
| <Step title="Submit"> | ||
| Submit the form. The connection's status moves through **Pending → Provisioning → Active**. | ||
| Provisioning typically takes 30–90 seconds. | ||
| </Step> | ||
| <Step title="Verify"> | ||
| Once **Active**, the dashboard shows the assigned private IP. Plug it into the | ||
| connection-string environment variable your task already uses (for example, `DATABASE_URL` set | ||
| on the **Environment Variables** page) and your tasks will reach the resource over | ||
| PrivateLink. | ||
| </Step> | ||
| </Steps> | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| See the dedicated [Troubleshooting](/private-networking/troubleshooting) page for common problems | ||
| such as the "Private link not found" wizard error. A few quick checks specific to this setup flow: | ||
|
|
||
| <Expandable title="Status stays at Pending or Provisioning for several minutes"> | ||
| - Confirm Trigger.dev's AWS account ID is in your endpoint service's **Allow principals** list. | ||
| - Confirm the endpoint service is **Available** in the AWS console. | ||
| - Confirm "Require acceptance" is set to **No** on the endpoint service. If it's set to **Yes**, | ||
| the request is sitting in your pending queue and you must approve it manually. | ||
| </Expandable> | ||
|
|
||
| <Expandable title="Status is Active but my task can't connect"> | ||
| - Confirm the NLB has a target registered and the target's health check is passing. | ||
| - Confirm the listener port matches the port your task code is dialing. | ||
| - Confirm the security group on your resource allows inbound traffic from the NLB or the VPC's | ||
| private IP range. | ||
| - Try connecting from inside the VPC first (e.g., a bastion host) to rule out resource-side | ||
| issues. | ||
| </Expandable> | ||
|
|
||
| <Expandable title="Connection works but is slow"> | ||
| - Cross-region connections add ~10–30ms RTT depending on the regions involved. If your tasks run | ||
| in a different region than your resource, expect higher latency. | ||
| - The NLB and target group's health checks influence connection setup time. Tighter health check | ||
| intervals reduce failover time after a backend goes unhealthy. | ||
| </Expandable> | ||
|
|
||
| <Expandable title="I want to remove a connection"> | ||
| Delete the connection from **Organization Settings → Private Connections** in the Trigger.dev | ||
| dashboard. We'll tear down our VPC Endpoint and remove the network policy automatically. You can | ||
| then delete your VPC Endpoint Service, NLB, and target group on the AWS side. | ||
| </Expandable> | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.