Skip to content

Release v0.11.2

Choose a tag to compare

View all tags
@github-actions github-actions released this 28 Jan 14:17
· 32 commits to main since this release
v0.11.2
3b76b91

@quiltdata/benchling-webhook@0.11.2 deploy:notes
bash scripts/release-notes.sh 0.11.2 712023778557.dkr.ecr.us-east-1.amazonaws.com/quiltdata/benchling:0.11.2 false @quiltdata/benchling-webhook

Benchling Webhook Integration for Quilt

The Benchling Webhook creates a seamless connection between Benchling's Electronic Lab Notebook (ELN) and Quilt's Scientific Data Managements System (SDMS) for Amazon S3.
It not only allows you to view Benchling metadata and attachments inside Quilt packages, but also enables users to browse Quilt package descriptions from inside Benchling notebookes.

The webhook works through a Benchling App that must be installed in your Organization by a Benchling Administrator and configured to call your stack's unique webhook (see Installation, below).

Availability

It is available in the Quilt Platform (1.65 or later) or as a standalone CDK stack via the @quiltdata/benchling-webhook npm package.

Functionality

Auto-Packaging

Packaged Notebook

When scientists create notebook entries in Benchling, this webhook automatically:

  • Creates a dedicated Quilt package for each notebook entry
  • Synchronizes metadata from Benchling (experiment IDs, authors, etc.) into that package
  • Copies attachments from that notebook into Amazon S3 as part of the package.
  • Enables orgnizational data discovery by making contents available in ElasticSearch, and metadata available in Amazon Athena.

Package Linking

experiment_id

In addition, Quilt users can 'tag' additional packages by setting the experiment_id (or a custom metadta key) to the display ID of a Benchling notebook, e.g., EXP00001234.

From inside the Quilt Catalog:

  1. Navigate to the package of interest
  2. Click 'Revise Package'
  3. Go the metadata editor in the bottom left
  4. In the bottom row, enter experiment_id as key and the display ID as the value.
  5. Set the commit message and click 'Save'

Benchling App Canvas

App Canvas - Home

The webhook includes a Benchling App Canvas, which allows Benchling users to view, browse, and sync the associated Quilt packages.

  • Clicking the package name opens it in the Quilt Catalog
  • The sync button will open the package or file in QuiltSync, if you have it installed.
  • The Update button refreshes the package, as Benchling only notifies Quilt of changes when the metadata fields are modified.

The canvas also allows you to browse package contents:

App Canvas - Browse

and view package metadata:

App Canvas - Metadata

Inserting a Canvas

If the App Canvas is not already part of your standard notebook template, Benchling users can add it themselves:

  1. Create a notebook entry
  2. Select "Insert" → "Canvas"
  3. Choose "Quilt Package"
  4. After it is inserted, click the "Create" button

App Canvas - Insert

Security Features

Single Authentication Layer:

  • FastAPI HMAC Verification - All webhook requests verified against Benchling secret
  • Signatures computed over raw request body
  • Invalid signatures return 403 Forbidden

Optional Network Filtering:

  • Resource Policy IP Filtering - Free alternative to AWS WAF ($7/month saved)
  • Blocks unknown IPs at API Gateway edge (applies to all endpoints)
  • BREAKING CHANGE (v1.1.0+): Health endpoints NO LONGER exempt from IP filtering
  • External monitoring services must be added to allowlist or IP filtering disabled
  • NLB health checks unaffected (bypass API Gateway)
  • IP filtering does NOT replace authentication (it's defense-in-depth)

Infrastructure Security:

  • Private network (ECS in private subnets, no public IPs)
  • VPC Link encrypted connection between API Gateway and NLB
  • TLS 1.2+ encryption on all API Gateway endpoints
  • CloudWatch audit trail for HMAC verification and resource policy decisions
  • Least-privilege IAM roles

Installation

1. Installing the Benchling App

This requires a Benchling admin to use npx from NodeJS version 18 or later.

1.1 Generate a manifest

npx @quiltdata/benchling-webhook@latest manifest

This will generate an app-manifest.yaml file in your local folder

1.2 Upload the manifest to Benchling

  • Follow Benchling's create and install instructions.
  • Save the App Definition ID, Client ID, and Client Secret for the next step.

2. Configuring the Benchling App

Your command-line environment must have AWS credentials for the account containing your Quilt stack.
All you need to do is use npx to run the package:

npx @quiltdata/benchling-webhook@latest

The wizard will guide you through:

  1. Catalog discovery - Detect your Quilt catalog configuration
  2. Stack validation - Extract settings from your CloudFormation stack
  3. Credential collection - Enter Benchling app credentials
  4. Deployment mode selection:
    • Integrated: Uses your Quilt stack's built-in webhook, if any
    • Standalone: Deploys a separate webhook stack for testing

Note: Configuration is stored in ~/.config/benchling-webhook/ using the XDG Base Directory standard, supporting multiple profiles.

3. Configure Webhook URL

Add the webhook URL (displayed after setup) to your Benchling app settings.

Important: The endpoint URL format is https://{api-id}.execute-api.{region}.amazonaws.com/{stage}/webhook (includes stage prefix like /prod/webhook or /dev/webhook).

4. Test Integration

In Benchling:

  1. Create a notebook entry
  2. Insert Canvas → Select "Quilt Package"
  3. Click "Create"

A Quilt package will be automatically created and linked to your notebook entry.
If you run into problems, contact Quilt Support

Multi-Stack Deployments (v0.9.8+)

Starting with version 0.9.8, you can deploy multiple webhook stacks in the same AWS account/region. This is useful for:

  • Multi-tenant deployments - Separate stacks for each customer
  • Environment isolation - Dev, staging, prod in same account
  • A/B testing - Parallel stacks with different configurations

Profile-Based Stack Names

Each profile automatically gets its own CloudFormation stack:

# Default profile uses legacy name (backwards compatible)
npx @quiltdata/benchling-webhook@latest deploy --profile default
# Creates: BenchlingWebhookStack

# Other profiles get unique names
npx @quiltdata/benchling-webhook@latest deploy --profile sales
# Creates: BenchlingWebhookStack-sales

npx @quiltdata/benchling-webhook@latest deploy --profile customer-acme
# Creates: BenchlingWebhookStack-customer-acme

Custom Stack Names

You can also specify a custom stack name in your profile configuration:

{
  "deployment": {
    "stackName": "MyCustomWebhookStack",
    ...
  }
}

Managing Multiple Stacks

All commands support the --profile flag:

# Deploy a specific profile
npx @quiltdata/benchling-webhook@latest deploy --profile sales

# Check status
npx @quiltdata/benchling-webhook@latest status --profile sales

# View logs
npx @quiltdata/benchling-webhook@latest logs --profile sales

# Destroy stack
npx @quiltdata/benchling-webhook@latest destroy --profile sales

NPM Package

npm install @quiltdata/benchling-webhook@0.11.2

View on npmjs.com: @quiltdata/benchling-webhook@0.11.2

Docker Image

For custom deployments, use the following Docker image:

712023778557.dkr.ecr.us-east-1.amazonaws.com/quiltdata/benchling:0.11.2

Pull and run:

docker pull 712023778557.dkr.ecr.us-east-1.amazonaws.com/quiltdata/benchling:0.11.2

Changes

Added

  • Logs now show which service generated each message with [service-name] labels
  • Logs display relative timestamps (e.g., "4 minutes ago") for easier reading

Fixed

  • Containers automatically restart after secret updates to pick up new values
  • Only services using the updated secret restart (not all services in the stack)
  • Improved tenant name validation in secrets manager

Resources

What's Changed

  • chore(deps): update aws to v3.958.0 by @renovate[bot] in #316
  • fix(deps): update dependency boto3 to v1.42.17 by @renovate[bot] in #315
  • fix(deps): update dependency uvicorn to v0.40.0 by @renovate[bot] in #310
  • Release v0.11.2: Enhanced logging and intelligent secret sync by @drernie in #330

Full Changelog: v0.11.1...v0.11.2

Contributors

drernie and renovate
Assets 3
Loading