v1.1.0: Post-Run Script SQS Integration

🚀 New Post-Run Script Architecture

This release introduces a new post-run script approach for SQS integration, replacing the previous workflow.onComplete hook method.

Key Features

📋 Comprehensive Metadata Collection

The new post-run script (scripts/post_run_sqs.py) collects metadata from three sources:

1. Seqera Platform API - Fetches complete workflow details including:

   • Resolved parameters (outdir, sqs_queue_url, etc.)
   • Run status, session ID, launch info
   • Compute environment ID, work directory, duration
   • Full config text

2. TOWER_ Environment Variables* - Captures runtime context:

   • Workflow ID, run name, project/workspace IDs
   • User ID, workflow status, timeline
   • Pipeline URL, revision, Nextflow version

3. WRROC Metadata Files - Parses workflow provenance:

   • Author information (name, ORCID)
   • Runtime platform, programming language
   • Workflow start/end times, run ID
   • License and publication dates

🎯 Benefits Over Previous Approach

• No workflow code changes required - Works with any Nextflow workflow
• Richer metadata - Includes API data and WRROC provenance
• Better error handling - Comprehensive logging and debugging
• Easier testing - Can be run independently of workflows
• No size limits - Script hosted on GitHub, bypasses 1024-byte limit

🔧 Setup Required

Before using the post-run script, you must complete the SQS setup:

1. Configure IAM Permissions - Grant TowerForge roles access to your SQS queue:

   ./setup-sqs.py --profile sales --yes

2. Verify Setup - Confirm permissions are correct:

   ./check-sqs.sh --profile sales

3. Configure Post-Run Hook - In Seqera Platform, set the post-run script to:

   curl -sSfL https://raw.githubusercontent.com/data-yaml/seqera-smoke-test/v1.1.0/scripts/post-run-sqs.sh | bash

4. Launch with SQS Parameters - Workflows must include the SQS queue URL:

   tw launch ... --params.sqs_queue_url='https://sqs.REGION.amazonaws.com/ACCOUNT/QUEUE'

See README-SQS.md for complete setup instructions.

🧪 Quick Test

After setup, verify end-to-end integration:

./test-sqs.sh --yes

📚 Documentation Improvements

• Consolidated README-SQS-SETUP.md into README-SQS.md
• Simplified main README with references to detailed SQS docs
• Removed duplicate and outdated files
• Fixed markdown linting issues

All SQS setup, configuration, and troubleshooting now in one place: README-SQS.md

🔧 What Changed

Added:

• scripts/post_run_sqs.py - Main Python integration script
• scripts/post-run-sqs.sh - Bash launcher
• scripts/post-run-debug.sh - Debugging utility
• Enhanced test-sqs.sh with parameter passing

Modified:

• nextflow.config - Added WRROC provenance support
• lib/tw-common.sh - Added helper functions
• README-SQS.md - Consolidated documentation

Removed:

• README-SQS-SETUP.md - Merged into README-SQS.md
• main-sqs.nf - Replaced by post-run approach
• Duplicate README files

📦 Full Changelog

See PR #6 for detailed changes: #6

---

Note: This release is on the parse-wrroc branch. Merge to main after testing.

v1.0 - Initial Stable Release

Release v1.0

Initial stable release of the Seqera smoke test suite.

Features

• SQS Integration Testing: Comprehensive SQS verification tools and integration tests
• nf-prov (WRROC) Support: Workflow provenance configuration for Nextflow workflows
• Profile-Based Configuration: Support for multiple deployment configurations
• Git Validation: Enhanced git validation and workflow testing
• CLI Test Workflow: Improved command-line testing capabilities

Recent Changes

• Add nf-prov (WRROC) workflow provenance configuration
• Add SQS verification tools and improve git validation
• Add SQS integration test script with validation and verification
• Add profile-based configuration and enhanced CLI test workflow
• Add test-results/ to gitignore

Documentation

See the README.md for installation and usage instructions.