Specification Generator with AWS Step Functions

This is a prototype application showing how a fully serverless approach to analyse files of different programming languages, and generate a specification document from it.

The logic for this is implemented using AWS Step Functions. The state machine is triggered from an Amazon EventBridge rule when a new object has been uploaded to an input Amazon S3 bucket. The specification itself is generated using an LLM via Amazon Bedrock (supports Claude, Nova, and other models). It is then written into a markdown file, and moved to an output Amazon S3 bucket.

A user is notified via email when a new specification has been generated. This email contains a pre-signed URL for the specification, so that they do not need to have access to the AWS console or AWS CLI.

Project Structure

spec-generation-step-functions/
├── src/
│   ├── lambda/                    # Lambda function implementations
│   │   ├── read-file/            # File reading Lambda
│   │   ├── process-with-claude/  # LLM processing Lambda (supports Claude, Nova, etc.)
│   │   ├── write-specification/  # Specification writing Lambda
│   │   └── send-notification/    # Notification Lambda
│   └── shared/                   # Shared utilities and types
│       ├── types.ts             # TypeScript type definitions
│       ├── utils.ts             # Utility functions
│       └── constants.ts         # Application constants
├── infrastructure/               # AWS CDK infrastructure code
│   ├── stacks/                  # CDK stack definitions
│   │   └── s3-spec-generator-stack.ts
│   ├── config/                  # Environment configuration
│   │   └── environment.ts
│   └── app.ts                   # CDK application entry point
├── deployment/                  # Deployment configurations
│   ├── environments/            # Environment-specific configs
│   │   ├── dev.json
│   │   ├── staging.json
│   │   └── prod.json
│   ├── env-templates/           # Environment variable templates
│   ├── DEPLOYMENT_CHECKLIST.md
│   └── README.md
├── scripts/                     # Deployment and utility scripts
├── test/                        # Test files
│   └── setup.ts
├── test_data/                   # Sample test files
├── package.json                 # Node.js dependencies and scripts
├── tsconfig.json               # TypeScript configuration
├── tsconfig.lambda.json        # Lambda-specific TypeScript config
├── cdk.json                    # CDK configuration
├── jest.config.js              # Jest testing configuration
└── .gitignore                  # Git ignore rules

Getting Started

1. Install dependencies:

   npm install

2. Configure environment settings:

   # Update the environment configuration file for your target environment
   # Edit deployment/environments/dev.json for development
   # Edit deployment/environments/staging.json for staging  
   # Edit deployment/environments/prod.json for production

3. Configure AWS credentials and update the environment JSON file with your settings (especially notificationEmail)

4. Build the project (compiles TypeScript and prepares Lambda functions):

   npm run build

5. Deploy infrastructure:

   # Deploy to development environment
   npm run deploy:dev
   
   # Or deploy to other environments
   npm run deploy:staging
   npm run deploy:prod

Development

• npm run build - Compile TypeScript
• npm run watch - Watch for changes and recompile
• npm test - Run tests
• npm run cdk synth - Synthesize CloudFormation template
• npm run cdk diff - Show differences between deployed and local stack

Architecture

The system uses an event-driven architecture with AWS Step Functions orchestrating the processing pipeline:

1. Files uploaded to S3 input bucket trigger Step Functions execution
2. ReadFileFunction reads and validates the uploaded file
3. LLM processing function sends content to Bedrock (supports Claude, Nova, and other models)
4. WriteSpecificationFunction saves generated specifications to output bucket
5. SNS notifications inform users of processing status

Requirements

• Node.js 18+
• AWS CLI configured
• AWS CDK CLI installed
• TypeScript

Configuration

Environment-specific configuration is managed through JSON files in deployment/environments/:

• dev.json - Development environment settings
• staging.json - Staging environment settings
• prod.json - Production environment settings

Key configuration parameters:

• notificationEmail - Email address for deployment notifications
• region - AWS region for deployment
• parameters.enableXRayTracing - Enable AWS X-Ray tracing
• resources.lambdaMemorySize - Memory allocation for Lambda functions

See .env.example for additional environment variables that can be used during development.