Agenda architecture

Overview

This document outlines the architecture for a backend system designed to efficiently distribute event agenda data to clients. The system utilizes Sessionize as the source of truth for session information while implementing a caching and distribution mechanism to prevent API throttling issues during high traffic periods.

Problem Statement

The event's speakers team uses Sessionize to manage the schedule, which exposes an API with session information. However, several challenges exist:

1. The agenda needs frequent updates throughout the event day
2. Sessionize's API has throttling limits
3. Direct client polling would likely exceed these limits during peak usage
4. Some users might be unable to access the agenda during high traffic periods

Solution Architecture

We've designed a serverless architecture that efficiently caches and distributes agenda data while minimizing API calls to Sessionize.

Architecture Diagram

flowchart TD
    A[Sessionize API] --> B[AWS Lambda\nAgenda Fetcher]
    B -->|Put Object| C[S3 Bucket\nAgenda Cache]
    B -->|If Changed| D[GraphQL API\nDummy Mutation]
    D -->|Triggers| E[GraphQL Subscription]
    F[Client Apps] -->|Subscribe| E
    F -->|Query| G[GraphQL API\nResolver]
    G -->|Get Object| C
    H[EventBridge\nScheduler] -->|Trigger every minute| B

Loading

Component Details

1. AWS EventBridge Scheduler

• Triggers the Lambda function every minute
• Ensures regular checks for agenda updates

2. AWS Lambda Function (Agenda Fetcher)

• Fetches session data from Sessionize API
• Calculates MD5 hash of response
• Compares against cached version in S3
• Uses conditional write operations to update S3 if needed
• Triggers GraphQL mutation when updates occur

3. S3 Bucket (Agenda Cache)

• Stores the latest agenda data as JSON
• Uses ETag for version control (MD5 hash)
• Serves as the data source for GraphQL resolvers

4. GraphQL API

• Provides a custom resolver to retrieve agenda data from S3
• Includes subscription mechanism for real-time updates
• Contains a dummy mutation to trigger subscriptions

Sequence Diagrams

Initial Data Flow

sequenceDiagram
    participant EB as EventBridge
    participant LF as Lambda Function
    participant SA as Sessionize API
    participant S3 as S3 Bucket
    participant GQL as GraphQL API
    participant C as Client
    
    EB->>LF: Trigger (every minute)
    LF->>SA: Fetch agenda data
    SA->>LF: Return JSON response
    LF->>LF: Calculate MD5 hash
    LF->>S3: PUT object with ETag (If-Match condition)
    S3->>LF: Confirm update
    LF->>GQL: Invoke dummy mutation
    GQL->>C: Notify via subscription
    C->>GQL: Query for updated agenda
    GQL->>S3: Get agenda data
    S3->>GQL: Return JSON data
    GQL->>C: Deliver updated agenda

Loading

No-Change Scenario

sequenceDiagram
    participant EB as EventBridge
    participant LF as Lambda Function
    participant SA as Sessionize API
    participant S3 as S3 Bucket
    
    EB->>LF: Trigger (every minute)
    LF->>SA: Fetch agenda data
    SA->>LF: Return JSON response
    LF->>LF: Calculate MD5 hash
    LF->>S3: Compare hash with S3 object ETag
    Note over LF,S3: Hashes match (no changes)
    LF->>LF: No further action needed

Loading

Monitoring and Alerts

For effective operation of this system, consider implementing the following monitoring:

1. Lambda CloudWatch Alarms:

   • Error rate exceeding threshold
   • Duration approaching timeout

2. S3 Metrics:

   • Number of PUT operations (to track update frequency)
   • 4xx/5xx errors on operations

3. GraphQL API:

   • API error rate
   • API latency
   • Connection counts for subscriptions

Sample API endpoint

https://sessionize.com/api/v2/d3orwuq1/view/All

Implementation Considerations

1. Error Handling: The Lambda function should implement robust error handling with retry logic for transient failures when accessing the Sessionize API.

2. Rate Limiting: Consider implementing backoff strategies if Sessionize API returns rate limiting errors.

3. Scaling: The architecture is designed to scale automatically for read operations. The single Lambda function updating S3 serves as a throttle for write operations.

4. Cost Optimization: The frequency of the EventBridge trigger can be adjusted based on the expected update frequency to balance cost vs. freshness.

5. Security: Ensure proper IAM permissions are configured for the Lambda function and S3 bucket. Use API keys or other authentication for the GraphQL API.