Skip to content

infra(gsi): gsi changes#19

Merged
hackmajoris merged 2 commits into
mainfrom
infra/gsi-optimisation
Dec 20, 2025
Merged

infra(gsi): gsi changes#19
hackmajoris merged 2 commits into
mainfrom
infra/gsi-optimisation

Conversation

@hackmajoris
Copy link
Copy Markdown
Owner

Summary

Optimize DynamoDB single-table design by consolidating from 5 GSIs to 1 GSI, reducing infrastructure costs and complexity while maintaining query performance.

Changes

Infrastructure (deployments/app/cdk.go)

  • Refactored DynamoDB table to use EntityType as partition key instead of entity_id
  • Consolidated 5 Global Secondary Indexes into 1 optimized GSI (BySkill)
  • Removed GSIs: SkillsByLevel, ByUser, SkillsByCategory, ByEntityType, BySkillID
  • New GSI uses composite sort key pattern: SkillName (PK) + ProficiencyLevel (SK)
  • Updated table structure to support entity-type-based queries on main table

Database Layer (cmd/app/internal/database/)

  • Updated constants.go with new GSI name and attribute constants
  • Modified user_skill_repository_dynamodb.go to use new query patterns:
    • ListUsersBySkill: Now queries BySkill GSI with SkillName partition key
    • ListUsersBySkillAndLevel: Queries BySkill GSI with skill name and proficiency level
    • ListUserSkills: Uses main table with EntityType partition key
  • Removed dependencies on deprecated GSIs

Service Layer (cmd/app/internal/service/skill_service.go)

  • Added new methods for cross-user skill queries:
    • ListUsersBySkill(skillName): Find all users with a specific skill
    • ListUsersBySkillAndLevel(skillName, level): Find users by skill and proficiency
  • Enhanced logging for new query operations

Handler Layer (cmd/app/internal/handler/)

  • Added ListUsersBySkillHandler: GET /skills/{skillName}/users
  • Added ListUsersBySkillAndLevelHandler: GET /skills/{skillName}/users?level={level}
  • Implemented path parameter extraction and query parameter parsing
  • Added proper error handling and response formatting

Router (cmd/app/internal/router/router.go)

  • Registered new cross-user skill query endpoints
  • Configured JWT authentication for new routes

Benefits

  • Reduced DynamoDB costs by eliminating 4 GSIs
  • Simplified infrastructure maintenance
  • Maintained all existing query patterns
  • Added new cross-user skill discovery features
  • Improved query efficiency with optimized index design

Query Pattern Support

  • List all users: EntityType = "User" (main table)
  • List all skills: EntityType = "Skill" (main table)
  • User's skills: EntityType = "UserSkill" with entity_id prefix (main table)
  • Users with skill: SkillName = "Python" (BySkill GSI)
  • Users with skill at level: SkillName = "Python" AND ProficiencyLevel = "Expert" (BySkill GSI)

@hackmajoris hackmajoris added the enhancement New feature or request label Dec 20, 2025
Copy link
Copy Markdown
Contributor

@amazon-q-developer amazon-q-developer Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

This PR attempts to optimize DynamoDB design by consolidating GSIs, but contains critical blocking issues that will prevent deployment and cause runtime failures:

🚨 Critical Issues (Must Fix)

  1. DynamoDB Configuration Error: The CDK configuration attempts to define multiple sort keys for a GSI, which is not supported by DynamoDB. This will cause deployment failure.

  2. Query Pattern Mismatch: The database queries use 3-attribute key conditions (Category + SkillName + ProficiencyLevel) but DynamoDB only supports partition key + sort key queries.

  3. Duplicate Code: There's a duplicate PutItem call in the CreateSkill method that will cause the operation to execute twice.

📋 Required Actions

  1. Fix CDK GSI Configuration: Update the GSI to use only one partition key and one sort key
  2. Align Query Patterns: Modify database queries to match DynamoDB's key condition limitations
  3. Remove Duplicate Code: Fix the duplicate PutItem call

💡 Recommendations

Consider using a composite sort key approach where you concatenate values (e.g., SkillName#ProficiencyLevel#YearsOfExperience) as a single sort key, or use FilterExpression for additional filtering on non-key attributes.

The optimization goal is sound, but the implementation needs significant corrections to work with DynamoDB's constraints.


You can now have the agent implement changes and create commits directly on your pull request's source branch. Simply comment with /q followed by your request in natural language to ask the agent to make changes.

Comment thread cmd/app/internal/database/user_skill_repository_dynamodb.go
Comment thread deployments/app/cdk.go
Comment thread cmd/app/internal/database/user_skill_repository_dynamodb.go
@hackmajoris hackmajoris merged commit 18181e1 into main Dec 20, 2025
1 check passed
@hackmajoris hackmajoris deleted the infra/gsi-optimisation branch December 21, 2025 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant