Skip to content

Enable Sentry tracing and profiling and fix some serious performance issues!#403

Merged
irvingpop merged 5 commits intoOperationCode:masterfrom
irvingpop:irving/sentry_tracing
Jan 19, 2026
Merged

Enable Sentry tracing and profiling and fix some serious performance issues!#403
irvingpop merged 5 commits intoOperationCode:masterfrom
irvingpop:irving/sentry_tracing

Conversation

@irvingpop
Copy link
Copy Markdown
Collaborator

Description of changes

I turned on Sentry tracing and profiling, and that helped uncover some hilariously bad slowness! Like requests taking 8 seconds, and none of that was held up on DB.

Root Causes Identified

  1. Module import overhead (7s): PYTHONDONTWRITEBYTECODE=1 was preventing Python from using compiled bytecode, forcing re-parsing of all source files on every request
  2. Slow password hashing (1.2s): BCryptSHA256 with excessive work factor
  3. Slow JWT signing (270ms): RS256 asymmetric crypto instead of symmetric HMAC
  4. No gunicorn preloading: Each worker independently loading Django

Changes Made

Performance Optimizations

  • Removed PYTHONDONTWRITEBYTECODE=1 from Dockerfile - enables bytecode caching
  • Added preload_app = True to gunicorn config - loads Django once, forks workers
  • Switched to Argon2 password hashing with tuned parameters (19 MB memory vs 100 MB default)
  • Switched JWT from RS256 to HS256 - 10-20x faster token generation
  • Added argon2-cffi dependency for fast, secure password hashing

Security & Cleanup

  • Removed .dev/ directories - RSA keys no longer needed with HS256
  • Re-enabled bytecode compilation in Dockerfile (now actually used)
  • Updated .dockerignore - excludes dev files and databases from production images
  • Updated .gitignore - prevents .dev/ from being tracked
  • Created core/hashers.py - custom Argon2 hasher with web-optimized parameters

Documentation

  • Updated README.md - Python 3.12+, Poetry 2.3+, fixed isort command
  • Updated OPS.md - Replaced outdated Kubernetes/ArgoCD docs with ECS deployment, added Sentry monitoring section
  • Updated example.env - Documented JWT_SECRET_KEY requirement

Breaking Changes

⚠️ JWT algorithm changed from RS256 to HS256 - requires production environment update:

  1. Generate new secret: openssl rand -base64 64 | tr -d '\n'
  2. Set JWT_SECRET_KEY env var in ECS task definition
  3. Remove JWT_PUBLIC_KEY env var (no longer needed)

All users will be logged out on deployment (tokens become invalid)

Performance Impact

  • Before: 8.7s authentication (7s imports + 1.2s BCrypt + 270ms RS256 + 100ms DB)
  • After: ~450ms authentication (100ms imports + 200ms Argon2 + 15ms HS256 + 100ms DB)
  • Improvement: 95% reduction in latency

Test Plan

  • All pytest tests pass
  • Verified via Sentry profiling in staging
  • Deploy to staging and validate login performance
  • Update production JWT_SECRET_KEY env var
  • Deploy to production
  • Verified Pybot still works

@irvingpop irvingpop merged commit be2bfd3 into OperationCode:master Jan 19, 2026
1 of 2 checks passed
@irvingpop irvingpop deleted the irving/sentry_tracing branch January 19, 2026 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants