Skip to content

Optimizes report queries and improves session pairing logic#172

Merged
dkmstr merged 1 commit into
v4.0from
dev/v4.0-stats-reports-optimizations
May 8, 2026
Merged

Optimizes report queries and improves session pairing logic#172
dkmstr merged 1 commit into
v4.0from
dev/v4.0-stats-reports-optimizations

Conversation

@aschumann-virtualcable
Copy link
Copy Markdown
Contributor

This pull request introduces several optimizations and correctness improvements to the statistics and reporting modules, focusing on more efficient data aggregation, improved database query patterns, and better handling of time zones. The changes reduce the number of queries (eliminating N+1 issues), leverage database window functions for accurate event pairing, and ensure consistent and correct time handling in reports.

Database Query Optimization and Event Pairing:

  • Replaced multiple per-interval queries with single, bulk queries for time-bucketed statistics in pools_performance.py and user_access.py, significantly improving performance and scalability. [1] [2]
  • Used SQL window functions (Lag over partitions) for accurate LOGIN/LOGOUT event pairing in pool_users_summary.py and usage_by_pool.py, ensuring "last login wins" semantics and eliminating manual event grouping in Python. [1] [2]

Time Zone Handling and Date Formatting:

  • Ensured all date formatting and timestamp conversions are time zone aware using Django's timezone.make_aware, preventing issues with naive datetimes in reports and charts. [1] [2] [3] [4] [5]

Code Simplification and Correctness:

  • Refactored interval/bucket logic to precompute bucket bounds and aggregate results in Python, reducing code complexity and improving maintainability in pools_performance.py and user_access.py. [1] [2]
  • Updated minimum value for sampling points to 2 and enforced bounds in code and UI, preventing invalid chart configurations. [1] [2] [3] [4]

Data Model and API Improvements:

  • Changed pool selection logic to use more efficient queryset and value extraction, avoiding unnecessary model instantiation and improving performance in pools_performance.py and usage_by_pool.py. [1] [2]

Other Minor Improvements:

  • Improved IP address extraction logic and streamlined event processing in usage_by_pool.py.

These changes collectively improve the accuracy, efficiency, and maintainability of the reporting features.

DEV Explain

As we did in version 5.0, we are backporting here the optimizations to report queries and the improvement to the session matching logic.

Refactors report generation to reduce database queries by batching and bucketizing event data in Python, significantly improving performance for time-based aggregations. Updates session calculation to use SQL window functions for accurate LOGIN/LOGOUT pairing, ensuring correctness even with out-of-order events. Standardizes time zone handling and interval validation for consistency across reports.
@dkmstr dkmstr merged commit 57ae9dd into v4.0 May 8, 2026
2 checks passed
@aschumann-virtualcable aschumann-virtualcable deleted the dev/v4.0-stats-reports-optimizations branch May 8, 2026 23:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants