Refactor HistoricalTrendService for performance optimization

[phonghpham]

Resolves #4116

Description

Issue #4116 highlighted prformance concerns with HistoricalDataCacheJob. Upon investigation, this job calls out to HistoricalTrendService where n+1 query was identified when items were queried and then for each item a query made to check if there are eligible line_items for item. New implementation replaces this check by preloading filtered line_items.

Additionally, moved aggregation of line_items into ruby instead of via db queries to reduce db load.

Benchmark measure of historical_trend_service_spec.rb call to HistoricalTrendService.series results before refactoring:

  #series performance
  0.087621   0.008420   0.096041 (  0.147053)
  0.014439   0.001048   0.015487 (  0.020131)
  0.014943   0.001097   0.016040 (  0.020658)
  0.015538   0.001313   0.016851 (  0.021404)
  0.013140   0.001055   0.014195 (  0.018298)
  0.012821   0.001000   0.013821 (  0.017912)
  0.012873   0.001067   0.013940 (  0.019809)
  0.012985   0.001229   0.014214 (  0.019213)
  0.014079   0.000990   0.015069 (  0.019707)
  0.012705   0.001032   0.013737 (  0.018755)

and after:

  #series performance
  0.067465   0.006804   0.074269 (  0.117842)
  0.000854   0.000076   0.000930 (  0.001406)
  0.000619   0.000026   0.000645 (  0.001147)
  0.000517   0.000025   0.000542 (  0.001021)
  0.000548   0.000047   0.000595 (  0.001053)
  0.000502   0.000018   0.000520 (  0.000970)
  0.000494   0.000021   0.000515 (  0.001167)
  0.000535   0.000034   0.000569 (  0.001194)
  0.000488   0.000025   0.000513 (  0.001114)
  0.000480   0.000020   0.000500 (  0.001255)

Type of change

• Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

• existing tests not broken by refactor

[dorner]

Looks good! Lint is failing though 😄

[github-actions]

@phonghpham: Your PR Refactor HistoricalTrendService for performance optimization is part of today's Human Essentials production release: 2024.03.03.
Thank you very much for your contribution!

[cielf]

Hey @phonghpham Just thought you'd like to know that in the week since your fix was released, we have had no time out errors on the historical trends. Thanks again!