Skip to content

Optimization of Computation Times and Memory Management

Choose a tag to compare

@MartinHessler MartinHessler released this 18 Jan 19:43
· 17 commits to main since this release

With this release various functionalities in the trend_extrapolation package are optimized regarding computation times and memory management:

  1. The numpy.roll function is replaced by fancy indexing during accessing the change point configurations in the batched change point analysis. This notably reduces the computation time.
  2. The numpy.roll function is avoided during the marginalization of the probability density functions of the change point positions.
  3. The parallel workflow during the marginalization of the probability density functions of the change point positions is optimized. The number of jobs per parallel worker is optimally distributed before starting the parallel processing pool. Each worker gets the same number of jobs if possible. If this is not possible, one worker additionally processes the remainder.
  4. For marginalizing the joint probability density function of the change point configurations, a queue_management is implemented that restarts the pool by a definable number of times with a subset of jobs to avoid memory errors.

This release acts as a preparation for the final PhD release that is planned to represent the documented version in the corresponding PhD thesis.