Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor job store system for greater scalability #25

Closed
agronholm opened this issue Jul 3, 2012 · 0 comments
Closed

Refactor job store system for greater scalability #25

agronholm opened this issue Jul 3, 2012 · 0 comments

Comments

@agronholm
Copy link
Owner

Originally reported by: Alex Grönholm (Bitbucket: agronholm, GitHub: agronholm)


The job stores currently load all the jobs into memory. This was originally done for performance reasons when jobs are executed frequently. However, this turned out not to reflect real world needs. Instead, real world use cases can require massive amounts of jobs, which would quickly deplete the available memory. As such, the job store semantics will have to be changed so that the jobs are kept in the storage, and are only loaded on demand, based on their next run times. This will cause more frequent hitting of the backend, but it will also enable APScheduler to scale much better.


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant