Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementation of Postgres PollDataDAO with write caching #82

Merged
merged 1 commit into from
Mar 5, 2024

Conversation

bjpirt
Copy link
Contributor

@bjpirt bjpirt commented Feb 21, 2024

Pull Request type

  • Feature

Changes in this PR

  • Split the PostgresPollDataDAO out into its own file
  • Add caching with a periodic flush to the database
  • Add read caching with a configurable stale period
  • Add a postgresql document to the docs to bring all of the Postgres config together

@bjpirt bjpirt force-pushed the poll-data-flush branch 3 times, most recently from 3772d16 to 1da6a84 Compare February 21, 2024 10:27
"spring.flyway.clean-disabled=false"
})
@SpringBootTest
public class PostgresPollDataDAOCacheTest {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for adding test

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're welcome :-)

@bjpirt bjpirt marked this pull request as ready for review February 23, 2024 12:42
@bjpirt bjpirt force-pushed the poll-data-flush branch 2 times, most recently from 0b89593 to 78a20d1 Compare February 29, 2024 12:05
@bjpirt
Copy link
Contributor Author

bjpirt commented Mar 1, 2024

Some examples of this in action - takes a lot of load off the database:
Screenshot 2024-03-01 at 12 01 03
Screenshot 2024-03-01 at 12 00 34
Screenshot 2024-03-01 at 12 00 27

The deploy was at around 11:50 and you can see the drop in resource usage straight away

@bjpirt bjpirt force-pushed the poll-data-flush branch 2 times, most recently from 7c8420a to ba732e3 Compare March 4, 2024 11:23
@v1r3n v1r3n merged commit 71c9f95 into conductor-oss:main Mar 5, 2024
2 checks passed
denniscodes pushed a commit to denniscodes-oss/conductor that referenced this pull request Mar 5, 2024
matiasbur added a commit to preqin/conductor that referenced this pull request May 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants