[CAMEL-16770] Added JDBC Idempotent Repository caching #5767
Conversation
|
@davsclaus there is absolutely no rush on this. I just wanted to check that I did everything you needed from me on this PR and associated Jira ticket. I think it's right but if it's sitting because I forgot something, I'd like to address right away. Thank you! |
| private int hitCount; | ||
| private int missCount; | ||
| private String queryAllString | ||
| = "SELECT messageId, CAST(COUNT(*) AS INTEGER) AS messageCount FROM CAMEL_MESSAGEPROCESSED WHERE processorName = ? GROUP BY messageId"; |
There was a problem hiding this comment.
Is CAST a standard SQL function?
There was a problem hiding this comment.
Actually, you brought up a good question for me around jdbcTemplate. The SQL statement clause for COUNT(*) returns a long on some SQL servers and an integer on others. While I was able to solve the problem on the SQL side for this low counts by casting (ANSI SQL-92 feature) to an integer, I feel like this should be done like the existing code in JdbcMessageIdRepository.java but I don't know how to cast two columns using jdbcTemplate. Are you or another Camel developer familiar with this so I can make this match the existing code style?
There was a problem hiding this comment.
In the code of the jdbc template it checks for what type you cast to Long vs Integer and does at rs.getInteger vs rs.getLong and then its up to the JDBC driver how to handle that.
If the number can be really really big, then we should maybe change that code to use Long instead of Integer.
There was a problem hiding this comment.
Ideally the number for an idempotent repository is 1 per processorName/messageId combination. To be honest, I'm not even sure why the count is even involved in the code. I just respected it for either past or future implementations. The documentation for the camel-sql component even says:
When working with concurrent consumers it is crucial to create a unique constraint on the columns
processorNameandmessageId. Because the syntax for this constraint differs from database to database, we do not show it here.
This would result in the COUNT(*) always being 1.
The Long or Integer expectation comes from COUNT(*) and its relationship to the maximum number of rows for the underlying SQL/JDBC implementation. For Derby, it's an Integer and for Snowflake, it's a Long. For this use case of idempotency, there's no reasonable way it could be an actual long. I would keep the existing integer code.
I'll look into how to properly cast at the JDBC level and try to keep the SQL code a little more generic.
Brief documentation added. Not really sure it needs more than "use this if you want the cache behavior". Would you expect more? |
…ng-jdbc-idempotent
|
Okay. This seems closer to what we want. The |
For slow per-query SQL implementations like Snowflake, this new class caches the JDBC Idempotent Repository in one query. Just like the SQL Component's batch mode, this makes interactions with the server performant.