Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HZ-771] Align query-cache and near-cache default behavior #20265

Merged
merged 7 commits into from Jan 17, 2022

Conversation

ahmetmircik
Copy link
Member

@ahmetmircik ahmetmircik commented Dec 22, 2021

closes #19487

Modifications

  • Improved get/getAll/values/keySet/entrySet methods of QueryCache to eliminate extra deserialization when OBJECT in memory format is used.
  • Added serializeKeys flag to QueryCacheConfig to align behavior with NearCache

Docs:
hazelcast/hz-docs#353

Client Protocol Changes: hazelcast/hazelcast-client-protocol#407

Benchmarks

Simulator
test file: hazelcast/hazelcast-simulator#1972

ThisPR(red) vs 5.0.z(blue)
Comparison of QueryCache#values throughput
perf

JMH
5.0.z

Benchmark (entryCount) (keySize) Mode Cnt Score Error Units
QueryCacheBenchmark.testEntries 100000 1024 avgt 4 24438.972 ± 571.089 us/op
QueryCacheBenchmark.testEntriesWithPredicate 100000 1024 avgt 4 7018.963 ± 9050.022 us/op
QueryCacheBenchmark.testGet 100000 1024 avgt 4 1.308 ± 0.390 us/op
QueryCacheBenchmark.testKeys 100000 1024 avgt 4 8008.583 ± 446.637 us/op
QueryCacheBenchmark.testKeysWithPredicate 100000 1024 avgt 4 6351.436 ± 1420.961 us/op
QueryCacheBenchmark.testValues 100000 1024 avgt 4 10298.833 ± 1736.740 us/op
QueryCacheBenchmark.testValuesWithPredicate 100000 1024 avgt 4 6101.246 ± 5076.028 us/op

ThisPR

Benchmark (entryCount) (keySize) Mode Cnt Score Error Units
QueryCacheBenchmark.testEntries 100000 1024 avgt 4 1730.931 ± 913.834 us/op
QueryCacheBenchmark.testEntriesWithPredicate 100000 1024 avgt 4 5615.171 ± 911.053 us/op
QueryCacheBenchmark.testGet 100000 1024 avgt 4 0.135 ± 0.036 us/op
QueryCacheBenchmark.testKeys 100000 1024 avgt 4 1253.615 ± 150.389 us/op
QueryCacheBenchmark.testKeysWithPredicate 100000 1024 avgt 4 5707.071 ± 424.259 us/op
QueryCacheBenchmark.testValues 100000 1024 avgt 4 1310.399 ± 457.784 us/op
QueryCacheBenchmark.testValuesWithPredicate 100000 1024 avgt 4 5499.491 ± 503.329 us/op

@ahmetmircik ahmetmircik added this to the 5.1 milestone Dec 22, 2021
@ahmetmircik ahmetmircik changed the title initial WIP Dec 22, 2021
@ahmetmircik ahmetmircik changed the title WIP [HZ-771] WIP Dec 22, 2021
@ahmetmircik ahmetmircik force-pushed the fix/5.1/QC_and_NC branch 2 times, most recently from 81bdcb5 to 8aa400b Compare December 22, 2021 15:51
@ahmetmircik ahmetmircik changed the title [HZ-771] WIP [HZ-771] Align query-cache and near-cache default behavior Dec 23, 2021
@ahmetmircik ahmetmircik force-pushed the fix/5.1/QC_and_NC branch 2 times, most recently from 96c9bad to 1adbb38 Compare December 24, 2021 10:49
@hazelcast hazelcast deleted a comment from hz-devops-test Dec 24, 2021
@hazelcast hazelcast deleted a comment from hz-devops-test Dec 24, 2021
@hazelcast hazelcast deleted a comment from hz-devops-test Dec 24, 2021
@hazelcast hazelcast deleted a comment from hz-devops-test Dec 24, 2021
@ahmetmircik ahmetmircik force-pushed the fix/5.1/QC_and_NC branch 4 times, most recently from 9fa68ca to 96af0ca Compare December 24, 2021 15:16
@hazelcast hazelcast deleted a comment from hz-devops-test Dec 24, 2021
@hazelcast hazelcast deleted a comment from hz-devops-test Dec 24, 2021
@hazelcast hazelcast deleted a comment from hz-devops-test Dec 24, 2021
@hazelcast hazelcast deleted a comment from hz-devops-test Dec 24, 2021
@ahmetmircik ahmetmircik force-pushed the fix/5.1/QC_and_NC branch 3 times, most recently from b9b2f72 to 048b8e6 Compare December 27, 2021 09:06
@ahmetmircik ahmetmircik marked this pull request as ready for review January 4, 2022 11:09
@ahmetmircik ahmetmircik requested a review from a team as a code owner January 4, 2022 11:09
@ahmetmircik ahmetmircik force-pushed the fix/5.1/QC_and_NC branch 2 times, most recently from 8f04a7c to cb376dc Compare January 7, 2022 14:14
@ahmetmircik
Copy link
Member Author

run-lab-run

@hazelcast hazelcast deleted a comment from hz-devops-test Jan 7, 2022
@ahmetmircik ahmetmircik force-pushed the fix/5.1/QC_and_NC branch 4 times, most recently from ca625d8 to 84e5597 Compare January 11, 2022 15:22
@hazelcast hazelcast deleted a comment from hz-devops-test Jan 11, 2022
Copy link
Contributor

@ramizdundar ramizdundar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall, I have small comments around configs and code structure.

hazelcast/src/main/resources/hazelcast-config-5.1.json Outdated Show resolved Hide resolved
Comment on lines +263 to 267
if (areKeyValueObjectType) {
queryEntry.initWithObjectKeyValue(queryCacheKey, rawValue);
} else {
queryEntry.init(queryCacheKey, rawValue);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fail to understand the difference here. What would happen if we were to only call queryEntry.init(queryCacheKey, rawValue); instead of this if else block?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, it is an optimization and JMH shows it made some difference positively.initWithObjectKeyValue has no checks in it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, thanks, can you leave a small comment for this then. This way someone like me wouldn't simplify it accidentally.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i already did it at the start of method:

// needed for optimization where key and value are not an instance of Data type
final boolean areKeyValueObjectType = !queryCacheConfig.isSerializeKeys()
                && InMemoryFormat.OBJECT == queryCacheConfig.getInMemoryFormat();

@hz-devops-test
Copy link

The job Hazelcast-pr-builder of your PR failed. (Hazelcast internal details: build log, artifacts).
Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log file
--------------------------
-------TEST FAILURE-------
--------------------------
[INFO] Results:
[INFO] 
[ERROR] Errors: 
[ERROR]   YamlConfigBuilderTest.testQueryCacheFullConfig:2718->buildConfig:1843 ? SchemaViolationConfiguration
[INFO] 
[ERROR] Tests run: 46689, Failures: 0, Errors: 1, Skipped: 1015
[INFO] 

[ERROR] There are test failures.

Copy link
Contributor

@ramizdundar ramizdundar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I only have one minor comment: #20265 (comment)

@ahmetmircik ahmetmircik merged commit df234c7 into hazelcast:master Jan 17, 2022
@ahmetmircik ahmetmircik deleted the fix/5.1/QC_and_NC branch January 17, 2022 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Align query-cache and near-cache handling of OBJECT format [HZ-771]
5 participants