Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ISPN-14671 HSCAN #11051

Merged
merged 1 commit into from
Jun 28, 2023
Merged

ISPN-14671 HSCAN #11051

merged 1 commit into from
Jun 28, 2023

Conversation

jabolina
Copy link
Member

https://issues.redhat.com/browse/ISPN-14671

Create an iterator manager that handles a Map source. The downside is we have the complete entry loaded in memory to iterate, and the entry could be huge.

karesti
karesti previously approved these changes Jun 22, 2023
Copy link
Contributor

@karesti karesti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any particular thing but I did not do a deep review neither

Copy link
Contributor

@rigazilla rigazilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some considerations.
trying to get confidence with the PR

private CompletionStage<RespRequestHandler> iterate(Resp3Handler handler, IterationManager manager, String cursor,
IterationArguments arguments) {
if (manager == null) {
ByteBufferUtils.stringToByteBuf("*2\r\n$1\r\n0\r\n*0\r\n", handler.allocator());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better to give a meaningful name to this bytes seq

IterableIterationResult.Status status = result.getStatusCode();
if (status == IterableIterationResult.Status.InvalidIteration) {
// Let's just return 0
ByteBufferUtils.stringToByteBuf("*2\r\n$1\r\n0\r\n*0\r\n", handler.allocator());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

@Override
protected CacheStream<CacheEntry<Object, Object>> baseStream(AdvancedCache cache, IterationInitializationContext ctx) {
if (ctx == null) return EMPTY_STREAM;
if (!(ctx instanceof HashMapInitializationContext)) throw new IllegalStateException("Illegal context for iteration");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about turn ctx into a property of the specialized class, remove it from the method params and avoid this check?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this was something I was trying at first. I took this approach because the iteration manager is shared between all clients IIUC. So it would need a way to synchronize between retrieving the entry from the cache and invoking start. All this (trying to synchronize or add a ctx) could be avoided if the start method receives the source stream or something similar. Let me think more about this.


Map<Object, Object> source = ((HashMapInitializationContext) ctx).source;
AbstractLocalCacheStream.StreamSupplier<CacheEntry<Object, Object>, Stream<CacheEntry<Object, Object>>> ss = new HashMapStreamSupplier(source);
return new LocalCacheStream<>(ss, false, SecurityActions.getCacheComponentRegistry(cache));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if we really need a full cache stream even to iterate on an inner struct

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one also was a small paint point. I think this one might be necessary as we can use the existing code to apply the filters. But let me think about this, too.

* Create an iterator manager that handles a HashMap source.
@karesti karesti merged commit d71d49b into infinispan:main Jun 28, 2023
3 of 4 checks passed
@jabolina jabolina deleted the ISPN-14671 branch June 28, 2023 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants