Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix join short circuit when probe is empty #3957

Open
sopel39 opened this issue Jun 7, 2020 · 2 comments
Open

Fix join short circuit when probe is empty #3957

sopel39 opened this issue Jun 7, 2020 · 2 comments

Comments

@sopel39
Copy link
Member

sopel39 commented Jun 7, 2020

Join short circuit on empty probe does not work correctly. Previously (before join was ported to work processors) join operator was blocking on build side (lookupSourceProviderFuture) before getting first page.
Currently, it is still the case due to dynamic filter support. However, waiting for dynamic filters should be correctly solved by: #3414. When #3414 lands, it would be possible to eliminate lookupSourceProviderFuture.isDone() from:

    @Override
    public boolean needsInput()
    {
        return lookupSourceProviderFuture.isDone() && pageBuffer.isEmpty() && !pageBuffer.isFinished();
    }

This would allow to simplify join (make it plain WorkProcessorOperator) and to correctly support short circuiting on empty probe side.

@rohangarg
Copy link
Member

@sopel39 : can this issue be fixed now as you have explained - since #3414 has been merged to master?

@sopel39
Copy link
Member Author

sopel39 commented Sep 2, 2020

I've a draft commits that simplify join. However, I'm not sure if short circuit can be implemented that easily. This is because build side might required to be closed gracefully because of spill support. This requires investigation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants