Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add setting exact_rows_before_limit #25333

Merged
merged 12 commits into from
Jul 23, 2022

Conversation

MaxWk
Copy link
Contributor

@MaxWk MaxWk commented Jun 16, 2021

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Added a setting exact_rows_before_limit (0/1). When enabled, ClickHouse will provide exact value for rows_before_limit_at_least statistic, but with the cost that the data before limit will have to be read completely.
This closes #6613.

Detailed description / Documentation draft:

A setting exact_rows_before_limit = 0|1. When enabled, LimitBlockInputStream will always_read_till_end (we already have this feature in code) and the field rows_before_limit_at_least will have precise value.
#6613

By adding documentation, you'll allow users to try your new feature immediately, not when someone else will have time to document it later. Documentation is necessary for all features that affect user experience in any way. You can add brief documentation draft above, or add documentation right into your patch as Markdown files in docs folder.

If you are doing this for the first time, it's recommended to read the lightweight Contributing to ClickHouse Documentation guide first.

Information about CI checks: https://clickhouse.tech/docs/en/development/continuous-integration/

@abyss7 abyss7 self-assigned this Jun 16, 2021
@robot-clickhouse robot-clickhouse added doc-alert pr-feature Pull request with new product feature labels Jun 16, 2021
Copy link
Contributor

@abyss7 abyss7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changelog entry (and the in-code description) is obscure for users: you should describe the new setting at the user-level scenarios - since typical user doesn't know what is LimitBlockInputStream, where it's located in the pipeline and how it affects typical SELECT queries.

Also please add the test.

@MaxWk MaxWk requested a review from abyss7 June 16, 2021 15:53
@MaxWk
Copy link
Contributor Author

MaxWk commented Jun 16, 2021

Changelog entry (and the in-code description) is obscure for users: you should describe the new setting at the user-level scenarios - since typical user doesn't know what is LimitBlockInputStream, where it's located in the pipeline and how it affects typical SELECT queries.

Also please add the test.

Hi abyss7 ,thank you for your quick reply. I have edit the Changelog entry.

@MaxWk MaxWk marked this pull request as draft June 24, 2021 17:18
@CLAassistant
Copy link

CLAassistant commented Sep 28, 2021

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
3 out of 4 committers have signed the CLA.

✅ MaxWk
✅ alexey-milovidov
✅ qoega
❌ 万康


万康 seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@alexey-milovidov
Copy link
Member

@MaxWk Why it is in draft stage? The change should be quite easy and we need this feature...

Copy link
Member

@alexey-milovidov alexey-milovidov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks alright :)

@alexey-milovidov alexey-milovidov self-assigned this Dec 23, 2021
@alexey-milovidov alexey-milovidov marked this pull request as ready for review December 23, 2021 08:39
@alexey-milovidov alexey-milovidov added the can be tested Allows running workflows for external contributors label Dec 25, 2021
@alexey-milovidov
Copy link
Member

@Mergifyio update

@mergify
Copy link
Contributor

mergify bot commented Dec 25, 2021

update

✅ Branch has been successfully updated

@alexey-milovidov
Copy link
Member

@Mergifyio update

@mergify
Copy link
Contributor

mergify bot commented Dec 28, 2021

update

✅ Branch has been successfully updated

@MaxWk
Copy link
Contributor Author

MaxWk commented Jan 20, 2022

feature

Sorry for late reply, This pull request was forgotten by me, I thought it had been merged

@alexey-milovidov
Copy link
Member

Still not merged - the test about this feature has failed.

@m1khal3v
Copy link

Any news here?

@Cimus
Copy link

Cimus commented Mar 30, 2022

Update

@supermarat
Copy link

really need

@MaxWk
Copy link
Contributor Author

MaxWk commented Apr 5, 2022

update

@MaxWk MaxWk requested review from alexey-milovidov and removed request for abyss7 April 6, 2022 02:08
@MaxWk
Copy link
Contributor Author

MaxWk commented Apr 6, 2022

@alexey-milovidov @abyss7 I'm sorry it took so long to do this, I've been so busy with work lately, please help me review
this

@MaxWk
Copy link
Contributor Author

MaxWk commented Apr 15, 2022

@Mergifyio update

@mergify
Copy link
Contributor

mergify bot commented Apr 15, 2022

update

✅ Branch has been successfully updated

@MaxWk
Copy link
Contributor Author

MaxWk commented Jul 12, 2022

@qoega These errors seems not relate to my update

@qoega
Copy link
Member

qoega commented Jul 23, 2022

Yes. Flaky check just shows that timeout was hit before the number of ribs or wanted to run. Absolutely normal if test is long.

Copy link
Member

@alexey-milovidov alexey-milovidov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code LGTM.

@alexey-milovidov alexey-milovidov merged commit c489ad7 into ClickHouse:master Jul 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
can be tested Allows running workflows for external contributors pr-feature Pull request with new product feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow to provide exact value instead of "rows_before_limit_at_least"
9 participants