Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation for history based optimizer #21830

Closed
tdcmeehan opened this issue Jan 31, 2024 · 6 comments · Fixed by #22065
Closed

Add documentation for history based optimizer #21830

tdcmeehan opened this issue Jan 31, 2024 · 6 comments · Fixed by #22065
Assignees

Comments

@tdcmeehan
Copy link
Contributor

Many prominent improvements are being added to the history based optimizer. We already have documentation for the Redis HBO provider, yet lack documentation for how HBO works and how to enable it. We should add this documentation to encourage its use.

It seems like this should be added to the query optimizer section.

CC: @pranjalssh @mlyublena @feilong-liu @jaystarshot

@mbasmanova
Copy link
Contributor

A list of optimizations:

  • DetermineJoinDistributionType
  • PushPartialAggregationThroughExchange
  • DetermineSemiJoinDistributionType
  • ReorderJoins
  • ScaledWriterRule
  • RandomizeNullKeyInOuterJoin

CC: @mlyublena

@mlyublena
Copy link
Contributor

@feilong-liu is working on adding documentation.

@feilong-liu
Copy link
Contributor

Just drafted a doc on HBO, since it's mainly for Presto user, hence I skip the technical details, but focus on the brief description of the supported optimizers and associated session properties to turn them on. https://docs.google.com/document/d/1Y6ylfV4yfbJ5X5lQwG5gTiC5nSVyV0QoyP0VLyJfLME/edit?usp=sharing

@steveburnett Hi Steve, I know that you are working on Presto documentation, wonder if you could help to polish the draft before adding it to the documentation (and share a email account so that I can assign edit access to you or copy it into another doc whichever works better for you). Thanks!

@steveburnett
Copy link
Contributor

Just drafted a doc on HBO, since it's mainly for Presto user, hence I skip the technical details, but focus on the brief description of the supported optimizers and associated session properties to turn them on. https://docs.google.com/document/d/1Y6ylfV4yfbJ5X5lQwG5gTiC5nSVyV0QoyP0VLyJfLME/edit?usp=sharing

@steveburnett Hi Steve, I know that you are working on Presto documentation, wonder if you could help to polish the draft before adding it to the documentation (and share a email account so that I can assign edit access to you or copy it into another doc whichever works better for you). Thanks!

@feilong-liu I would be happy to help! I see that you've already received some good feedback on the draft in your link. You can use the email address attached to my GitHub profile if you like.

@feilong-liu
Copy link
Contributor

Just drafted a doc on HBO, since it's mainly for Presto user, hence I skip the technical details, but focus on the brief description of the supported optimizers and associated session properties to turn them on. https://docs.google.com/document/d/1Y6ylfV4yfbJ5X5lQwG5gTiC5nSVyV0QoyP0VLyJfLME/edit?usp=sharing
@steveburnett Hi Steve, I know that you are working on Presto documentation, wonder if you could help to polish the draft before adding it to the documentation (and share a email account so that I can assign edit access to you or copy it into another doc whichever works better for you). Thanks!

@feilong-liu I would be happy to help! I see that you've already received some good feedback on the draft in your link. You can use the email address attached to my GitHub profile if you like.

Hi @steveburnett, thanks for helping with it. I've addressed the comments and added you as editor of the doc.

@steveburnett
Copy link
Contributor

Hi @steveburnett, thanks for helping with it. I've addressed the comments and added you as editor of the doc.

Hi @feilong-liu! I'm sorry I took longer than I expected to get to this, but I just completed a light edit of the draft - light mostly because it's an excellent draft, especially after you've already addressed the other reviewers' technical recommendations.
I suggested some minor rephrasings, and added some formatting of items as unordered lists to break up big paragraphs to help readability. When you use this draft in a PR for the Presto docs I'll likely focus on formatting of property names and similar concerns. Thanks for the draft, and for inviting me to contribute, I hope you find my suggestions helpful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Status: Done
Development

Successfully merging a pull request may close this issue.

5 participants