New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FLINK-23458][docs] Added the network buffer documentation along wit… #16988
Conversation
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community Automated ChecksLast check on commit e3a0f5f (Wed Aug 25 16:55:38 UTC 2021) Warnings:
Mention the bot in a comment to re-run the automated checks. Review Progress
Please see the Pull Request Review Guide for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commandsThe @flinkbot bot supports the following commands:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a section about manually configuring the amount of in-flight buffers? That for max throughput we recommend the default settings of exclusive buffers and floating buffers. For manually minimising the amount of the in-flight data the exclusive buffers can be set to 0
and the memory segment size can be decreased.
Additionally I think we don't have any warning/info message logged if not all configured amount of buffers was acquired. I think we should add this on the warning level. Especially after FLINK-16641, if we fail to allocate even a single exclusive buffer on the input gates, the performance would suffer dramatically and I think at the moment users would have no way of knowing that this has happened.
Also as we discussed off line, I think the best place for this docs is a sibling of this page: https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/memory/mem_tuning/ And can you add cross references from the configuration and memory configuration docs to this networ memory tuning page? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One last comment from my side. I would ask Daisy to take a look, as I'm pretty sure there are hundreds of linguistic errors that we can not catch (like missing the
, an
, a
🙄 )
@infoverload, thanks for the suggestions. I have applied them all. If you have more comments I will wait for them if not, please, approve the PR if it looks good to you. |
It looks like everybody has finished with the review of this PR so if there are no objections I will squash commits and then we can merge it. @pnowojski , @infoverload |
Yes, please squash the commits. |
… the buffer debloat doc
c3bb931
to
e75ac5d
Compare
…h the buffer debloat doc
What is the purpose of the change
The documentation for network buffer along with buffer debloat
Brief change log
Verifying this change
Does this pull request potentially affect one of the following parts:
@Public(Evolving)
: (yes / no)Documentation