New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FLINK-8668] Document how to set HADOOP_CLASSPATH for Flink #5531
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure if this is the right place. The configuration page as it stands is purely about flink-conf.yaml
and not about environment variables.
In any case we should also add references to the relevant (all?) "Clusters&Deployment" sections.
docs/ops/config.md
Outdated
that are running Flink components. | ||
|
||
When running on YARN, this is usually not a problem because the components | ||
running inside YARN will be started with the Hadoop classpaths anyways but it |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove anyways
and replace it with a comma.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixing
docs/ops/config.md
Outdated
classpath that is used when starting Flink components such as the Client, | ||
JobManager, or TaskManager. Most Hadoop distributions and cloud environments | ||
will not set this variable by default so if the Hadoop classpath should be | ||
picked up by Flink the environment variable should be exported on all machines |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
replace "should" with "must"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixing
We may instead want to add a whole new page under "Clusters&Operations" for hadoop related things. |
@zentol Yes, I struggled with where exactly to put this. I think I will just create a "Hadoop" page under "Clusters&Operations" that has only this section for now. WDYT? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor comment, otherwise +1.
job to YARN. For this, it's usually enough to do a | ||
|
||
``` | ||
export HADOOP_CLASSPATH=`hadoop classpath` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add <
>
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's the hadoop
binary with classpath
as argument
R: @zentol @StephanEwen