Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-8668] Document how to set HADOOP_CLASSPATH for Flink #5531

Closed

Conversation

aljoscha
Copy link
Contributor

Copy link
Contributor

@zentol zentol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is the right place. The configuration page as it stands is purely about flink-conf.yaml and not about environment variables.

In any case we should also add references to the relevant (all?) "Clusters&Deployment" sections.

that are running Flink components.

When running on YARN, this is usually not a problem because the components
running inside YARN will be started with the Hadoop classpaths anyways but it
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove anyways and replace it with a comma.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixing

classpath that is used when starting Flink components such as the Client,
JobManager, or TaskManager. Most Hadoop distributions and cloud environments
will not set this variable by default so if the Hadoop classpath should be
picked up by Flink the environment variable should be exported on all machines
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace "should" with "must"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixing

@zentol
Copy link
Contributor

zentol commented Feb 20, 2018

We may instead want to add a whole new page under "Clusters&Operations" for hadoop related things.

@aljoscha
Copy link
Contributor Author

@zentol Yes, I struggled with where exactly to put this. I think I will just create a "Hadoop" page under "Clusters&Operations" that has only this section for now. WDYT?

Copy link
Contributor

@zentol zentol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comment, otherwise +1.

job to YARN. For this, it's usually enough to do a

```
export HADOOP_CLASSPATH=`hadoop classpath`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add < >?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's the hadoop binary with classpath as argument

@aljoscha aljoscha closed this Feb 21, 2018
@aljoscha aljoscha deleted the jira-8668-doc-hadoop-classpath branch February 21, 2018 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants