Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extra hdfs directories #569

Merged
merged 2 commits into from
Jun 6, 2022
Merged

Conversation

vjrj
Copy link
Collaborator

@vjrj vjrj commented May 17, 2022

Just some extra hdfs directories that should be created initially. To prevent this error, for instance during clustering:

WARN : Putting block rdd_37_2 failed due to exception org.apache.beam.sdk.util.UserCodeException: org.apache.hadoop.security.AccessControlException: Permission denied: user=spark, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-xs

@vjrj vjrj requested a review from djtfmartin May 17, 2022 09:41
@vjrj
Copy link
Collaborator Author

vjrj commented Jun 6, 2022

I'll rebase this

@vjrj vjrj merged commit a406b3b into AtlasOfLivingAustralia:master Jun 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants