Skip to content

HADOOP-18026. Fix default value of Magic committer #3723

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Nov 29, 2021
Merged

HADOOP-18026. Fix default value of Magic committer #3723

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
3c51579
Merge pull request #1 from apache/trunk
GuoPhilipse Nov 29, 2020
a449d04
Merge pull request #2 from apache/trunk
GuoPhilipse Dec 12, 2020
d4f968b
Merge pull request #3 from apache/trunk
GuoPhilipse Apr 7, 2021
d522abf
Merge pull request #4 from apache/trunk
GuoPhilipse Apr 21, 2021
de633b8
Merge pull request #5 from apache/trunk
GuoPhilipse Oct 16, 2021
66ce06b
Merge pull request #6 from apache/trunk
GuoPhilipse Nov 8, 2021
ee06d41
'triggerrebuild'
tclxgf13871 Nov 25, 2021
eeada77
'improvedoc'
tclxgf13871 Nov 25, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/committers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -547,7 +547,7 @@ Conflict management is left to the execution engine itself.
|--------|---------|---------|
| `mapreduce.fileoutputcommitter.marksuccessfuljobs` | Write a `_SUCCESS` file on the successful completion of the job. | `true` |
| `fs.s3a.buffer.dir` | Local filesystem directory for data being written and/or staged. | `${hadoop.tmp.dir}/s3a` |
| `fs.s3a.committer.magic.enabled` | Enable "magic committer" support in the filesystem. | `false` |
| `fs.s3a.committer.magic.enabled` | Enable "magic committer" support in the filesystem. | `true` |
| `fs.s3a.committer.abort.pending.uploads` | list and abort all pending uploads under the destination path when the job is committed or aborted. | `true` |
| `fs.s3a.committer.threads` | Number of threads in committers for parallel operations on files. | 8 |
| `fs.s3a.committer.generate.uuid` | Generate a Job UUID if none is passed down from Spark | `false` |
Expand Down