Separate hadoop and native batch docs more #6120
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jihoonson
approved these changes
Aug 7, 2018
jon-wei
force-pushed
the
batch_doc_stuff
branch
from
76edf45
to
790a8e3
Compare
Merged
jihoonson
approved these changes
Aug 8, 2018
|
👍 |
dclim
requested changes
Aug 8, 2018
| If you are having dependency conflicts between Druid and your version of Hadoop, you can try | ||
| searching for a solution in the [Druid user groups](https://groups.google.com/forum/#!forum/druid- | ||
| user), or reading the Druid [Different Hadoop Versions](../operations/other-hadoop.html) documentation. | ||
| Hadoop can be used for batch ingestion. The Hadoop-based batch ingestion will be faster and more scalable than the native batch ingestion. See [here](../ingestion/hadoop.html) for more details. | ||
|
|
||
| ## Command Line Hadoop Indexer | ||
|
|
||
| If you don't want to use a full indexing service to use Hadoop to get data into Druid, you can also use the standalone command line Hadoop indexer. |
There was a problem hiding this comment.
I feel like this 'option' for the command line Hadoop indexer makes more sense as a note at the beginning of the Hadoop indexing page.
There was a problem hiding this comment.
Moved this to the beginning of the hadoop page
docs/content/toc.md
Outdated
| @@ -17,6 +17,8 @@ layout: toc | |||
| * [Schema Design](/docs/VERSION/ingestion/schema-design.html) | |||
| * [Schema Changes](/docs/VERSION/ingestion/schema-changes.html) | |||
| * [Batch File Ingestion](/docs/VERSION/ingestion/batch-ingestion.html) | |||
| * [Native Batch Ingestion](docs/VERSION/native-batch.html) | |||
There was a problem hiding this comment.
/ingestion/native-batch.html?
docs/content/toc.md
Outdated
| @@ -17,6 +17,8 @@ layout: toc | |||
| * [Schema Design](/docs/VERSION/ingestion/schema-design.html) | |||
| * [Schema Changes](/docs/VERSION/ingestion/schema-changes.html) | |||
| * [Batch File Ingestion](/docs/VERSION/ingestion/batch-ingestion.html) | |||
| * [Native Batch Ingestion](docs/VERSION/native-batch.html) | |||
| * [Hadoop Batch Ingestion](docs/VERSION/hadoop.html) | |||
|
|
||
| ## Segment publishing modes | ||
|
|
||
| While ingesting data using the Index task, it creates segments from the input data and publishes them. For segment publishing, the Index task supports two segment publishing modes, i.e., _bulk publishing mode_ and _incremental publishing mode_ for [perfect rollup and best-effort rollup](./design/index.html), respectively. |
There was a problem hiding this comment.
Should be ../design instead of ./design
There was a problem hiding this comment.
i updated this in ingestion/native_tasks.md (current page for native batch indexing with the parallel index task)
dclim
approved these changes
Aug 9, 2018
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Slight adjustment to batch ingestion docs, creating a separate page for hadoop and restructuring the existing batch ingestion page to be a directory page pointing to specific task type docs