Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify compress option in configuration properties #535

Closed
ppf2 opened this issue Aug 17, 2015 · 1 comment
Closed

Clarify compress option in configuration properties #535

ppf2 opened this issue Aug 17, 2015 · 1 comment

Comments

@ppf2
Copy link
Member

ppf2 commented Aug 17, 2015

The description (https://github.com/elastic/elasticsearch-hadoop/tree/master/repository-hdfs#configuration-properties) of the compress option suggests that there is compression of data snapshoted to HDFS. I think we mean to say whether to compress the metadata files (eg. index settings/mappings), not the data itself.

compress: "false"               # optional - whether to compress the data or not (default)

repositories
hdfs:
uri: "hdfs://:/" # optional - Hadoop file-system URI
path: "some/path" # required - path with the file-system where data is stored/loaded
load_defaults: "true" # optional - whether to load the default Hadoop configuration (default) or not
conf_location: "extra-cfg.xml" # optional - Hadoop configuration XML to be loaded (use commas for multi values)
conf. : "" # optional - 'inlined' key=value added to the Hadoop configuration
concurrent_streams: 5 # optional - the number of concurrent streams (defaults to 5)
compress: "false" # optional - whether to compress the data or not (default)
chunk_size: "10mb" # optional - chunk size (disabled by default)

@ppf2 ppf2 added the doc label Aug 17, 2015
@costin
Copy link
Member

costin commented Aug 20, 2015

Fixed - cheers for the report!

@costin costin closed this as completed Aug 20, 2015
costin added a commit that referenced this issue Aug 20, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants