Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upload multiple files to the same index through File Uploader #35363

Closed
Winterflower opened this issue Apr 19, 2019 · 4 comments
Closed

Upload multiple files to the same index through File Uploader #35363

Winterflower opened this issue Apr 19, 2019 · 4 comments
Labels
enhancement New value added to drive a business result :ml

Comments

@Winterflower
Copy link
Contributor

Hi team! I'm really enjoying using the File Uploader, but today I came across a potential issue that might make a good enhancement in the future.

Describe the feature:
Currently the File Uploader workflow on Kibana and Elasticsearch 7.0.0 is the following:

  1. Select file
  2. Check that Summary to see if timefield and related config are ok
  3. Click import, create an index, let's call it "Index A" and tick "Create Index Pattern" if applicable

However, suppose after a while I discover that I have another file whose documents I would like to import into "Index A". When I go through the Import procedure steps listed above and type the existing index name into the index field, I get an "Index name already exists" error and no possibility to upload into an already existing index.

@Winterflower Winterflower added enhancement New value added to drive a business result :ml labels Apr 19, 2019
@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui

@jgowdyelastic
Copy link
Member

This was a conscious decision to dissuade people from ingesting huge amounts of data through the file data viz. As it was not designed to be a replacement for the existing ingest methods.
One work around would be to upload the next file into a new index which will be matched by the index pattern created when adding the first file
e.g.

  1. upload file into index my-data-jan, edit the index pattern to be my-data-*
  2. upload second file into index my-data-feb, deselect the option to create the index pattern.

step 2 could be repeated as many times as needed.

If we wanted to add the option to add data to an existing index, we'd need to ensure the mappings are the same and possibly reuse the existing ingest pipeline.

@droberts195
Copy link
Contributor

This is a duplicate of #31380, so closing in favour of that

@barbixxxa
Copy link

This was a conscious decision to dissuade people from ingesting huge amounts of data through the file data viz. As it was not designed to be a replacement for the existing ingest methods. One work around would be to upload the next file into a new index which will be matched by the index pattern created when adding the first file e.g.

1. upload file into index `my-data-jan`, edit the index pattern to be `my-data-*`

2. upload second file into index `my-data-feb`, deselect the option to create the index pattern.

step 2 could be repeated as many times as needed.

If we wanted to add the option to add data to an existing index, we'd need to ensure the mappings are the same and possibly reuse the existing ingest pipeline.

Worked for me, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New value added to drive a business result :ml
Projects
None yet
Development

No branches or pull requests

5 participants