New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compressed snapshot for backing up and restoring #646

Closed
weiqiangyuan opened this Issue Jan 7, 2016 · 2 comments

Comments

Projects
None yet
2 participants
@weiqiangyuan

weiqiangyuan commented Jan 7, 2016

I have two questions as followed:
We use the hdfs as our elasticsearch repositories to back up and restore.We have alread installed the 'elasticsearch-repository-hdfs' plugin in the elasticsearch data nodes.
1)If the snapshot files in the hdfs were compressed, such as in '.gz' format, can we still use the compressed snapshot to restore?

  1. By the way, as mentioned above, we have installed plugins in the data nodes, do we also need to install the 'elasticsearch-repository-hdfs' on the master nodes in order to create snapshot?

Thank you very much!

@costin

This comment has been minimized.

Show comment
Hide comment
@costin

costin Jan 7, 2016

Member

For questions and discussions please use the forum and not the bug tracker.

  1. No. The output folder needs to be in the same format as it was when the snapshot occurred. In this case this means simply gunziping the files.
  2. For consistency, it is best to have the plugin installed across your entire cluster. while the master might not use this plugin, if the node changes behaviour or if somebody tries to access the plugin (in case it's a site one) on the master and it is not there, things might break.
    So to keep things simple, best practice is to install the plugin through out your entire cluster.
Member

costin commented Jan 7, 2016

For questions and discussions please use the forum and not the bug tracker.

  1. No. The output folder needs to be in the same format as it was when the snapshot occurred. In this case this means simply gunziping the files.
  2. For consistency, it is best to have the plugin installed across your entire cluster. while the master might not use this plugin, if the node changes behaviour or if somebody tries to access the plugin (in case it's a site one) on the master and it is not there, things might break.
    So to keep things simple, best practice is to install the plugin through out your entire cluster.

@costin costin closed this Jan 7, 2016

@weiqiangyuan

This comment has been minimized.

Show comment
Hide comment
@weiqiangyuan

weiqiangyuan Jan 16, 2016

Thank you for your answers. As you have said, if the snapshot files in the hdfs were compressed, such as in '.gz' format, we can't use the compressed snapshots to restore.

I have another question:
As the backup progress is incremental, if the existing snapshots were compressed, can we still continuing backing up new snapshots?

Thank you!

weiqiangyuan commented Jan 16, 2016

Thank you for your answers. As you have said, if the snapshot files in the hdfs were compressed, such as in '.gz' format, we can't use the compressed snapshots to restore.

I have another question:
As the backup progress is incremental, if the existing snapshots were compressed, can we still continuing backing up new snapshots?

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment