Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow restart operations when aborted or failed #11

Open
juarezr opened this issue May 14, 2020 · 0 comments
Open

Allow restart operations when aborted or failed #11

juarezr opened this issue May 14, 2020 · 0 comments
Labels
enhancement New feature or request

Comments

@juarezr
Copy link
Owner

juarezr commented May 14, 2020

Operations in backup or restore can be interrupted by:

  1. Errors in source docs:
    • Documents without required fields
    • Corrupted documents
    • Generally when user has changed schema after imported docs
  2. Server/Cluster problems:
    • Server overload
    • Huge transaction log when restoring (avoided by using --commit hard)
    • Replication throubles
    • Network problems between cluster instances
    • Out of memory in JVM / Solr
  3. Interruptions:
    • Power failure, computer restart, process killed
    • Lost ssh connection
    • can avoided using the screen command before running solrcopy

Todo:

  1. Create a mecanism for recording checkpoints for backup/restore:
    • Backup can require using the --order for checkpointing
    • Restore already has --order using zip filename that could be extended for checkpointing
  2. Create a new switch like --continue that could:
    • detect when the command failed before
    • restart from the point interrupted
    • can skip in case of invalid/corrupted docs
@juarezr juarezr added the enhancement New feature or request label May 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant