Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow snapshot #8914

Closed
abaxanean opened this issue Dec 12, 2014 · 4 comments
Closed

Slow snapshot #8914

abaxanean opened this issue Dec 12, 2014 · 4 comments

Comments

@abaxanean
Copy link

Env : ES 1.4.1, Windows Server 2012 R2 Standard, Intel Xeon E7-4870, ES_HEAP_SIZE = 3g
Having a single node, 45 shards, no replicas, 190 GB of data.
No indexing/searching is running against the node.
Created a repository on the local drive :

{
   "my_backup": {
      "type": "fs",
      "settings": {
         "compress": "true",
         "location": "C:/ES/BackUp",
         "max_snapshot_bytes_per_sec": "-1"
      }
   }
}

Started a snapshot :

PUT _snapshot/my_backup/snapshot_1

After half an hour of running, the backup folder has only 170 MB, the snapshot status is :

      {
         "snapshot": "snapshot_1",
         "repository": "my_backup",
         "state": "STARTED",
         "shards_stats": {
            "initializing": 41,
            "started": 4,
            "finalizing": 0,
            "done": 0,
            "failed": 0,
            "total": 45
         },
         "stats": {
            "number_of_files": 1103,
            "processed_files": 20,
            "total_size_in_bytes": 20020458715,
            "processed_size_in_bytes": 46249254,
            "start_time_in_millis": 0,
            "time_in_millis": 0
         },
         "indices": {
         ...
         }
       }

The disk isn't heavy used at all, the same for memory, cpu (screenshots attached)
What are the recommendations to speed up the snapshot process ?
Fewer shards / shared network disk as storage don't help in any way.

cpu_memory
disk_usage
visual_vm

@tlrx
Copy link
Member

tlrx commented Dec 12, 2014

Thanks for this great report! It looks like you hit the issue resolved by #8748

@clintongormley
Copy link

@bax1989 once 1.4.2 is out, could you try it out and confirm that this is fixed for you please?

@abaxanean
Copy link
Author

I've built 1.4.2-SNAPSHOT from sources and tried it, the issue is fixed.
Is there an approximate date for 1.4.2 release ?

@clintongormley
Copy link

awesome - thanks for checking @bax1989

All I can say is the release will be soon :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants