Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase wait time for snapshots #104

Closed
EmptyM opened this issue Oct 14, 2019 · 1 comment

Comments

@EmptyM
Copy link
Contributor

@EmptyM EmptyM commented Oct 14, 2019

Sometimes the world gets really big (~30GB) and it takes a very long time to create an snapshot and gamocosm aborts as it takes too long but it does complete. The snapshot took ~40 mins for ~30GB
Logs:

2019 Sep 16 (22:40:47 PDT)app/workers/wait_for_snapshotting_server_worker.rb:35:in `perform'
Digital Ocean took too long to snapshot server. Aborting
2019 Sep 16 (22:40:47 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 256 times
2019 Sep 16 (22:39:58 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 248 times
2019 Sep 16 (22:39:02 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 240 times
2019 Sep 16 (22:37:59 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 232 times
2019 Sep 16 (22:37:09 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 224 times
2019 Sep 16 (22:36:20 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 216 times
2019 Sep 16 (22:35:34 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 208 times
2019 Sep 16 (22:34:36 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 200 times
2019 Sep 16 (22:33:29 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 192 times
2019 Sep 16 (22:32:35 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 184 times
2019 Sep 16 (22:31:40 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 176 times
2019 Sep 16 (22:30:37 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 168 times
2019 Sep 16 (22:29:43 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 160 times
2019 Sep 16 (22:28:49 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 152 times
2019 Sep 16 (22:27:51 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 144 times
2019 Sep 16 (22:26:50 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 136 times
2019 Sep 16 (22:25:47 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 128 times
2019 Sep 16 (22:24:09 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 120 times
2019 Sep 16 (22:23:15 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 112 times
2019 Sep 16 (22:22:13 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 104 times
2019 Sep 16 (22:21:23 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 96 times
2019 Sep 16 (22:20:24 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 88 times
2019 Sep 16 (22:19:32 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 80 times
2019 Sep 16 (22:18:41 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 72 times
2019 Sep 16 (22:17:42 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 64 times
2019 Sep 16 (22:16:53 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 56 times
2019 Sep 16 (22:16:00 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 48 times
2019 Sep 16 (22:15:13 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 40 times
2019 Sep 16 (22:14:18 PDT)app/workers/wait_for_snapshotting_server_worker.rb:32:in `perform'
Still waiting for Digital Ocean server to snapshot, tried 32 times
@Raekye

This comment has been minimized.

Copy link
Member

@Raekye Raekye commented Nov 17, 2019

Hi @EmptyM,

I'm very sorry for the delayed response/update. And thank you for the PR. I've merged it now, although I'm concerned since the old 256 should have been a lot of time already (it was increased from 64 not too long ago). If the issue persists I may have to investigate manually what's going on with Digital Ocean. I'll try to be on time next time

Kind regards,
Raekye

@Raekye Raekye closed this Nov 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.