Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

restoring backup leads to error #1655

Open
scheiblr opened this Issue Jun 29, 2018 · 6 comments

Comments

Projects
None yet
5 participants
@scheiblr
Copy link

scheiblr commented Jun 29, 2018

I tried to restore a backup and get the following error after the database scripts have been processed:

Restoring repositories ...
 * ESID/OSSE-XML-export ... rake aborted!
GRPC::Unavailable: 14:Connect Failed
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/active_call.rb:31:in `check_status'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/active_call.rb:180:in `attach_status_results_and_complete_call'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/active_call.rb:372:in `request_response'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/client_stub.rb:178:in `block in request_response'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/interceptors.rb:170:in `intercept!'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/client_stub.rb:177:in `request_response'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/service.rb:170:in `block (3 levels) in rpc_stub_class'
/home/git/gitlab/lib/gitlab/gitaly_client.rb:139:in `call'
/home/git/gitlab/lib/gitlab/gitaly_client/namespace_service.rb:35:in `gitaly_client_call'
/home/git/gitlab/lib/gitlab/gitaly_client/namespace_service.rb:17:in `add'
/home/git/gitlab/lib/gitlab/shell.rb:297:in `add_namespace'
/home/git/gitlab/app/models/storage/legacy_project.rb:27:in `ensure_storage_path_exists'
/home/git/gitlab/app/models/project.rb:54:in `ensure_storage_path_exists'
/home/git/gitlab/lib/backup/repository.rb:138:in `block in restore'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.10/lib/active_record/relation/batches.rb:51:in `block (2 levels) in find_each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.10/lib/active_record/relation/batches.rb:51:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.10/lib/active_record/relation/batches.rb:51:in `block in find_each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.10/lib/active_record/relation/batches.rb:124:in `find_in_batches'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.10/lib/active_record/relation/batches.rb:50:in `find_each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.10/lib/active_record/querying.rb:9:in `find_each'
/home/git/gitlab/lib/backup/repository.rb:135:in `restore'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:87:in `block (4 levels) in <top (required)>'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:60:in `block (3 levels) in <top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'
Tasks: TOP => gitlab:backup:repo:restore
(See full trace by running task with --trace)
@bradleydwyer

This comment has been minimized.

Copy link

bradleydwyer commented Jul 3, 2018

I'm seeing this too. Did you discover the cause?

EDIT: Ah, I think it was just that I followed some invalid older instructions which turned off too much of the system (instead of just shutting down unicorn/sidekiq).

@scheiblr

This comment has been minimized.

Copy link
Author

scheiblr commented Jul 4, 2018

so, how did you do it? could you please give an example of the commands?

@bradleydwyer

This comment has been minimized.

Copy link

bradleydwyer commented Jul 4, 2018

The instructions from https://docs.gitlab.com/ce/raketasks/backup_restore.html#restore-for-omnibus-installations

I had previously done a gitlab-ctl stop which takes down everything and the gRPC error is obvious in that case.

In the end the restore process is:

sudo gitlab-ctl stop unicorn
sudo gitlab-ctl stop sidekiq
sudo gitlab-rake gitlab:backup:restore BACKUP=<your backup file in /var/opt/gitlab/backups>

And manually copying over gitlab-secrets.json in /etc/gitlab if required (which I did).

To be fair, I'm doing all this in a Docker container on Kubernetes hosted on GKE, so I'm not actually using the artefacts of this repo, but the process should be the same/very similar. I launched the new Gitlab instance in a new container/pod and connected to that pod and executed the above commands after copying my backup to /var/opt/gitlab/backups.

@mikew

This comment has been minimized.

Copy link
Contributor

mikew commented Jul 5, 2018

There are specific instructions for this in this project's README: https://github.com/sameersbn/docker-gitlab#restoring-backups

@scheiblr what was the command you ran?

@serenader2014

This comment has been minimized.

Copy link

serenader2014 commented Jul 15, 2018

I'm having the same issue, follow the backup/restore guide in this project's README always throw the GRPC::Unavailable: 14:Connect Failed error(Gitlab version 11.0.1 and 11.0.3 both have this problem). After some research, I found a way to fix this.

The error is because of some service is not running when you're doing backup or restore, so to fix the problem, just make sure your gitlab is fully running when you are doing backup or restore.

For me, the procedure is:

  1. Starting the gitlab container
  2. Enter the gitlab's bash shell
  3. Execute /sbin/entrypoint.sh app:rake gitlab:backup:restore to restore a backup
  4. When restore finish, just restart gitlab container, and all is done.

For backup procedure, simply execute /sbin/entrypoint.sh app:rake gitlab:backup:create command when you're in gitlab container's shell.

@nanquanyuhao

This comment has been minimized.

Copy link

nanquanyuhao commented Oct 18, 2018

I'm having the same issue, follow the backup/restore guide in this project's README always throw the GRPC::Unavailable: 14:Connect Failed error(Gitlab version 11.0.1 and 11.0.3 both have this problem). After some research, I found a way to fix this.

The error is because of some service is not running when you're doing backup or restore, so to fix the problem, just make sure your gitlab is fully running when you are doing backup or restore.

For me, the procedure is:

1. Starting the gitlab container

2. Enter the gitlab's bash shell

3. Execute `/sbin/entrypoint.sh app:rake gitlab:backup:restore` to restore a backup

4. When restore finish, just restart gitlab container, and all is done.

For backup procedure, simply execute /sbin/entrypoint.sh app:rake gitlab:backup:create command when you're in gitlab container's shell.

I will try...

It's very good.., Very 3Q...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.