New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

raketask gitlab:backup:restore aborts #1042

Open
ClarkHartness opened this Issue Jan 4, 2017 · 10 comments

Comments

Projects
None yet
6 participants
@ClarkHartness

This appears it may be an Upstream issue but I wanted to post here to see if anyone has had the problem and overcome it in some way.

Current Instance and New Instance are running 8.15.2

rsync backup to new server, chmod 0644 file.

When I execute the docker to run the restore raketask the first sign of an issue is the "date: invalid date"

I can choose the backup file, the task starts but aborts at ** Execute gitlab:backup:restore

Thanks in Advance!
Clark

--trace below:
----- Lines Removed for brevity -----------
Configuring nginx...
Configuring nginx::gitlab...

date: invalid date '@1483543531_2017_01_04'
⣠1483543531_2017_01_04_gitlab_backup.tar (created at )
date: invalid date '@1483533029_2017_01_04'
⣠1483533029_2017_01_04_gitlab_backup.tar (created at )

Select a backup to restore:

Select a backup to restore: 1483533029_2017_01_04_gitlab_backup.tar
Running raketask gitlab:backup:restore...
** Invoke gitlab:backup:restore (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute gitlab:backup:restore
rake aborted!
TypeError: no implicit conversion of nil into String
/home/git/gitlab/lib/backup/manager.rb:122:in exist?' /home/git/gitlab/lib/backup/manager.rb:122:inunpack'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:32:in block (3 levels) in <top (required)>' /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:240:inblock in execute'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in each' /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:inexecute'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:179:in block in invoke_with_call_chain' /usr/lib/ruby/2.3.0/monitor.rb:214:inmon_synchronize'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:172:in invoke_with_call_chain' /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:165:ininvoke'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:150:in invoke_task' /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:inblock (2 levels) in top_level'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in each' /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:inblock in top_level'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:115:in run_with_threads' /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:100:intop_level'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:78:in block in run' /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:176:instandard_exception_handling'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:75:in run' /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/bin/rake:33:in<top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/bin/rake:22:in load' /home/git/gitlab/vendor/bundle/ruby/2.3.0/bin/rake:22:in

'
Tasks: TOP => gitlab:backup:restore

@ClarkHartness

This comment has been minimized.

Show comment
Hide comment
@ClarkHartness

ClarkHartness Jan 4, 2017

I was able to Workaround this issue by running the raketask through docker exec

docker exec -it gitlab sudo -HEu git bundle exec rake gitlab:backup:restore RAILS_ENV=production

I had to remove all but one backup as I did not try to feed the backup directly to the rake command.

I will play around with it a bit before I put this instance into production.

I was able to Workaround this issue by running the raketask through docker exec

docker exec -it gitlab sudo -HEu git bundle exec rake gitlab:backup:restore RAILS_ENV=production

I had to remove all but one backup as I did not try to feed the backup directly to the rake command.

I will play around with it a bit before I put this instance into production.

@adamniedzielski

This comment has been minimized.

Show comment
Hide comment
@adamniedzielski

adamniedzielski Jan 10, 2017

The relevant issue in GitLab issue tracker is https://gitlab.com/gitlab-org/gitlab-ce/issues/26090. It will be fixed by https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8477 and released in a patch release to 8.15.

As a heads up to the maintainers of this project: we changed the timestamp format from 1483533029 to 1483533029_2017_01_04 in 8.15. If your code relies on the previous format of the timestamp (which may be the case because of date: invalid date in the logs), please update it. I'm sorry for the inconvenience!

The relevant issue in GitLab issue tracker is https://gitlab.com/gitlab-org/gitlab-ce/issues/26090. It will be fixed by https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8477 and released in a patch release to 8.15.

As a heads up to the maintainers of this project: we changed the timestamp format from 1483533029 to 1483533029_2017_01_04 in 8.15. If your code relies on the previous format of the timestamp (which may be the case because of date: invalid date in the logs), please update it. I'm sorry for the inconvenience!

@Niceplace

This comment has been minimized.

Show comment
Hide comment
@Niceplace

Niceplace Jan 16, 2017

I have the same error here, running

Gitlab : 8.15.4
Postgre : 9.6-1

Up from

Gitlab : 8.13.6
Postgre : 9.5-3

I can't restore my data from previous backup and the upgrade flushed all the data (repositories, groups, users).

What should I do ?

Niceplace commented Jan 16, 2017

I have the same error here, running

Gitlab : 8.15.4
Postgre : 9.6-1

Up from

Gitlab : 8.13.6
Postgre : 9.5-3

I can't restore my data from previous backup and the upgrade flushed all the data (repositories, groups, users).

What should I do ?

@Niceplace

This comment has been minimized.

Show comment
Hide comment
@Niceplace

Niceplace Jan 16, 2017

@sameersbn It seems that the restore system is broken for 8.15.6, can you confirm this ? Its a major blocker for me right now.

@sameersbn It seems that the restore system is broken for 8.15.6, can you confirm this ? Its a major blocker for me right now.

@solidnerd

This comment has been minimized.

Show comment
Hide comment
@solidnerd

solidnerd Jan 16, 2017

Collaborator

Hey,
what you can do is to format your file correctly and try to restore it again.

So copy the EPOCH from your file and check the date with date -d @1482721258 from your backup 1482721258_gitlab_backup.tar .

Rename your file like @adamniedzielski said so in this case it would be 1482721258_2016_12_26_gitlab_backup.tar.

Now you can exec the command as follow:

docker-compose run --rm gitlab app:rake gitlab:backup:restore BACKUP=1482721258_2016_12_26
Collaborator

solidnerd commented Jan 16, 2017

Hey,
what you can do is to format your file correctly and try to restore it again.

So copy the EPOCH from your file and check the date with date -d @1482721258 from your backup 1482721258_gitlab_backup.tar .

Rename your file like @adamniedzielski said so in this case it would be 1482721258_2016_12_26_gitlab_backup.tar.

Now you can exec the command as follow:

docker-compose run --rm gitlab app:rake gitlab:backup:restore BACKUP=1482721258_2016_12_26
@Niceplace

This comment has been minimized.

Show comment
Hide comment
@Niceplace

Niceplace Jan 16, 2017

Hi Niclas, thanks for the quick answer ! I'll give it a try !

Hi Niclas, thanks for the quick answer ! I'll give it a try !

@waterdrops

This comment has been minimized.

Show comment
Hide comment
@waterdrops

waterdrops Jan 19, 2017

Restoring from backup,I encounter a similar error,running


System information
System:		Ubuntu 14.04
Current User:	git
Using RVM:	no
Ruby Version:	2.3.3p222
Gem Version:	2.5.2
Bundler Version:1.13.7
Rake Version:	10.5.0
Sidekiq Version:4.2.7

GitLab information
Version:	8.15.4
Revision:	a0b1379
Directory:	/home/git/gitlab
DB Adapter:	postgresql
Using LDAP:	yes
Using Omniauth:	no

GitLab Shell
Version:	4.1.1
Git:		/usr/bin/git

the trace info below:


bundle exec rake gitlab:backup:restore BACKUP=1484769615_2017_01_18 RAILS_ENV=production --trace

** Invoke gitlab:backup:restore (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute gitlab:backup:restore
rake aborted!
TypeError: no implicit conversion of nil into String
/home/git/gitlab/lib/backup/manager.rb:122:in `exist?'
/home/git/gitlab/lib/backup/manager.rb:122:in `unpack'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:32:in `block (3 levels) in <top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'
/usr/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:150:in `invoke_task'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block (2 levels) in top_level'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block in top_level'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:115:in `run_with_threads'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:100:in `top_level'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:78:in `block in run'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:176:in `standard_exception_handling'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:75:in `run'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/bin/rake:33:in `<top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/bin/rake:22:in `load'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/bin/rake:22:in `<main>'
Tasks: TOP => gitlab:backup:restore

waterdrops commented Jan 19, 2017

Restoring from backup,I encounter a similar error,running


System information
System:		Ubuntu 14.04
Current User:	git
Using RVM:	no
Ruby Version:	2.3.3p222
Gem Version:	2.5.2
Bundler Version:1.13.7
Rake Version:	10.5.0
Sidekiq Version:4.2.7

GitLab information
Version:	8.15.4
Revision:	a0b1379
Directory:	/home/git/gitlab
DB Adapter:	postgresql
Using LDAP:	yes
Using Omniauth:	no

GitLab Shell
Version:	4.1.1
Git:		/usr/bin/git

the trace info below:


bundle exec rake gitlab:backup:restore BACKUP=1484769615_2017_01_18 RAILS_ENV=production --trace

** Invoke gitlab:backup:restore (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute gitlab:backup:restore
rake aborted!
TypeError: no implicit conversion of nil into String
/home/git/gitlab/lib/backup/manager.rb:122:in `exist?'
/home/git/gitlab/lib/backup/manager.rb:122:in `unpack'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:32:in `block (3 levels) in <top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'
/usr/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:150:in `invoke_task'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block (2 levels) in top_level'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block in top_level'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:115:in `run_with_threads'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:100:in `top_level'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:78:in `block in run'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:176:in `standard_exception_handling'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:75:in `run'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-10.5.0/bin/rake:33:in `<top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/bin/rake:22:in `load'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/bin/rake:22:in `<main>'
Tasks: TOP => gitlab:backup:restore
@xueshanf

This comment has been minimized.

Show comment
Hide comment
@xueshanf

xueshanf Feb 2, 2017

Contributor

@waterdrops

This works for me (8.15.4) - It looks like the BACKUP needs a full file name, not just timestamp:

docker exec gitlab.service /bin/bash -c "cd /home/git/gitlab; sudo -u git -H bundle exec rake gitlab:backup:restore force=yes RAILS_ENV=production BACKUP=1485922557_2017_02_01_gitlab_backup.tar
Contributor

xueshanf commented Feb 2, 2017

@waterdrops

This works for me (8.15.4) - It looks like the BACKUP needs a full file name, not just timestamp:

docker exec gitlab.service /bin/bash -c "cd /home/git/gitlab; sudo -u git -H bundle exec rake gitlab:backup:restore force=yes RAILS_ENV=production BACKUP=1485922557_2017_02_01_gitlab_backup.tar
@waterdrops

This comment has been minimized.

Show comment
Hide comment
@waterdrops

waterdrops Feb 7, 2017

@xueshanf thanks,it works :); BACKUP file format changed in this version(8.15.4),thanks again 👍

@xueshanf thanks,it works :); BACKUP file format changed in this version(8.15.4),thanks again 👍

@Niceplace

This comment has been minimized.

Show comment
Hide comment
@Niceplace

Niceplace Feb 7, 2017

I tested in another environment and followed the original procedure of sameersbn/docker-gitlab found here and it worked (providing I kept the same database version).

The thing I wasn't doing correctly was creating the backup with the rake task with the NEW gitlab image. I was trying to restore a backup created with the old image into the new one, which failed.

As Gitlab docs specify : "You can only restore a backup to exactly the same version of GitLab on which it was created. The best way to migrate your repositories from one server to another is through backup restore" source

Niceplace commented Feb 7, 2017

I tested in another environment and followed the original procedure of sameersbn/docker-gitlab found here and it worked (providing I kept the same database version).

The thing I wasn't doing correctly was creating the backup with the rake task with the NEW gitlab image. I was trying to restore a backup created with the old image into the new one, which failed.

As Gitlab docs specify : "You can only restore a backup to exactly the same version of GitLab on which it was created. The best way to migrate your repositories from one server to another is through backup restore" source

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment