Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

adds git repo & wiki backups too

  • Loading branch information...
commit 53fa56a8d22175842fbe56b1fcb2074057102bdc 1 parent 82c39c1
@mcortesi mcortesi authored
Showing with 152 additions and 44 deletions.
  1. +81 −0 .rvmrc
  2. +2 −2 Gemfile
  3. +8 −0 Gemfile.lock
  4. +10 −6 README.md
  5. +51 −36 lib/backup-github.rb
View
81 .rvmrc
@@ -0,0 +1,81 @@
+#!/usr/bin/env bash
+
+# This is an RVM Project .rvmrc file, used to automatically load the ruby
+# development environment upon cd'ing into the directory
+
+# First we specify our desired <ruby>[@<gemset>], the @gemset name is optional.
+environment_id="ruby-1.9.3-p0@backup-github"
+
+#
+# Uncomment the following lines if you want to verify rvm version per project
+#
+# rvmrc_rvm_version="1.10.2" # 1.10.1 seams as a safe start
+# eval "$(echo ${rvm_version}.${rvmrc_rvm_version} | awk -F. '{print "[[ "$1*65536+$2*256+$3" -ge "$4*65536+$5*256+$6" ]]"}' )" || {
+# echo "This .rvmrc file requires at least RVM ${rvmrc_rvm_version}, aborting loading."
+# return 1
+# }
+#
+
+#
+# Uncomment following line if you want options to be set only for given project.
+#
+# PROJECT_JRUBY_OPTS=( --1.9 )
+#
+# The variable PROJECT_JRUBY_OPTS requires the following to be run in shell:
+#
+# chmod +x ${rvm_path}/hooks/after_use_jruby_opts
+#
+
+#
+# First we attempt to load the desired environment directly from the environment
+# file. This is very fast and efficient compared to running through the entire
+# CLI and selector. If you want feedback on which environment was used then
+# insert the word 'use' after --create as this triggers verbose mode.
+#
+if [[ -d "${rvm_path:-$HOME/.rvm}/environments" \
+ && -s "${rvm_path:-$HOME/.rvm}/environments/$environment_id" ]]
+then
+ \. "${rvm_path:-$HOME/.rvm}/environments/$environment_id"
+
+ if [[ -s "${rvm_path:-$HOME/.rvm}/hooks/after_use" ]]
+ then
+ . "${rvm_path:-$HOME/.rvm}/hooks/after_use"
+ fi
+else
+ # If the environment file has not yet been created, use the RVM CLI to select.
+ if ! rvm --create "$environment_id"
+ then
+ echo "Failed to create RVM environment '${environment_id}'."
+ return 1
+ fi
+fi
+
+#
+# If you use an RVM gemset file to install a list of gems (*.gems), you can have
+# it be automatically loaded. Uncomment the following and adjust the filename if
+# necessary.
+#
+# filename=".gems"
+# if [[ -s "$filename" ]]
+# then
+# rvm gemset import "$filename" | grep -v already | grep -v listed | grep -v complete | sed '/^$/d'
+# fi
+
+# If you use bundler, this might be useful to you:
+# if [[ -s Gemfile ]] && ! command -v bundle >/dev/null
+# then
+# printf "%b" "The rubygem 'bundler' is not installed. Installing it now.\n"
+# gem install bundler
+# fi
+# if [[ -s Gemfile ]] && command -v bundle
+# then
+# bundle install
+# fi
+
+if [[ $- == *i* ]] # check for interactive shells
+then
+ echo "Using: $(tput setaf 2)$GEM_HOME$(tput sgr0)" # show the user the ruby and gemset they are using in green
+else
+ echo "Using: $GEM_HOME" # don't use colors in interactive shells
+fi
+
View
4 Gemfile
@@ -5,8 +5,8 @@ group :development do
gem "rdoc", "~> 3.12"
gem "bundler", "~> 1.0.0"
gem "jeweler", "~> 1.8.3"
- gem "mocha", "~> 0.10.3"
- # gem "pry" # active for debug
+ gem "mocha", "~> 0.10.3"
+ #gem "pry" # active for debug
end
group :default do
View
8 Gemfile.lock
@@ -11,6 +11,7 @@ GEM
remote: http://rubygems.org/
specs:
addressable (2.2.6)
+ coderay (1.0.5)
diff-lcs (1.1.3)
faraday (0.7.6)
addressable (~> 2.2)
@@ -30,6 +31,7 @@ GEM
rdoc
json (1.6.5)
metaclass (0.0.1)
+ method_source (0.7.0)
mime-types (1.17.2)
mocha (0.10.3)
metaclass (~> 0.0.1)
@@ -41,6 +43,10 @@ GEM
faraday_middleware (~> 0.7.0)
hashie (~> 1.2.0)
multi_json (~> 1.0.2)
+ pry (0.9.8.2)
+ coderay (~> 1.0.5)
+ method_source (~> 0.7)
+ slop (>= 2.4.4, < 3)
rack (1.4.1)
rake (0.9.2.2)
rdoc (3.12)
@@ -53,6 +59,7 @@ GEM
rspec-expectations (2.8.0)
diff-lcs (~> 1.1.2)
rspec-mocks (2.8.0)
+ slop (2.4.4)
trollop (1.16.2)
PLATFORMS
@@ -66,6 +73,7 @@ DEPENDENCIES
json (~> 1.6.4)
mocha (~> 0.10.3)
octokit (~> 0.6.5)
+ pry
rdoc (~> 3.12)
rspec (~> 2.8.0)
trollop (~> 1.16.2)
View
16 README.md
@@ -1,6 +1,6 @@
# backup-github
-Utility tool to backup github repository issues. It will backup up all the issues from all the repositories from a given github account.
+Utility tool to backup github repository issues. It will backup all repositories from a given github account. The backup comprises the git repostory, the wiki and the issues.
It works with a local git repository, on which issues from all repositories of a github account are serialized to json and commited. Thanks to git, all changes are versioned, and saved.
@@ -13,7 +13,7 @@ To install you can just download the gem with:
Then, to run the backup command:
```bash
- backup-github -r path/to/my/backup/dir -u github_user -p github_pwd
+ backup-github -r path/to/my/backup/dir -u github_user -p github_pwd
```
If, instead you want to backup up all the issues from an Organization within github, then you run:
@@ -26,19 +26,23 @@ If, instead you want to backup up all the issues from an Organization within git
To properly configure the backup system, we recommend setting a cron job to run the command once a day.
+## Dependencies
+
+Besides the gem dependencies, the git binary is required.
+
## Contributing to backup-github
-
+
* Check out the latest master to make sure the feature hasn't been implemented
or the bug hasn't been fixed yet.
-* Check out the issue tracker to make sure someone already hasn't requested it
+* Check out the issue tracker to make sure someone already hasn't requested it
and/or contributed it.
* Fork the project.
* Start a feature/bugfix branch.
* Commit and push until you are happy with your contribution.
* Make sure to add tests for it. This is important so I don't break it in a
future version unintentionally.
-* Please try not to mess with the Rakefile, version, or history. If you want to
- have your own version, or is otherwise necessary, that is fine, but please
+* Please try not to mess with the Rakefile, version, or history. If you want to
+ have your own version, or is otherwise necessary, that is fine, but please
isolate to its own commit so I can cherry-pick around it.
## Copyright
View
87 lib/backup-github.rb
@@ -7,8 +7,11 @@
require 'pathname'
require 'logger'
+#Grit.debug=true
+
+Issue = Struct.new "Issue", :number, :content
$logger = Logger.new($stdout)
-
+
class GithubBackup
def initialize(github)
@@ -19,58 +22,71 @@ def serialize_issue(issue)
JSON.pretty_generate( issue.content )
end
- def save_issue(dir, issue)
- file = dir + ('%04d.js' % issue.number)
- file.open('w') { |f| f.write(serialize_issue(issue)) }
- @gitrepo.add(file.to_s)
+ def save_issue(issue)
+ File.open('%04d.js' % issue.number, "w") { |f| f.write(serialize_issue(issue)) }
end
- def run( orgname, local_repo )
- Dir.chdir(local_repo) do |path|
- @gitrepo = begin
- Grit::Repo.new(".")
- rescue Grit::InvalidGitRepositoryError => e
- $logger.warn "Not a git Repository, creating a new one"
- Grit::Repo.init(".")
+ def backup_gitrepo( giturl, path )
+ $logger.info "Backuping #{path.to_path}"
+ if path.directory?
+ $logger.info "Backuping #{path.to_path} -> UPDATE"
+ gritty = Grit::Git.new(path.realpath.to_path)
+ gritty.fetch({:timeout=>false})
+ else
+ $logger.info "Backuping #{path.to_path} -> NEW"
+ path.mkpath
+ gritty = Grit::Git.new(".")
+ gritty.clone({:mirror=>true, :timeout=>false}, giturl, path.realpath.to_path)
end
+ end
+
+ def run( orgname, local_repo )
+ @github.repositories(orgname)[0,3].each do |repo|
+ reponame = "#{orgname}/#{repo.name}"
+ $logger.info "Backuping #{reponame}"
+
+ repopath = Pathname.new(local_repo) + repo.name
+ repopath.mkpath
- orgdir = Pathname.new orgname
-
- repositories = @github.repositories orgname
-
- repositories.each do |repo|
- reponame = "#{orgname}/#{repo.name}"
- $logger.info "Backuping #{reponame}"
-
- if @github.has_issues? reponame
- projectdir = orgdir + repo.name
- projectdir.mkpath
-
- @github.on_issues(reponame) {|issue| save_issue(projectdir, issue)}
- else
- $logger.info "No issues for #{reponame}"
+ $logger.info("Backup Reporsitory")
+ backup_gitrepo(repo.ssh_url, repopath + "git-bare" )
+
+ $logger.info("Backup Wiki")
+ backup_gitrepo(repo.ssh_url.gsub(".git", ".wiki.git"), repopath + "wiki-bare" )
+
+ # Backup Issues
+ if @github.has_issues? reponame
+ issuespath = repopath + "issues"
+ issuespath.mkpath
+ Dir.chdir(issuespath) do
+ # After endless problems with Grit, fallback to sytem exec
+ `git init` unless File.directory? ".git"
+ $logger.info("Fetching issues")
+ @github.on_issues(reponame) {|issue| save_issue(issue)}
+
+ $logger.info("Add & Commit issues")
+ `git add .`
+ `git commit -m backup`
end
+ else
+ $logger.info "No issues for #{reponame}"
end
-
- @gitrepo.commit_index("Backup...")
- $logger.info 'Backup Finished'
end
end
-
end
class GithubAPIAdapter
@@ISSUE_STATES = %w{open closed}
-
- def initialize(client)
+
+ def initialize(client)
@client = client
end
def has_issues?(reponame)
@client.repository(reponame).has_issues
end
-
+
def on_issues(reponame, &block)
@@ISSUE_STATES.each do |status|
page = 0
@@ -78,8 +94,7 @@ def on_issues(reponame, &block)
begin
issues = @client.list_issues( reponame, :page => page, :state => status )
issues.each do |issue|
- yield OpenStruct.new(:number => issue.number,
- :content => @client.issue(reponame, issue.number))
+ yield Issue.new(issue.number, @client.issue(reponame, issue.number))
end
page += 1
end until issues.length != 10
Please sign in to comment.
Something went wrong with that request. Please try again.