Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark backups #238

Merged
merged 3 commits into from
Jul 26, 2016
Merged

Benchmark backups #238

merged 3 commits into from
Jul 26, 2016

Conversation

snh
Copy link
Member

@snh snh commented Jul 8, 2016

Same as #219, but for backups and includes per-node stats for cluster restores.

Standard backup looks something like this:

ghe-backup-store-version took 0s
ghe-backup-settings took 1s
ghe-export-authorized-keys took 0s
ghe-export-ssh-host-keys took 0s
ghe-export-mysql took 1s
ghe-backup-redis took 1s
ghe-backup-es-audit-log took 2s
ghe-backup-es-hookshot took 0s
ghe-backup-repositories-rsync took 1s
ghe-backup-userdata - pages took 1s
ghe-backup-pages-rsync took 1s
ghe-backup-userdata - alambic_assets took 0s
ghe-backup-userdata - storage took 0s
ghe-backup-userdata - hookshot took 0s
ghe-backup-userdata - git-hooks/repos took 0s
ghe-backup-es-rsync took 1s

Cluster backup looks something like this:

ghe-backup-store-version took 0s
ghe-backup-settings took 3s
ghe-export-authorized-keys took 0s
ghe-export-ssh-host-keys took 0s
ghe-export-mysql took 7s
ghe-backup-redis-cluster took 2s
ghe-backup-es-audit-log took 1s
ghe-backup-es-hookshot took 1s
ghe-backup-repositories-cluster - data-0 took 5s
ghe-backup-repositories-cluster - data-1 took 5s
ghe-backup-repositories-cluster - data-2 took 4s
ghe-backup-repositories-cluster took 17s
ghe-backup-pages-cluster - data-0 took 1s
ghe-backup-pages-cluster - data-1 took 1s
ghe-backup-pages-cluster - data-2 took 1s
ghe-backup-pages-cluster took 4s
ghe-backup-alambic-cluster - data-0 took 1s
ghe-backup-alambic-cluster - data-1 took 1s
ghe-backup-alambic-cluster - data-2 took 1s
ghe-backup-alambic-cluster took 3s
ghe-backup-git-hooks-cluster took 3s

The logs are timestamped and stored in the benchmarks directory, along with the data that was backed up.

/cc @github/backup-utils @rubiojr

@snh snh added the enhancement label Jul 8, 2016
@rubiojr
Copy link
Member

rubiojr commented Jul 25, 2016

❤️ thanks @snh.

@rubiojr rubiojr merged commit ef4c5ef into master Jul 26, 2016
@rubiojr rubiojr deleted the snh/benchmark-backups branch July 26, 2016 10:45
rubiojr added a commit that referenced this pull request Jul 27, 2016
Patch release fixing a regression and instrumenting backups.

Changes the cluster restore behaviour a bit since it doesn't restore the
cluster configuration (cluster.conf) any more.
Backup restores to the same cluster and are not affected.
Backup restores from a single VM to a cluster (i.e. migrating from a single
GitHub Enterprise VM to a cluster using backup-utils) are also unaffected.

* Instrument/benchmark backups #238
* Cluster: remove restoring cluster.conf on restore #242
rubiojr added a commit that referenced this pull request Jul 27, 2016
Patch release fixing a regression and instrumenting backups.

Changes the cluster restore behaviour a bit since it doesn't restore the
cluster configuration (cluster.conf) any more.
Backup restores to the same cluster and are not affected.
Backup restores from a single VM to a cluster (i.e. migrating from a single
GitHub Enterprise VM to a cluster using backup-utils) are also unaffected.

* Instrument/benchmark backups #238
* Cluster: remove restoring cluster.conf on restore #242
@rubiojr rubiojr mentioned this pull request Jul 27, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants