Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse code

Initial Github pages import

  • Loading branch information...
commit 93918ddf96295b681efee66ba7793fb9a46a0ccb 1 parent 27d0461
Hongli Lai FooBarWidget authored
59 README.md
Source Rendered
... ... @@ -1,59 +0,0 @@
1   -# Phusion Server Tools
2   -
3   -A collection of server administration tools that we use. Everything is
4   -written in Ruby and are designed to work with Debian. These scripts may
5   -work with other operating systems or distributions as well, but it's not
6   -tested.
7   -
8   -Install with:
9   -
10   - git clone ... /tools
11   -
12   -It's not necessary to install to /tools, you can install to anywhere, but this document assumes that you have installed to /tools.
13   -
14   -Each tool has its own prerequities, but here are some common prerequities:
15   -
16   - * Ruby (obviously)
17   - * `pv` - `apt-get install pv`. Not required but very useful; allows display of progress bars.
18   -
19   -
20   -## Included tools
21   -
22   -### backup-mysql - Rotating MySQL dumps
23   -
24   -A script which backs up all MySQL databases to `/var/backups/mysql`. At most 10 backups are kept. All backups are compressed with gzip.
25   -
26   -It uses `mysql` to obtain a list of databases and `mysqldump` to dump the database contents. If you want to run this script unattended you should therefore set the right login information in `~/.my.cnf`, sections `[mysql]` and `[mysqldump]`.
27   -
28   -Make it run daily at 0:00 AM in cron:
29   -
30   - 0 0 * * * /tools/silence-unless-failed /tools/backup-mysql
31   -
32   -### permit and deny - Easily set fine-grained permissions using ACLs
33   -
34   -Scripts for giving a user access to a single file, or recursive access to a directory, using ACLs. The standard `setfacl` tool is too hard to use and sometimes does stupid things such as unexpectedly making files executable. These scripts are simple and work as expected.
35   -
36   - # Recursively give web server read-only access to /webapps/foo.
37   - /tools/permit www-data /webapps/foo
38   -
39   - # Recursively give user 'deploy' read-write access to /webapps/bar.
40   - /tools/permit deploy /webapps/bar --read-write
41   -
42   - # Recursively remove all ACLs for user 'joe' on /secrets/area66.
43   - /tools/deny joe /secrets/area66
44   -
45   -You need the `getfacl` and `setfacl` commands:
46   -
47   - apt-get install acl
48   -
49   -You must also make sure your filesystem is mounted with ACL support, e.g.:
50   -
51   - mount -o remount,acl /
52   -
53   -Don't forget to update /etc/fstab too.
54   -
55   -### silcence-unless-failed
56   -
57   -Runs the given command but only print its output (both STDOUT and STDERR) if its exit code is non-zero. The script's own exit code is the same as the command's exit code.
58   -
59   - /tools/silence-unless-failed my-command arg1 arg2 --arg3
26 backup-mysql
... ... @@ -1,26 +0,0 @@
1   -#!/usr/bin/env ruby
2   -require File.expand_path(File.dirname(__FILE__) + '/shared')
3   -BACKUP_DIR_ROOT = "/var/backups/mysql"
4   -MAX_BACKUPS = 10
5   -
6   -databases = `echo show databases | mysql`.strip.split("\n")
7   -databases.shift
8   -databases.delete("information_schema")
9   -databases.delete("mysql")
10   -
11   -now = Time.now.strftime("%Y-%m-%d-%H:%M:%S")
12   -backup_dir = "#{BACKUP_DIR_ROOT}/#{now}"
13   -
14   -sh "mkdir -p #{backup_dir}"
15   -for database in databases
16   - sh "mysqldump #{database} | gzip --best | #{pv_or_cat} > #{backup_dir}/#{database}.sql.gz"
17   -end
18   -
19   -puts "Cleaning up..."
20   -dirs = Dir["#{BACKUP_DIR_ROOT}/*"].sort.reverse
21   -keep = dirs[0..MAX_BACKUPS]
22   -delete = dirs - keep
23   -delete.each do |dir|
24   - sh "rm -rf #{dir}"
25   -end
26   -sh "chmod -R o-rwx #{BACKUP_DIR_ROOT}"
27 deny
... ... @@ -1,27 +0,0 @@
1   -#!/usr/bin/env ruby
2   -def help
3   - puts "Usage: permit <USERNAME> <DIR>"
4   - puts "Removes access for USERNAME to directory DIR."
5   - exit 1
6   -end
7   -
8   -def sh(command, *args)
9   - puts "#{command} #{args.join(' ')}"
10   - if !system(command, *args)
11   - STDERR.puts "*** ERROR"
12   - exit 1
13   - end
14   -end
15   -
16   -help if ARGV.size != 2
17   -username, dir = ARGV
18   -puts "cd #{dir}"
19   -Dir.chdir(dir) do
20   - executable_files = "/tmp/executable-files.#{$$}"
21   - sh "find -type f -executable -print0 > #{executable_files}"
22   - sh "find -print0 | xargs -0 -n 1000 -r setfacl -x user:#{username}"
23   - sh "find -type d -print0 | xargs -0 -n 1000 -r setfacl -d -x user:#{username}"
24   - sh "find -type f -print0 | xargs -0 -n 1000 -r chmod -x"
25   - sh "cat #{executable_files} | xargs -0 -n 1000 -r chmod +x"
26   - sh "rm -f #{executable_files}"
27   -end
14 display-queue
... ... @@ -1,14 +0,0 @@
1   -#!/usr/bin/env ruby
2   -puts "Name Messages = Ready + Unack Consumers Memory (MB)"
3   -puts "-------------------------------------------------------------------------------"
4   -lines = `rabbitmqctl -q list_queues name messages messages_ready messages_unacknowledged consumers memory`
5   -lines = lines.split("\n")
6   -lines.each do |line|
7   - name, messages, ready, unack, consumers, memory = line.split(/[ \t]+/)
8   - messages = messages.to_i
9   - ready = ready.to_i
10   - unack = unack.to_i
11   - consumers = consumers.to_i
12   - memory = memory.to_i / 1024.0 / 1024
13   - printf("%-23s %-8d = %-5d + %-5d %-9d %1.f\n", name, messages, ready, unack, consumers, memory)
14   -end
38 fix-capistrano-permissions
... ... @@ -1,38 +0,0 @@
1   -#!/bin/bash
2   -CAPISTRANO_DIR=/u/apps
3   -WWW_USER=www-data
4   -
5   -export PATH="/tools:$PATH"
6   -
7   -chmod -R g+w,o-rwx $CAPISTRANO_DIR
8   -setfacl -m user:$WWW_USER:--x $CAPISTRANO_DIR
9   -setfacl -d -m user:$WWW_USER:r-x $CAPISTRANO_DIR
10   -for DIR in $CAPISTRANO_DIR/*; do
11   - # Give the web server read-only access to everything.
12   - # We tighten up permissions in later commands.
13   - permit $WWW_USER $DIR
14   -
15   - # Make the application directory itself executable-only
16   - # by the web server.
17   - if [[ -d $DIR/releases || -d $DIR/shared ]]; then
18   - setfacl -m user:$WWW_USER:--x $DIR
19   - setfacl -d -m user:$WWW_USER:--x $DIR
20   - fi
21   -
22   - # Make the 'releases' directory and immediate
23   - # subdirectories executable-only by the web server.
24   - if [[ -d $DIR/releases ]]; then
25   - setfacl -m user:$WWW_USER:--x $DIR/releases
26   - setfacl -m user:$WWW_USER:--x $DIR/releases/*
27   - fi
28   -
29   - # Deny web server access to the 'shared' directory.
30   - if [[ -d $DIR/shared ]]; then
31   - deny $WWW_USER $DIR/shared
32   - # If you store attachment files in the 'shared'
33   - # directory then you can allow only access to that:
34   - #
35   - # setfacl -m user:$WWW_USER:--x $DIR/shared
36   - # permit $DIR/shared/attachments
37   - fi
38   -done
20 gc-git-repos
... ... @@ -1,20 +0,0 @@
1   -#!/usr/bin/env ruby
2   -# This script is automatically called in a cron job by root.
3   -require 'etc'
4   -
5   -DIRS = Dir[
6   - "/u/apps/**/.git"
7   -]
8   -DIRS.each do |dir|
9   - puts "# Garbage collecting #{dir}"
10   - Dir.chdir(dir) do
11   - stat = File.stat(dir)
12   - username = Etc.getpwuid(stat.uid).name
13   - command = "su -c 'git gc' #{username}"
14   - puts command
15   - if !system(command)
16   - abort "'git gc' failed"
17   - end
18   - end
19   - puts
20   -end
14 index.html
... ... @@ -0,0 +1,14 @@
  1 +<!DOCTYPE html>
  2 +<html>
  3 + <head>
  4 + <meta charset="utf-8">
  5 + <meta name="apple-mobile-web-app-capable" content="yes">
  6 + <meta name="viewport" content="width=device-width, initial-scale=1.0">
  7 + <title></title>
  8 + <script src="http://cdnjs.cloudflare.com/ajax/libs/documentup/latest.min.js"></script>
  9 + <script>
  10 + DocumentUp.document("phusion/phusion-server-tools");
  11 + </script>
  12 + </head>
  13 + <body></body>
  14 +</html>
21 notify-if-queue-becomes-large
... ... @@ -1,21 +0,0 @@
1   -#!/usr/bin/env ruby
2   -THRESHOLD=1000
3   -
4   -warnings = []
5   -`rabbitmqctl list_queues -q name messages`.split("\n").each do |line|
6   - name, messages = line.split(/[ \t]+/)
7   - messages = messages.to_i
8   - if messages > THRESHOLD
9   - warnings << "Queue '#{name}' has more than #{THRESHOLD} messages: #{messages}"
10   - end
11   -end
12   -
13   -if !warnings.empty?
14   - IO.popen("sendmail -t", "w") do |f|
15   - f.puts "To: info@phusion.nl"
16   - f.puts "From: noreply@unionstationapp.com"
17   - f.puts "Subject: [Union Station] Queue going out of control"
18   - f.puts
19   - f.puts warnings.join("\n")
20   - end
21   -end
40 permit
... ... @@ -1,40 +0,0 @@
1   -#!/usr/bin/env ruby
2   -def help
3   - puts "Usage: permit <USERNAME> <DIR> [--read-write]"
4   - puts "Give USERNAME read-only or read-write permission to directory DIR."
5   - puts "Default is read-only unless --read-write is given."
6   - exit 1
7   -end
8   -
9   -def sh(command, *args)
10   - puts "#{command} #{args.join(' ')}"
11   - if !system(command, *args)
12   - STDERR.puts "*** ERROR"
13   - exit 1
14   - end
15   -end
16   -
17   -help if ARGV.size != 2 && ARGV.size != 3
18   -username, dir, read_write = ARGV
19   -if read_write && read_write != "--read-write"
20   - STDERR.puts "*** Invalid option #{read_write}"
21   - help
22   -end
23   -
24   -puts "cd #{dir}"
25   -Dir.chdir(dir) do
26   - executable_files = "/tmp/executable-files.#{$$}"
27   - sh "find -type f -executable -print0 > #{executable_files}"
28   - if read_write
29   - sh "find -type f -print0 | xargs -0 -n 1000 -r setfacl -m user:#{username}:rw-"
30   - sh "find -type d -print0 | xargs -0 -n 1000 -r setfacl -m user:#{username}:rwx"
31   - sh "find -type d -print0 | xargs -0 -n 1000 -r setfacl -d -m user:#{username}:rwx"
32   - else
33   - sh "find -type f -print0 | xargs -0 -n 1000 -r setfacl -m user:#{username}:r-"
34   - sh "find -type d -print0 | xargs -0 -n 1000 -r setfacl -m user:#{username}:r-x"
35   - sh "find -type d -print0 | xargs -0 -n 1000 -r setfacl -d -m user:#{username}:r-x"
36   - end
37   - sh "find -type f -print0 | xargs -0 -n 1000 -r chmod -x"
38   - sh "cat #{executable_files} | xargs -0 -n 1000 -r chmod +x"
39   - sh "rm -f #{executable_files}"
40   -end
11 purge-queue
... ... @@ -1,11 +0,0 @@
1   -#!/usr/bin/env ruby
2   -if !ARGV[0]
3   - STDERR.puts "You must specify a queue name."
4   - exit 1
5   -end
6   -require 'rubygems'
7   -require 'bunny'
8   -b = Bunny.new
9   -b.start
10   -q = b.queue(ARGV[0], :durable => true)
11   -q.purge
34 shared.rb
... ... @@ -1,34 +0,0 @@
1   -def sh(command, *args)
2   - puts "# #{command} #{args.join(' ')}"
3   - if !system(command, *args)
4   - STDERR.puts "*** ERROR"
5   - exit 1
6   - end
7   -end
8   -
9   -# Check whether the specified command is in $PATH, and return its
10   -# absolute filename. Returns nil if the command is not found.
11   -#
12   -# This function exists because system('which') doesn't always behave
13   -# correctly, for some weird reason.
14   -def find_command(name)
15   - name = name.to_s
16   - ENV['PATH'].to_s.split(File::PATH_SEPARATOR).detect do |directory|
17   - path = File.join(directory, name)
18   - if File.file?(path) && File.executable?(path)
19   - return path
20   - end
21   - end
22   - return nil
23   -end
24   -
25   -# Returns "pv" if that command is installed, or "cat" if not.
26   -# "pv" is the Pipe Viewer tool, very useful for displaying
27   -# progress bars in pipe operations (apt-get install pv).
28   -def pv_or_cat
29   - if find_command('pv')
30   - return 'pv'
31   - else
32   - return 'cat'
33   - end
34   -end
12 silence-unless-failed
... ... @@ -1,12 +0,0 @@
1   -#!/bin/bash
2   -# Executes the given command, silencing all its output.
3   -# Its output is only printed to STDERR if the command failed.
4   -
5   -output="/tmp/output.$$"
6   -"$@" >$output 2>&1
7   -exit_code=$?
8   -if [[ $exit_code != 0 ]]; then
9   - cat $output >&2
10   -fi
11   -rm -f $output
12   -exit $exit_code
4 truncate
... ... @@ -1,4 +0,0 @@
1   -#!/bin/bash
2   -for F in "$@"; do
3   - echo -n > "$F"
4   -done
4 watch-queue
... ... @@ -1,4 +0,0 @@
1   -#!/bin/bash
2   -dir=`dirname "$0"`
3   -cd "$dir"
4   -exec watch -d ./display-queue

0 comments on commit 93918dd

Please sign in to comment.
Something went wrong with that request. Please try again.