diff --git a/CHANGELOG.md b/CHANGELOG.md index bfcf9da..3429666 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,15 @@ ### master +## v5.0.0, 2018-06-05 +- Code cleanup +- Removal of legacy add hstore method +- Using execute and test properly so we can see what the gem is doing in the STDOUT +- Expanded remove_all task to actually cover everything +- Added deploy config option pg_generate_random_password, instead of using it by default when pg_password is excluded +- issues/53: Bug fixed for updates to the archetype when using random password +- projects/1: Prep for RSPEC testing project + ## v4.9.1, 2018-06-04 - Added back set :pg_ask_for_password, false and ask_for_or_generate_password diff --git a/README.md b/README.md index 972199f..2573ed7 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,9 @@ # Capistrano::PostgreSQL -**Note: this plugin works only with Capistrano 3.** Plase check the capistrano +**Note: this plugin works only with Capistrano 3.** Please check the capistrano gem version you're using before installing this gem: `$ bundle show | grep capistrano` - -Plugin for Capistrano 2 [is here](https://github.com/bruno-/capistrano2-postgresql). +The plugin for Capistrano 2 [is here](https://github.com/bruno-/capistrano2-postgresql). ### About @@ -14,11 +13,9 @@ tasks for PostgreSQL when deploying rails apps. Here are the specific things this plugin does for your capistrano deployment process: -* creates a new PostgreSQL database and database user on the server -* generates and populates `database.yml` file on all release nodes - (no need to ssh to the server and do this manually!) -* zero-config -* support for multi-server setup: separate `db` and `app` nodes (from version 4.0) +* Creates a new PostgreSQL database and database user on the server +* Generates and populates `database.yml` file on all release nodes (using ssh) +* Support for multi-server setup: separate `db` and `app` nodes ( versions > 4.0 ) **Note**: gem version 4 introduces some breaking changes. If you installed gem version 3 or below you might want to follow the @@ -30,7 +27,7 @@ Put the following in your application's `Gemfile`: group :development do gem 'capistrano', '~> 3.2.0' - gem 'capistrano-postgresql', '~> 4.8.0' + gem 'capistrano-postgresql', '~> 5.0.0' end Then: @@ -39,31 +36,55 @@ Then: ### Usage -If you're deploying a standard rails app, all you need to do is put -the following in `Capfile` file: +In a standard RAILS app, you need to do is put the following in `Capfile` file: ``` require 'capistrano/postgresql' ``` -* Make sure the `deploy_to` path exists and has the right privileges on the -server (i.e. `/var/www/myapp`). Warning: The ~ symbol (i.e. `~/myapp`) is not supported. -* Within your app/config/deploy/{env}.rb files, you need to specify at least one :app and one :db server. -* It's also suggested to specify `:primary => true` on the end of your primary :db server line. -* Optionally, you can run psql commands WITHOUT sudo if needed. Set the following (which defaults to false): `set :pg_without_sudo, true` +You need to include ONLY ONE of the following in your config/deploy/*.rb files: + +``` +set :pg_password, ENV['DATABASE_USER_PASSWORD'] +set :pg_ask_for_password, true +set :pg_generate_random_password, true +``` + +Example config: + +``` +server 'growtrader.dev', user: 'growtrader', roles: %w{app db} +set :stage, :development +set :branch, 'development' +# ================== +# Postgresql setup +set :pg_without_sudo, false +set :pg_host, 'growtrader.dev' +set :pg_database, 'growtrader' +set :pg_username, 'growtrader' +#set :pg_generate_random_password, true +#set :pg_ask_for_password, true +set :pg_password, ENV['GROWTRADER_PGPASS'] +set :pg_extensions, ['citext','hstore'] +set :pg_encoding, 'UTF-8' +set :pg_pool, '100' +``` Finally, to setup the server(s), run: $ bundle exec cap production setup -### Gotchas +### Requirements -Be sure to remove `config/database.yml` from your application's version control. +* Be sure to remove `config/database.yml` from your application's version control. +* Your pg_hba.conf must include `local all all trust` +* Make sure the `deploy_to` path exists and has the right privileges on your servers. The ~ symbol (i.e. `~/myapp`) is not supported. +* Within your app/config/deploy/{env}.rb files, you need to specify at least one :app and one :db server. +* If you have multiple :db role hosts, it's necessary to specify `:primary => true` on the end of your primary :db server. ### How it works [How the plugin works](https://github.com/capistrano-plugins/capistrano-postgresql/wiki/How-it-works) -wiki page contains a list of actions the plugin executes. Read it only if you want to learn more about the plugin internals. @@ -87,7 +108,7 @@ Check out [capistrano-plugins](https://github.com/capistrano-plugins) github org Contributions and improvements are very welcome. -If something is not working for you, or you find a bug please report it. +If something is not working for you, or you find a bug, please report it. ### Thanks diff --git a/lib/capistrano/postgresql/helper_methods.rb b/lib/capistrano/postgresql/helper_methods.rb index a2b07bd..4f08d84 100644 --- a/lib/capistrano/postgresql/helper_methods.rb +++ b/lib/capistrano/postgresql/helper_methods.rb @@ -4,6 +4,22 @@ module Capistrano module Postgresql module HelperMethods + def extension_exists?(extension) + psql 'test', fetch(:pg_system_db), '-tAc', %Q{"SELECT 1 FROM pg_extension WHERE extname='#{extension}';" | grep -q 1} + end + + def remove_extensions + if Array( fetch(:pg_extensions) ).any? + on roles :db do + # remove in reverse order if extension is present + Array( fetch(:pg_extensions) ).reverse.each do |ext| + next if [nil, false, ""].include?(ext) + psql 'execute', fetch(:pg_system_db), '-c', %Q{"DROP EXTENSION IF EXISTS #{ext};"} if extension_exists?(ext) + end + end + end + end + def generate_database_yml_io(password=fetch(:pg_password)) StringIO.open do |s| s.puts "#{fetch(:pg_env)}:" @@ -26,10 +42,10 @@ def generate_database_yml_io(password=fetch(:pg_password)) def pg_template(update=false,archetype_file=nil) config_file = "#{fetch(:pg_templates_path)}/postgresql.yml.erb" if update - raise('Updates need the original file to update from.') if archetype_file.nil? + raise('Regeneration of archetype database.yml need the original file to update from.') if archetype_file.nil? raise('Cannot update a custom postgresql.yml.erb file.') if File.exists?(config_file) # Skip custom postgresql.yml.erb if we're updating. It's not supported # Update yml file from settings - if fetch(:pg_password).nil? && fetch(:pg_ask_for_password) == false # User isn't generating a random password or wanting to set it manually from prompt + if fetch(:pg_generate_random_password) || !fetch(:pg_password) # We need to prevent updating the archetype file if we've done a random or "ask"ed password current_password = archetype_file.split("\n").grep(/password/)[0].split('password:')[1].strip generate_database_yml_io(current_password) else diff --git a/lib/capistrano/postgresql/password_helpers.rb b/lib/capistrano/postgresql/password_helpers.rb index b90f92c..758759d 100644 --- a/lib/capistrano/postgresql/password_helpers.rb +++ b/lib/capistrano/postgresql/password_helpers.rb @@ -8,15 +8,15 @@ def generate_random_password SecureRandom.hex(10) end - # This method is invoked only if :pg_password is not already set in config/#{:stage}/deploy.rb. Directly setting :pg_password has precedence. - def ask_for_or_generate_password + def pg_password_generate if fetch(:pg_ask_for_password) ask :pg_password, "Postgresql database password for the app: " - else + elsif fetch(:pg_generate_random_password) set :pg_password, generate_random_password + else + set :pg_password, nil # Necessary for pg_template end end - end end end diff --git a/lib/capistrano/postgresql/psql_helpers.rb b/lib/capistrano/postgresql/psql_helpers.rb index 473e004..070b638 100644 --- a/lib/capistrano/postgresql/psql_helpers.rb +++ b/lib/capistrano/postgresql/psql_helpers.rb @@ -2,39 +2,28 @@ module Capistrano module Postgresql module PsqlHelpers - def psql(*args) - # Reminder: -u #{fetch(:pg_system_user)} seen below differs slightly from -U, an option on the psql command: https://www.postgresql.org/docs/9.6/static/app-psql.html - args.unshift("-U #{fetch(:pg_system_user)}") if fetch(:pg_without_sudo) # Add the :pg_system_user to psql command since we aren't using sudo anymore - # test :sudo, "-u #{fetch(:pg_system_user)} psql", *args - cmd = [ :psql, *args ] - cmd = [ :sudo, "-u #{fetch(:pg_system_user)}", *cmd ] unless fetch(:pg_without_sudo) - test *cmd.flatten - end - - # Runs psql on the application database - def psql_on_app_db(*args) - psql_on_db(fetch(:pg_database), *args) + def psql(type, database, *args) + cmd = [ :psql, "-d #{database}", *args ] + if fetch(:pg_without_sudo) + args.unshift("-U #{fetch(:pg_system_user)}") # Add the :pg_system_user to psql command since we aren't using sudo anymore + else + cmd = [:sudo, "-i -u #{fetch(:pg_system_user)}", *cmd] + end + if type == 'test' + test *cmd.flatten + else + execute *cmd.flatten + end end - def db_user_exists? - psql_on_db fetch(:pg_system_db),'-tAc', %Q{"SELECT 1 FROM pg_roles WHERE rolname='#{fetch(:pg_username)}';" | grep -q 1} + def database_user_exists? + psql 'test', fetch(:pg_system_db),'-tAc', %Q{"SELECT 1 FROM pg_roles WHERE rolname='#{fetch(:pg_username)}';" | grep -q 1} end def database_exists? - psql_on_db fetch(:pg_system_db), '-tAc', %Q{"SELECT 1 FROM pg_database WHERE datname='#{fetch(:pg_database)}';" | grep -q 1} + psql 'test', fetch(:pg_system_db), '-tAc', %Q{"SELECT 1 FROM pg_database WHERE datname='#{fetch(:pg_database)}';" | grep -q 1} end - private - - def psql_on_db(db_name, *args) - args.unshift("-U #{fetch(:pg_system_user)}") if fetch(:pg_without_sudo) # Add the :pg_system_user to psql command since we aren't using sudo anymore - cmd = [ :psql, "-d #{db_name}", *args ] - cmd = [ :sudo, "-u #{fetch(:pg_system_user)}", *cmd ] unless fetch(:pg_without_sudo) - puts "Executing #{cmd.flatten}" - test *cmd.flatten - #test :sudo, "-u #{fetch(:pg_system_user)} psql -d #{db_name}", *args - end - end end end diff --git a/lib/capistrano/postgresql/version.rb b/lib/capistrano/postgresql/version.rb index d91b277..cf9d277 100644 --- a/lib/capistrano/postgresql/version.rb +++ b/lib/capistrano/postgresql/version.rb @@ -1,5 +1,5 @@ module Capistrano module Postgresql - VERSION = '4.9.2' + VERSION = '5.0.0' end end diff --git a/lib/capistrano/tasks/postgresql.rake b/lib/capistrano/tasks/postgresql.rake index b234785..d50fa5e 100644 --- a/lib/capistrano/tasks/postgresql.rake +++ b/lib/capistrano/tasks/postgresql.rake @@ -13,8 +13,9 @@ namespace :load do set :pg_database, -> { "#{fetch(:application)}_#{fetch(:stage)}" } set :pg_pool, 5 set :pg_username, -> { fetch(:pg_database) } - set :pg_ask_for_password, false - set :pg_password, -> { ask_for_or_generate_password } + set :pg_generate_random_password, nil + set :pg_ask_for_password, nil + set :pg_password, -> { pg_password_generate } set :pg_socket, '' set :pg_host, -> do # for multiple release nodes automatically use server hostname (IP?) in the database.yml release_roles(:all).count == 1 && release_roles(:all).first == primary(:db) ? 'localhost' : primary(:db).hostname @@ -44,86 +45,58 @@ namespace :postgresql do # undocumented, for a reason: drops database. Use with care! task :remove_all do on release_roles :all do - if test "[ -e #{database_yml_file} ]" - execute :rm, database_yml_file - end + execute :rm, database_yml_file if test "[ -e #{database_yml_file} ]" end - on primary :db do - if test "[ -e #{archetype_database_yml_file} ]" - execute :rm, archetype_database_yml_file - end + execute :rm, archetype_database_yml_file if test "[ -e #{archetype_database_yml_file} ]" end - on roles :db do - psql '-c', %Q{"DROP database \\"#{fetch(:pg_database)}\\";"} - psql '-c', %Q{"DROP user \\"#{fetch(:pg_username)}\\";"} + psql'execute', fetch(:pg_system_db), '-c', %Q{"DROP database \\"#{fetch(:pg_database)}\\";"} if database_exists? + psql 'execute', fetch(:pg_system_db),'-c', %Q{"DROP user \\"#{fetch(:pg_username)}\\";"}if database_user_exists? + remove_extensions end + puts 'Removed database.yml from all hosts, Database, Database User, and Removed Extensions' end task :remove_app_database_yml_files do # We should never delete archetype files. The generate_database_yml_archetype task will handle updates on release_roles :app do - if test "[ -e #{database_yml_file} ]" - execute :rm, database_yml_file - end + execute :rm, database_yml_file if test "[ -e #{database_yml_file} ]" end end desc 'Remove pg_extension from postgresql db' task :remove_extensions do - next unless Array( fetch(:pg_extensions) ).any? - on roles :db do - # remove in reverse order if extension is present - Array( fetch(:pg_extensions) ).reverse.each do |ext| - psql_on_app_db '-c', %Q{"DROP EXTENSION IF EXISTS #{ext};"} unless [nil, false, ""].include?(ext) - end - end - end - - desc 'Add the hstore extension to postgresql' - task :add_hstore do - next unless fetch(:pg_use_hstore) - on roles :db do - psql_on_app_db '-c', %Q{"CREATE EXTENSION IF NOT EXISTS hstore;"} - end + remove_extensions end desc 'Add pg_extension to postgresql db' task :add_extensions do - next unless Array( fetch(:pg_extensions) ).any? on roles :db do - Array( fetch(:pg_extensions) ).each do |ext| - next if [nil, false, ''].include?(ext) - if psql_on_app_db '-c', %Q{"CREATE EXTENSION IF NOT EXISTS #{ext};"} - puts "- Added extension #{ext} to #{fetch(:pg_database)}" - else - error "postgresql: adding extension #{ext} failed!" - exit 1 + if Array( fetch(:pg_extensions) ).any? + Array( fetch(:pg_extensions) ).each do |ext| + next if [nil, false, ''].include?(ext) + psql 'execute', fetch(:pg_system_db), '-c', %Q{"CREATE EXTENSION IF NOT EXISTS #{ext};"}unless extension_exists?(ext) end end end end - desc 'Create database' - task :create_database do + desc 'Create pg_username in database' + task :create_database_user do on roles :db do - next if database_exists? - unless psql_on_db fetch(:pg_system_db), '-c', %Q{"CREATE DATABASE \\"#{fetch(:pg_database)}\\" OWNER \\"#{fetch(:pg_username)}\\";"} - error 'postgresql: creating database failed!' - exit 1 + unless database_user_exists? + # If you use CREATE USER instead of CREATE ROLE the LOGIN right is granted automatically; otherwise you must specify it in the WITH clause of the CREATE statement. + psql 'execute', fetch(:pg_system_db), '-c', %Q{"CREATE USER \\"#{fetch(:pg_username)}\\" PASSWORD '#{fetch(:pg_password)}';"} end end end - desc 'Create DB user' - task :create_db_user do + desc 'Create database' + task :create_database do on roles :db do - next if db_user_exists? - # If you use CREATE USER instead of CREATE ROLE the LOGIN right is granted automatically; otherwise you must specify it in the WITH clause of the CREATE statement. - unless psql_on_db fetch(:pg_system_db), '-c', %Q{"CREATE USER \\"#{fetch(:pg_username)}\\" PASSWORD '#{fetch(:pg_password)}';"} - error "postgresql: creating database user \"#{fetch(:pg_username)}\" failed!" - exit 1 + unless database_exists? + psql 'execute', fetch(:pg_system_db), '-c', %Q{"CREATE DATABASE \\"#{fetch(:pg_database)}\\" OWNER \\"#{fetch(:pg_username)}\\";"} end end end @@ -148,7 +121,6 @@ namespace :postgresql do on primary :db do database_yml_contents = download! archetype_database_yml_file end - on release_roles :all do execute :mkdir, '-pv', File.dirname(database_yml_file) Net::SCP.upload!(self.host.hostname, self.host.user, StringIO.new(database_yml_contents), database_yml_file) @@ -163,26 +135,31 @@ namespace :postgresql do desc 'Postgresql setup tasks' task :setup do - puts "* ============================= * \n All psql commands will be run #{fetch(:pg_without_sudo) ? 'without sudo' : 'with sudo'}\n You can modify this in your app/config/deploy/#{fetch(:rails_env)}.rb by setting the pg_without_sudo boolean \n* ============================= *" + puts "* ===== Postgresql Setup ===== *\n" + puts " All psql commands will be run #{fetch(:pg_without_sudo) ? 'without sudo' : 'with sudo'}\n You can modify this in your app/config/deploy/#{fetch(:rails_env)}.rb by setting the pg_without_sudo boolean.\n" if release_roles(:app).empty? - puts "There are no servers in your app/config/deploy/#{fetch(:rails_env)}.rb with a :app role... Skipping Postgresql setup." + warn " WARNING: There are no servers in your app/config/deploy/#{fetch(:rails_env)}.rb with a :app role... Skipping Postgresql setup." else invoke 'postgresql:remove_app_database_yml_files' # Deletes old yml files from all servers. Allows you to avoid having to manually delete the files on your app servers to get a new pool size for example. Don't touch the archetype file to avoid deleting generated passwords. if release_roles(:db).empty? # Test to be sure we have a :db role host - puts "There is no server in your app/config/deploy/#{fetch(:rails_env)}.rb with a :db role... Skipping Postgresql setup." + warn " WARNING: There is no server in your app/config/deploy/#{fetch(:rails_env)}.rb with a :db role... Skipping Postgresql setup." + elsif !fetch(:pg_password) && !fetch(:pg_generate_random_password) && !fetch(:pg_ask_for_password) + warn " WARNING: There is no :pg_password set in your app/config/deploy/#{fetch(:rails_env)}.rb.\n If you don't wish to set it, 'set :pg_generate_random_password, true' or 'set :pg_ask_for_password, true' are available!" + elsif fetch(:pg_generate_random_password) && fetch(:pg_ask_for_password) + warn " WARNING: You cannot have both :pg_generate_random_password and :pg_ask_for_password enabled in app/config/deploy/#{fetch(:rails_env)}.rb." else - invoke 'postgresql:create_db_user' + invoke 'postgresql:create_database_user' invoke 'postgresql:create_database' - invoke 'postgresql:add_hstore' invoke 'postgresql:add_extensions' invoke 'postgresql:generate_database_yml_archetype' invoke 'postgresql:generate_database_yml' end end + puts "* ============================= *" end end desc 'Server setup tasks' task :setup do - invoke "postgresql:setup" + invoke 'postgresql:setup' end