Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Merge branch 'config_autogeneration' into track_upstream

  • Loading branch information...
commit 38a1704cabfbba203778e6ca3a112776720d54dd 2 parents b24929e + 160410c
@davidc-donorschoose davidc-donorschoose authored
View
94 README.md
@@ -88,6 +88,9 @@ be dynamically rendered from the attributes. The helpful commentary
will no longer be present. You should consult the PostgreSQL
documentation for specific configuration details.
+See __Recipes__ `config_initdb` and `config_pgtune` below to
+auto-generate many postgresql.conf settings.
+
For values that are "on" or "off", they should be specified as literal
`true` or `false`. String values will be used with single quotes. Any
configuration option set to the literal `nil` will be skipped
@@ -197,6 +200,96 @@ database, and manages the postgresql service. You should include the
`postgresql::server` recipe, which will include this on RHEL/Fedora
platforms.
+config\_initdb
+--------------
+
+Takes locale and timezone settings from the system configuration.
+This recipe creates `node.default['postgresql']['config']` attributes
+that conform to the system's locale and timezone. In addition, this
+recipe creates the same error reporting and logging settings that
+`initdb` provided: a rotation of 7 days of log files named
+postgresql-Mon.log, etc.
+
+The default attributes created by this recipe are easy to override with
+normal attributes because of Chef attribute precedence. For example,
+suppose a DBA wanted to keep log files indefinitely, rolling over daily
+or when growing to 10MB. The Chef installation could include the
+`postgresql::config_initdb` recipe for the locale and timezone settings,
+but customize the logging settings with these node JSON attributes:
+
+ "postgresql": {
+ "config": {
+ "log_rotation_age": "1d",
+ "log_rotation_size": "10MB",
+ "log_filename": "postgresql-%Y-%m-%d_%H%M%S.log"
+ }
+ }
+
+Credits: This `postgresql::config_initdb` recipe is based on algorithms
+in the [source code](http://doxygen.postgresql.org/initdb_8c_source.html)
+for the PostgreSQL `initdb` utility.
+
+config\_pgtune
+--------------
+
+Performance tuning.
+Takes the wimpy default postgresql.conf and expands the database server
+to be as powerful as the hardware it's being deployed on. This recipe
+creates a baseline configuration of `node.default['postgresql']['config']`
+attributes in the right general range for a dedicated Postgresql system.
+Most installations won't need additional performance tuning.
+
+The only decision you need to make is to choose a `db_type` from the
+following database workloads. (See the recipe code comments for more
+detailed descriptions.)
+
+ * "dw" -- Data Warehouse
+ * "oltp" -- Online Transaction Processing
+ * "web" -- Web Application
+ * "mixed" -- Mixed DW and OLTP characteristics
+ * "desktop" -- Not a dedicated database
+
+This recipe uses a performance model with three input parameters.
+These node attributes are completely optional, but it is obviously
+important to choose the `db_type` correctly:
+
+ * `node['postgresql']['config_pgtune']['db_type']` --
+ Specifies database type from the list of five choices above.
+ If not specified, the default is "mixed".
+
+ * `node['postgresql']['config_pgtune']['max_connections']` --
+ Specifies maximum number of connections expected.
+ If not specified, it depends on database type:
+ "web":200, "oltp":300, "dw":20, "mixed":80, "desktop":5
+
+ * `node['postgresql']['config_pgtune']['total_memory']` --
+ Specifies total system memory in kB. (E.g., "49416564kB".)
+ If not specified, it will be taken from Ohai automatic attributes.
+ This could be used to tune a system that isn't a dedicated database.
+
+The default attributes created by this recipe are easy to override with
+normal attributes because of Chef attribute precedence. For example, if
+you are running application benchmarks to try different buffer cache
+sizes, you would experiment with this node JSON attribute:
+
+ "postgresql": {
+ "config": {
+ "shared_buffers": "3GB"
+ }
+ }
+
+Note that the recipe uses `max_connections` in its computations. If
+you want to override that setting, you should specify
+`node['postgresql']['config_pgtune']['max_connections']` instead of
+`node['postgresql']['config']['max_connections']`.
+
+Credits: This `postgresql::config_pgtune` recipe is based on the
+[pgtune python script](https://github.com/gregs1104/pgtune)
+developed by
+[Greg Smith](http://notemagnet.blogspot.com/2008/11/automating-initial-postgresqlconf.html)
+and
+[other pgsql-hackers](http://www.postgresql.org/message-id/491C6CDC.8090506@agliodbs.com).
+
contrib
-------
@@ -305,6 +398,7 @@ License and Author
- Author:: Joshua Timberman (<joshua@opscode.com>)
- Author:: Lamont Granquist (<lamont@opscode.com>)
- Author:: Chris Roberts (<chrisroberts.code@gmail.com>)
+- Author:: David Crane (<davidc@donorschoose.org>)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
View
294 libraries/default.rb
@@ -0,0 +1,294 @@
+#
+# Cookbook Name:: postgresql
+# Library:: default
+# Author:: David Crane (<davidc@donorschoose.org>)
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+module Opscode
+ module PostgresqlHelpers
+
+#######
+# Function to truncate value to 4 significant bits, render human readable.
+# Used in recipes/config_initdb.rb to set this attribute:
+#
+# The memory settings (shared_buffers, effective_cache_size, work_mem,
+# maintenance_work_mem and wal_buffers) will be rounded down to keep
+# the 4 most significant bits, so that SHOW will be likely to use a
+# larger divisor. The output is actually a human readable string that
+# ends with "GB", "MB" or "kB" if over 1023, exactly what Postgresql
+# will expect in a postgresql.conf setting. The output may be up to
+# 6.25% less than the original value because of the rounding.
+def binaryround(value)
+
+ # Keep a multiplier which grows through powers of 1
+ multiplier = 1
+
+ # Truncate value to 4 most significant bits
+ while value >= 16
+ value = (value / 2).floor
+ multiplier = multiplier * 2
+ end
+
+ # Factor any remaining powers of 2 into the multiplier
+ while value == 2*((value / 2).floor)
+ value = (value / 2).floor
+ multiplier = multiplier * 2
+ end
+
+ # Factor enough powers of 2 back into the value to
+ # leave the multiplier as a power of 1024 that can
+ # be represented as units of "GB", "MB" or "kB".
+ if multiplier >= 1024*1024*1024
+ while multiplier > 1024*1024*1024
+ value = 2*value
+ multiplier = (multiplier/2).floor
+ end
+ multiplier = 1
+ units = "GB"
+
+ elsif multiplier >= 1024*1024
+ while multiplier > 1024*1024
+ value = 2*value
+ multiplier = (multiplier/2).floor
+ end
+ multiplier = 1
+ units = "MB"
+
+ elsif multiplier >= 1024
+ while multiplier > 1024
+ value = 2*value
+ multiplier = (multiplier/2).floor
+ end
+ multiplier = 1
+ units = "kB"
+
+ else
+ units = ""
+ end
+
+ # Now we can return a nice human readable string.
+ return "#{multiplier * value}#{units}"
+end
+
+#######
+# Locale Configuration
+
+# Function to test the date order.
+# Used in recipes/config_initdb.rb to set this attribute:
+# node.default['postgresql']['config']['datestyle']
+def locale_date_order
+ # Test locale conversion of mon=11, day=22, year=33
+ testtime = DateTime.new(2033,11,22,0,0,0,"-00:00")
+ #=> #<DateTime: 2033-11-22T00:00:00-0000 ...>
+
+ # %x - Preferred representation for the date alone, no time
+ res = testtime.strftime("%x")
+
+ if res.nil?
+ return 'mdy'
+ end
+
+ posM = res.index("11")
+ posD = res.index("22")
+ posY = res.index("33")
+
+ if (posM.nil? || posD.nil? || posY.nil?)
+ return 'mdy'
+ elseif (posY < posM && posM < posD)
+ return 'ymd'
+ elseif (posD < posM)
+ return 'dmy'
+ else
+ return 'mdy'
+ end
+end
+
+#######
+# Timezone Configuration
+require 'find'
+
+# Function to determine where the system stored shared timezone data.
+# Used in recipes/config_initdb.rb to detemine where it should have
+# select_default_timezone(tzdir) search.
+def pg_TZDIR()
+ # System time zone conversions are controlled by a timezone data file
+ # identified through environment variables (TZ and TZDIR) and/or file
+ # and directory naming conventions specific to the Linux distribution.
+ # Each of these timezone names will have been loaded into the PostgreSQL
+ # pg_timezone_names view by the package maintainer.
+ #
+ # Instead of using the timezone name configured as the system default,
+ # the PostgreSQL server uses ones named in postgresql.conf settings
+ # (timezone and log_timezone). The initdb utility does initialize those
+ # settings to the timezone name that corresponds to the system default.
+ #
+ # The system's timezone name is actually a filename relative to the
+ # shared zoneinfo directory. That is usually /usr/share/zoneinfo, but
+ # it was /usr/lib/zoneinfo in older distributions and can be anywhere
+ # if specified by the environment variable TZDIR. The tzset(3) manpage
+ # seems to indicate the following precedence:
+ tzdir = nil
+ if ::File.directory?("/usr/lib/zoneinfo")
+ tzdir = "/usr/lib/zoneinfo"
+ else
+ share_path = [ ENV['TZDIR'], "/usr/share/zoneinfo" ].compact.first
+ if ::File.directory?(share_path)
+ tzdir = share_path
+ end
+ end
+ return tzdir
+end
+
+#######
+# Function to support select_default_timezone(tzdir), which is
+# used in recipes/config_initdb.rb.
+def validate_zone(tzname)
+ # PostgreSQL does not support leap seconds, so this function tests
+ # the usual Linux tzname convention to avoid a misconfiguration.
+ # Assume that the tzdata package maintainer has kept all timezone
+ # data files with support for leap seconds is kept under the
+ # so-named "right/" subdir of the shared zoneinfo directory.
+ #
+ # The original PostgreSQL initdb is not Unix-specific, so it did a
+ # very complicated, thorough test in its pg_tz_acceptable() function
+ # that I could not begin to understand how to do in ruby :).
+ #
+ # Testing the tzname is good enough, since a misconfiguration
+ # will result in an immediate fatal error when the PostgreSQL
+ # service is started, with pgstartup.log messages such as:
+ # LOG: time zone "right/US/Eastern" appears to use leap seconds
+ # DETAIL: PostgreSQL does not support leap seconds.
+
+ if tzname.index("right/") == 0
+ return false
+ else
+ return true
+ end
+end
+
+# Function to support select_default_timezone(tzdir), which is
+# used in recipes/config_initdb.rb.
+def scan_available_timezones(tzdir)
+ # There should be an /etc/localtime zoneinfo file that is a link to
+ # (or a copy of) a timezone data file under tzdir, which should have
+ # been installed under the "share" directory by the tzdata package.
+ #
+ # The initdb utility determines which shared timezone file is being
+ # used as the system's default /etc/localtime. The timezone name is
+ # the timezone file path relative to the tzdir.
+
+ bestzonename = nil
+
+ if (tzdir.nil?)
+ Chef::Log.error("The zoneinfo directory not found (looked for /usr/share/zoneinfo and /usr/lib/zoneinfo)")
+ elsif !::File.exists?("/etc/localtime")
+ Chef::Log.error("The system zoneinfo file not found (looked for /etc/localtime)")
+ elsif ::File.directory?("/etc/localtime")
+ Chef::Log.error("The system zoneinfo file not found (/etc/localtime is a directory instead)")
+ elsif ::File.symlink?("/etc/localtime")
+ # PostgreSQL initdb doesn't use the symlink target, but this
+ # certainly will make sense to any system administrator. A full
+ # scan of the tzdir to find the shortest filename could result
+ # "US/Eastern" instead of "America/New_York" as bestzonename,
+ # in spite of what the sysadmin had specified in the symlink.
+ # (There are many duplicates under tzdir, with the same timezone
+ # content appearing as an average of 2-3 different file names.)
+ path = ::File.readlink("/etc/localtime")
+ bestzonename = path.gsub("#{tzdir}/","")
+ else # /etc/localtime is a file, so scan for it under tzdir
+ localtime_content = File.read("/etc/localtime")
+
+ Find.find(tzdir) do |path|
+ # Only consider files (skip directories or symlinks)
+ if !::File.directory?(path) && !::File.symlink?(path)
+ # Ignore any file named "posixrules" or "localtime"
+ if ::File.basename(path) != "posixrules" && ::File.basename(path) != "localtime"
+ # Do consider if content exactly matches /etc/localtime.
+ if localtime_content == File.read(path)
+ tzname = path.gsub("#{tzdir}/","")
+ if validate_zone(tzname)
+ if (bestzonename.nil? ||
+ tzname.length < bestzonename.length ||
+ (tzname.length == bestzonename.length &&
+ (tzname <=> bestzonename) < 0)
+ )
+ bestzonename = tzname
+ end
+ end
+ end
+ end
+ end
+ end
+ end
+
+ return bestzonename
+end
+
+# Function to support select_default_timezone(tzdir), which is
+# used in recipes/config_initdb.rb.
+def identify_system_timezone(tzdir)
+ resultbuf = scan_available_timezones(tzdir)
+
+ if !resultbuf.nil?
+ # Ignore Olson's rather silly "Factory" zone; use GMT instead
+ if (resultbuf <=> "Factory") == 0
+ resultbuf = nil
+ end
+
+ else
+ # Did not find the timezone. Fallback to use a GMT zone. Note that the
+ # Olson timezone database names the GMT-offset zones in POSIX style: plus
+ # is west of Greenwich.
+ testtime = DateTime.now
+ std_ofs = testtime.strftime("%:z").split(":")[0].to_i
+
+ resultbuf = [
+ "Etc/GMT",
+ (-std_ofs > 0) ? "+" : "",
+ (-std_ofs).to_s
+ ].join('')
+ end
+
+ return resultbuf
+end
+
+#######
+# Function to determine the name of the system's default timezone.
+# Used in recipes/config_initdb.rb to set these attributes:
+# node.default['postgresql']['config']['log_timezone']
+# node.default['postgresql']['config']['timezone']
+def select_default_timezone(tzdir)
+
+ system_timezone = nil
+
+ # Check TZ environment variable
+ tzname = ENV['TZ']
+ if !tzname.nil? && !tzname.empty? && validate_zone(tzname)
+ system_timezone = tzname
+
+ else
+ # Nope, so try to identify system timezone from /etc/localtime
+ tzname = identify_system_timezone(tzdir)
+ if validate_zone(tzname)
+ system_timezone = tzname
+ end
+ end
+
+ return system_timezone
+end
+
+# End the Opscode::PostgresqlHelper module
+ end
+end
View
148 recipes/config_initdb.rb
@@ -0,0 +1,148 @@
+#
+# Cookbook Name:: postgresql
+# Recipe:: config_initdb
+# Author:: David Crane (<davidc@donorschoose.org>)
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+#######
+# Load the locale_date_order() and select_default_timezone(tzdir)
+# methods from libraries/default.rb
+::Chef::Recipe.send(:include, Opscode::PostgresqlHelpers)
+
+#######
+# This recipe is derived from the setup_config() source code in the
+# PostgreSQL initdb utility. It determines postgresql.conf settings that
+# conform to the system's locale and timezone configuration, and also
+# sets the error reporting and logging settings.
+#
+# See http://doxygen.postgresql.org/initdb_8c_source.html for the
+# original initdb source code.
+#
+# By examining the system configuration, this recipe will set the
+# following node.default['postgresql']['config'] attributes:
+#
+# - Locale and Formatting -
+# * datestyle
+# * lc_messages
+# * lc_monetary
+# * lc_numeric
+# * lc_time
+# * default_text_search_config
+#
+# - Timezone Conversion -
+# * log_timezone
+# * timezone
+#
+# In addition, this recipe will recommend the same error reporting and
+# logging settings that initdb provided. These settings do differ from
+# the PostgreSQL default settings, which would log to stderr only. The
+# initdb settings rotate 7 days of log files named postgresql-Mon.log,
+# etc. through these node.default['postgresql']['config'] attributes:
+#
+# - Where to Log -
+# * log_destination = 'stderr'
+# * log_directory = 'pg_log'
+# * log_filename = 'postgresql-%a.log'
+# (Default was: postgresql-%Y-%m-%d_%H%M%S.log)
+# * logging_collector = true # on
+# (Turned on to capture stderr logging and redirect into log files)
+# (Default was: false # off)
+# * log_rotation_age = 1d
+# * log_rotation_size = 0
+# (Default was: 10MB)
+# * log_truncate_on_rotation = true # on
+# (Default was: false # off)
+
+#######
+# Locale Configuration
+
+# See libraries/default.rb for the locale_date_order() method.
+node.default['postgresql']['config']['datestyle'] = "iso, #{locale_date_order()}"
+
+# According to the locale(1) manpage, the locale settings are determined
+# by environment variables according to the following precedence:
+# LC_ALL > (LC_MESSAGES, LC_MONETARY, LC_NUMERIC, LC_TIME) > LANG.
+
+node.default['postgresql']['config']['lc_messages'] =
+ [ ENV['LC_ALL'], ENV['LC_MESSAGES'], ENV['LANG'] ].compact.first
+
+node.default['postgresql']['config']['lc_monetary'] =
+ [ ENV['LC_ALL'], ENV['LC_MONETARY'], ENV['LANG'] ].compact.first
+
+node.default['postgresql']['config']['lc_numeric'] =
+ [ ENV['LC_ALL'], ENV['LC_NUMERIC'], ENV['LANG'] ].compact.first
+
+node.default['postgresql']['config']['lc_time'] =
+ [ ENV['LC_ALL'], ENV['LC_TIME'], ENV['LANG'] ].compact.first
+
+node.default['postgresql']['config']['default_text_search_config'] =
+ case ENV['LANG']
+ when /da_.*/
+ 'pg_catalog.danish'
+ when /nl_.*/
+ 'pg_catalog.dutch'
+ when /en_.*/
+ 'pg_catalog.english'
+ when /fi_.*/
+ 'pg_catalog.finnish'
+ when /fr_.*/
+ 'pg_catalog.french'
+ when /de_.*/
+ 'pg_catalog.german'
+ when /hu_.*/
+ 'pg_catalog.hungarian'
+ when /it_.*/
+ 'pg_catalog.italian'
+ when /no_.*/
+ 'pg_catalog.norwegian'
+ when /pt_.*/
+ 'pg_catalog.portuguese'
+ when /ro_.*/
+ 'pg_catalog.romanian'
+ when /ru_.*/
+ 'pg_catalog.russian'
+ when /es_.*/
+ 'pg_catalog.spanish'
+ when /sv_.*/
+ 'pg_catalog.swedish'
+ when /tr_.*/
+ 'pg_catalog.turkish'
+ else
+ nil
+ end
+
+#######
+# Timezone Configuration
+
+# Determine the name of the system's default timezone and specify node
+# defaults for the postgresql.cof settings. If the timezone cannot be
+# identified, do as initdb would do: leave it unspecified so PostgreSQL
+# uses it's internal default of GMT.
+tzdirpath = pg_TZDIR() # See libraries/default.rb
+default_timezone = select_default_timezone(tzdirpath) # See libraries/default.rb
+if !default_timezone.nil?
+ node.default['postgresql']['config']['log_timezone'] = default_timezone
+ node.default['postgresql']['config']['timezone'] = default_timezone
+end
+
+#######
+# - Where to Log -
+node.default['postgresql']['config']['log_destination'] = 'stderr'
+node.default['postgresql']['config']['log_directory'] = 'pg_log'
+node.default['postgresql']['config']['log_filename'] = 'postgresql-%a.log'
+node.default['postgresql']['config']['logging_collector'] = true # on
+node.default['postgresql']['config']['log_rotation_age'] = '1d'
+node.default['postgresql']['config']['log_rotation_size'] = 0
+node.default['postgresql']['config']['log_truncate_on_rotation'] = true # on
View
280 recipes/config_pgtune.rb
@@ -0,0 +1,280 @@
+#
+# Cookbook Name:: postgresql
+# Recipe:: config_pgtune
+# Author:: David Crane (<davidc@donorschoose.org>)
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+#######
+# Load the binaryround(value) method from libraries/default.rb
+::Chef::Recipe.send(:include, Opscode::PostgresqlHelpers)
+
+#######
+# This recipe is based on Greg Smith's pgtune script (the Feb 1, 2012
+# version at https://github.com/gregs1104/pgtune). Introduction: pgtune
+# takes the wimpy default postgresql.conf and expands the database
+# server to be as powerful as the hardware it's being deployed on.
+#
+# The default postgresql.conf aims at a system with approximately 128MB
+# of RAM. This recipe recommends a baseline configuration in the right
+# general range for a dedicated Postgresql system.
+#
+# This recipe takes three optional parameters that may be passed in as
+# node['postgresql']['config_pgtune'] attributes:
+# * db_type -- Specifies database type as one of: dw, oltp,
+# web, mixed, desktop. If not specified, the default is mixed.
+# * max_connections -- Specifies number of maximum connections
+# expected. If not specified, it depends on database type.
+# * total_memory -- Specifies total system memory. If not specified,
+# it will be detected from the Ohai automatic attributes.
+#
+# Using those inputs, this recipe will compute and set the following
+# node.default['postgresql']['config'] attributes:
+# * max_connections
+# * shared_buffers
+# * effective_cache_size
+# * work_mem
+# * maintenance_work_mem
+# * checkpoint_segments
+# * checkpoint_completion_target
+# * wal_buffers
+# * default_statistics_target
+#
+# This recipe deviates from the original pgtune script for 2 settings:
+# shared_buffers is capped for large memory systems (which Greg
+# mentioned in a TODO.rst) and wal_buffers will auto-tune starting with
+# 9.1 (which is a feature that Greg built into Postgresql).
+
+#######
+# These are the workload characteristics of the five database types
+# that can be specified as node['postgresql']['config_pgtune']['db_type']:
+#
+# dw -- Data Warehouse
+# * Typically I/O- or RAM-bound
+# * Large bulk loads of data
+# * Large complex reporting queries
+# * Also called "Decision Support" or "Business Intelligence"
+#
+# oltp -- Online Transaction Processing
+# * Typically CPU- or I/O-bound
+# * DB slightly larger than RAM to 1TB
+# * 20-40% small data write queries
+# * Some long transactions and complex read queries
+#
+# web -- Web Application
+# * Typically CPU-bound
+# * DB much smaller than RAM
+# * 90% or more simple queries
+#
+# mixed -- Mixed DW and OLTP characteristics
+# * A wide mixture of queries
+#
+# desktop -- Not a dedicated database
+# * A general workstation, perhaps for a developer
+
+# Parse out db_type option, or use default.
+db_type = 'mixed'
+
+if (node['postgresql'].attribute?('config_pgtune') && node['postgresql']['config_pgtune'].attribute?('db_type'))
+ db_type = node['postgresql']['config_pgtune']['db_type']
+ if (!(["dw","oltp","web","mixed","desktop"].include?(db_type)))
+ Chef::Application.fatal!([
+ "Bad value (#{db_type})",
+ "for node['postgresql']['config_pgtune']['db_type'] attribute.",
+ "Valid values are one of dw, oltp, web, mixed, desktop."
+ ].join(' '))
+ end
+end
+
+# Parse out max_connections option, or use a value based on db_type.
+con =
+{ "web" => 200,
+ "oltp" => 300,
+ "dw" => 20,
+ "mixed" => 80,
+ "desktop" => 5
+}.fetch(db_type)
+
+if (node['postgresql'].attribute?('config_pgtune') && node['postgresql']['config_pgtune'].attribute?('max_connections'))
+ max_connections = node['postgresql']['config_pgtune']['max_connections']
+ if (max_connections.match(/\A[1-9]\d*\Z/) == nil)
+ Chef::Application.fatal!([
+ "Bad value (#{max_connections})",
+ "for node['postgresql']['config_pgtune']['max_connections'] attribute.",
+ "Valid values are non-zero integers only."
+ ].join(' '))
+ end
+ con = max_connections.to_i
+end
+
+# Parse out total_memory option, or use value detected by Ohai.
+total_memory = node['memory']['total']
+
+# Override max_connections with a node attribute if DevOps desires.
+# For example, on a system *not* dedicated to Postgresql.
+if (node['postgresql'].attribute?('config_pgtune') && node['postgresql']['config_pgtune'].attribute?('total_memory'))
+ total_memory = node['postgresql']['config_pgtune']['total_memory']
+ if (total_memory.match(/\A[1-9]\d*kB\Z/) == nil)
+ Chef::Application.fatal!([
+ "Bad value (#{total_memory})",
+ "for node['postgresql']['config_pgtune']['total_memory'] attribute.",
+ "Valid values are non-zero integers followed by kB (e.g., 49416564kB)."
+ ].join(' '))
+ end
+end
+
+# Ohai reports node[:memory][:total] in kB, as in "921756kB"
+mem = total_memory.split("kB")[0].to_i / 1024 # in MB
+
+#######
+# RAM-related settings computed as in Greg Smith's pgtune script.
+# Remember that con and mem were either chosen above based on the
+# db_type or the actual total memory, or were passed in attributes.
+
+# (1) max_connections
+# Sets the maximum number of concurrent connections.
+node.default['postgresql']['config']['max_connections'] = con
+
+# The calculations for the next four settings would not be optimal
+# for low memory systems. In that case, the calculation is skipped,
+# leaving the built-in Postgresql settings, which are actually
+# intended for those low memory systems.
+if (mem >= 256)
+
+ # (2) shared_buffers
+ # Sets the number of shared memory buffers used by the server.
+ shared_buffers =
+ { "web" => mem/4,
+ "oltp" => mem/4,
+ "dw" => mem/4,
+ "mixed" => mem/4,
+ "desktop" => mem/16
+ }.fetch(db_type)
+
+ # Robert Haas has advised to cap the size of shared_buffers based on
+ # the memory architecture: 2GB on 32-bit and 8GB on 64-bit machines.
+ # http://rhaas.blogspot.com/2012/03/tuning-sharedbuffers-and-walbuffers.html
+ case node['kernel']['machine']
+ when "i386" # 32-bit machines
+ if shared_buffers > 2*1024
+ shared_buffers = 2*1024
+ end
+ when "x86_64" # 64-bit machines
+ if shared_buffers > 8*1024
+ shared_buffers = 8*1024
+ end
+ end
+
+ node.default['postgresql']['config']['shared_buffers'] = binaryround(shared_buffers*1024*1024)
+
+ # (3) effective_cache_size
+ # Sets the planner's assumption about the size of the disk cache.
+ # That is, the portion of the kernel's disk cache that will be
+ # used for PostgreSQL data files.
+ effective_cache_size =
+ { "web" => mem * 3 / 4,
+ "oltp" => mem * 3 / 4,
+ "dw" => mem * 3 / 4,
+ "mixed" => mem * 3 / 4,
+ "desktop" => mem / 4
+ }.fetch(db_type)
+
+ node.default['postgresql']['config']['effective_cache_size'] = binaryround(effective_cache_size*1024*1024)
+
+ # (4) work_mem
+ # Sets the maximum memory to be used for query workspaces.
+ work_mem =
+ { "web" => mem / con,
+ "oltp" => mem / con,
+ "dw" => mem / con / 2,
+ "mixed" => mem / con / 2,
+ "desktop" => mem / con / 6
+ }.fetch(db_type)
+
+ node.default['postgresql']['config']['work_mem'] = binaryround(work_mem*1024*1024)
+
+ # (5) maintenance_work_mem
+ # Sets the maximum memory to be used for maintenance operations.
+ # This includes operations such as VACUUM and CREATE INDEX.
+ maintenance_work_mem =
+ { "web" => mem / 16,
+ "oltp" => mem / 16,
+ "dw" => mem / 8,
+ "mixed" => mem / 16,
+ "desktop" => mem / 16
+ }.fetch(db_type)
+
+ # Cap maintenence RAM at 1GB on servers with lots of memory
+ if (maintenance_work_mem > 1*1024)
+ maintenance_work_mem = 1*1024
+ end
+
+ node.default['postgresql']['config']['maintenance_work_mem'] = binaryround(maintenance_work_mem*1024*1024)
+
+end
+
+#######
+# Checkpoint-related parameters that affect transaction rate and
+# maximum tolerable recovery playback time.
+
+# (6) checkpoint_segments
+# Sets the maximum distance in log segments between automatic WAL checkpoints.
+checkpoint_segments =
+{ "web" => 8,
+ "oltp" => 16,
+ "dw" => 64,
+ "mixed" => 16,
+ "desktop" => 3
+}.fetch(db_type)
+
+node.default['postgresql']['config']['checkpoint_segments'] = checkpoint_segments
+
+# (7) checkpoint_completion_target
+# Time spent flushing dirty buffers during checkpoint, as fraction
+# of checkpoint interval.
+checkpoint_completion_target =
+{ "web" => "0.7",
+ "oltp" => "0.9",
+ "dw" => "0.9",
+ "mixed" => "0.9",
+ "desktop" => "0.5"
+}.fetch(db_type)
+
+node.default['postgresql']['config']['checkpoint_completion_target'] = checkpoint_completion_target
+
+# (8) wal_buffers
+# Sets the number of disk-page buffers in shared memory for WAL.
+# Starting with 9.1, wal_buffers will auto-tune if set to the -1 default.
+# For 8.X and 9.0, it needed to be specified, which pgtune did as follows.
+if node['postgresql']['version'].to_f < 9.1
+ wal_buffers = 512 * checkpoint_segments
+ # The pgtune seems to use 1kB units for wal_buffers
+ node.default['postgresql']['config']['wal_buffers'] = binaryround(wal_buffers*1024)
+else
+ node.default['postgresql']['config']['wal_buffers'] = "-1"
+end
+
+# (9) default_statistics_target
+# Sets the default statistics target. This applies to table columns
+# that have not had a column-specific target set via
+# ALTER TABLE SET STATISTICS.
+default_statistics_target =
+{ "web" => 100,
+ "oltp" => 100,
+ "dw" => 500,
+ "mixed" => 100,
+ "desktop" => 100
+}.fetch(db_type)
+
+node.default['postgresql']['config']['default_statistics_target'] = default_statistics_target
Please sign in to comment.
Something went wrong with that request. Please try again.