Skip to content

Commit

Permalink
Largish commit. In essence it implements a couple of features. multip…
Browse files Browse the repository at this point in the history 
…le instances and sentinel changes to support multiple instances. To achieve this, a few things were changed.

  Redis:
    - Support for multiple_instances
    - Replication master support for multiple instances
    - Seperated slaveof_port as instances have their own ports
    - redis_binary_path needs to be in parameters
    - configurable redis_binary_path to support multiple
    - Configurable client output buffer limits
  Instances:
    - Each instance has it's own name
    - Instance has it's own init file,log file and conf file
    - Instance has it's own redis_binary_path, this is helpful when yo do ps aux and see instance name in the output
  Replication:
    - Slaveof port is added (instances run on different ports)
    - Check that We should not be making a host a slave of itself
  Sentinel:
    - Redhat needs it's own sentinel init file
    - Configurable sentinel_log_file

  Discovered Masters:
    - Sentinel configuration is tricky, You can bake who the master is in the configuration, but as the infrastructure lives on, the reality of who the sentinel thinks is the master for an instance changes. With the discovered_master flag you can use a simple .erb script (discover_master.erb) to find out who all the sentinel think the master is for a given instance. This allows you to reflect the current master in the sentinel configuration. If none of the sentinels are reachable or if the qourem is not reached, it defaults to the first item of $::redis::sentinel::members

    discovered_masters  is disabled by default and works only for instances right now.

Details:
manifests/params.pp
manifests/init.pp
manifests/config.pp
    - $::redis::params::redis_init_template
      $::redis::params::redis_init_file_mode
      $::redis::redis_binary_path
      $::redis::redis_binary_name
       ps aux on the command line gives a bunch of redis-server processes, unless the name of the binary has the name of the instance, it becomes very confusing to see what process is running which instance. This makes this nice. We need to modify the init file in order to pass the instance name to the process name.

       ps: I don't have suse/bsd/debian to test on. this may be wrong for those platforms.

    - $::redis::params::slaveof_port
      Multiple instances run on different ports and need multiple entries for master/slave, thus, we need to move the slave port to be independent of the slave of host.

    - $::redis::params::client_output_buffer_limit_normal
      $::redis::params::client_output_buffer_limit_slave
      $::redis::params::client_output_buffer_limit_pubsub
      Made the output buffer limits configurable.

    - $::redis::params::instances
      Takes a hash of instances. All the config variables that can be tuned in config can be passed along with each instance. if nothing is passed. the redis defaults from params and config are observed.

    - $::redis::provider
      Allows one to overload the puppet provider in case you use a different one from the default puppet provider

    - $::redis::params::sentinel_init_template
      template/redis-sentinel.init.redhat.erb
      start-stop-daemon is not a CentOS thing, so we need a Redhat/CentOS specific init script.
manifests/sentinel.pp
    - $::redis::params::sentinel_log_file
      Sentinel needs its own log file and should not write to the same one as redis.

    - $::redis::instances
      Sentinel is configured with multiple instances. the name of sentinel master is the name of the instance.

    - $::redis::sentinel::members
      $::redis::params::sentinel_discover_master
      $::redis::params::sentinel_members
      $::redis::params::sentinel_multiple_instances

      See, Feature: Discovered Masters
  • Loading branch information
Pankaj Kaushal committed Oct 17, 2016
1 parent 9711564 commit 62ebea6
Show file tree
Hide file tree
Showing 11 changed files with 591 additions and 85 deletions.
122 changes: 72 additions & 50 deletions manifests/config.pp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
$slave_read_only = $::redis::slave_read_only
$slave_serve_stale_data = $::redis::slave_serve_stale_data
$slaveof = $::redis::slaveof
$slaveof_port = $::redis::slaveof_port
$slowlog_log_slower_than = $::redis::slowlog_log_slower_than
$slowlog_max_len = $::redis::slowlog_max_len
$stop_writes_on_bgsave_error = $::redis::stop_writes_on_bgsave_error
Expand All @@ -66,64 +67,85 @@
$workdir = $::redis::workdir
$zset_max_ziplist_entries = $::redis::zset_max_ziplist_entries
$zset_max_ziplist_value = $::redis::zset_max_ziplist_value
$client_output_buffer_limit_normal = $::redis::client_output_buffer_limit_normal
$client_output_buffer_limit_slave = $::redis::client_output_buffer_limit_slave
$client_output_buffer_limit_pubsub = $::redis::client_output_buffer_limit_pubsub
$redis_binary_path = $::redis::redis_binary_path
$redis_binary_name = $::redis::redis_binary_name

if $::redis::notify_service {
File {
owner => $::redis::config_owner,
group => $::redis::config_group,
mode => $::redis::config_file_mode,
notify => Service[$::redis::service_name],
}
} else {
File {
owner => $::redis::config_owner,
group => $::redis::config_group,
mode => $::redis::config_file_mode,
}
if ($::redis::instances ) {
file {
$::redis::config_dir:
ensure => directory,
mode => $::redis::config_dir_mode;
$::redis::log_dir:
ensure => directory,
group => $::redis::service_group,
mode => $::redis::log_dir_mode,
owner => $::redis::service_user;
}
create_resources(::redis::instance, $::redis::instances, {})
}
else {
if $::redis::notify_service {
File {
owner => $::redis::config_owner,
group => $::redis::config_group,
mode => $::redis::config_file_mode,
notify => Service[$::redis::service_name]
}
} else {
File {
owner => $::redis::config_owner,
group => $::redis::config_group,
mode => $::redis::config_file_mode,
}
}

file {
$::redis::config_dir:
ensure => directory,
mode => $::redis::config_dir_mode;
file {
$::redis::config_dir:
ensure => directory,
mode => $::redis::config_dir_mode;

$::redis::config_file_orig:
ensure => present,
content => template($::redis::conf_template);
$::redis::config_file_orig:
ensure => present,
content => template($::redis::conf_template);

$::redis::log_dir:
ensure => directory,
group => $::redis::service_group,
mode => $::redis::log_dir_mode,
owner => $::redis::service_user;
}
$::redis::log_dir:
ensure => directory,
group => $::redis::service_group,
mode => $::redis::log_dir_mode,
owner => $::redis::service_user;
}

exec {
"cp -p ${::redis::config_file_orig} ${::redis::config_file}":
path => '/usr/bin:/bin',
subscribe => File[$::redis::config_file_orig],
refreshonly => true;
} ~> Service <| title == $::redis::service_name |>
exec {
"cp -p ${::redis::config_file_orig} ${::redis::config_file}":
path => '/usr/bin:/bin',
subscribe => File[$::redis::config_file_orig],
refreshonly => true;
} ~> Service <| title == $::redis::service_name |>

# Adjust /etc/default/redis-server on Debian systems
case $::osfamily {
'Debian': {
file { '/etc/default/redis-server':
ensure => present,
group => $::redis::config_group,
mode => $::redis::config_file_mode,
owner => $::redis::config_owner,
}
# Adjust /etc/default/redis-server on Debian systems
case $::osfamily {
'Debian': {
file { '/etc/default/redis-server':
ensure => present,
group => $::redis::config_group,
mode => $::redis::config_file_mode,
owner => $::redis::config_owner,
}

if $::redis::ulimit {
augeas { 'redis ulimit' :
context => '/files/etc/default/redis-server',
changes => "set ULIMIT ${::redis::ulimit}",
}
}
}
if $::redis::ulimit {
augeas { 'redis ulimit' :
context => '/files/etc/default/redis-server',
changes => "set ULIMIT ${::redis::ulimit}",
}
}
}

default: {
}
default: {
}
}
}
}

60 changes: 40 additions & 20 deletions manifests/init.pp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -525,6 +525,8 @@
$auto_aof_rewrite_percentage = $::redis::params::auto_aof_rewrite_percentage,
$bind = $::redis::params::bind,
$conf_template = $::redis::params::conf_template,
$redis_init_template = $::redis::params::redis_init_template,
$redis_init_file_mode = $::redis::params::redis_init_file_mode,
$config_dir = $::redis::params::config_dir,
$config_dir_mode = $::redis::params::config_dir_mode,
$config_file = $::redis::params::config_file,
Expand Down Expand Up @@ -586,6 +588,7 @@
$slave_read_only = $::redis::params::slave_read_only,
$slave_serve_stale_data = $::redis::params::slave_serve_stale_data,
$slaveof = $::redis::params::slaveof,
$slaveof_port = $::redis::params::slaveof_port,
$slowlog_log_slower_than = $::redis::params::slowlog_log_slower_than,
$slowlog_max_len = $::redis::params::slowlog_max_len,
$stop_writes_on_bgsave_error = $::redis::params::stop_writes_on_bgsave_error,
Expand All @@ -600,34 +603,51 @@
$workdir = $::redis::params::workdir,
$zset_max_ziplist_entries = $::redis::params::zset_max_ziplist_entries,
$zset_max_ziplist_value = $::redis::params::zset_max_ziplist_value,
$client_output_buffer_limit_normal = $::redis::params::client_output_buffer_limit_normal,
$client_output_buffer_limit_slave = $::redis::params::client_output_buffer_limit_slave,
$client_output_buffer_limit_pubsub = $::redis::params::client_output_buffer_limit_pubsub,
$cluster_enabled = $::redis::params::cluster_enabled,
$cluster_config_file = $::redis::params::cluster_config_file,
$cluster_node_timeout = $::redis::params::cluster_node_timeout,
$instances = $::redis::params::instances,
$redis_binary_path = $::redis::params::redis_binary_path,
$redis_binary_name = $::redis::params::redis_binary_name,


) inherits redis::params {
anchor { 'redis::begin': }
anchor { 'redis::end': }

include ::redis::preinstall
include ::redis::install
include ::redis::config
include ::redis::service

if $::redis::notify_service {
Anchor['redis::begin'] ->
Class['redis::preinstall'] ->
Class['redis::install'] ->
Class['redis::config'] ~>
Class['redis::service'] ->
Anchor['redis::end']
if $::redis::instances {
include ::redis::preinstall
include ::redis::install
include ::redis::config
Anchor['redis::begin'] ->
Class['redis::preinstall'] ->
Class['redis::install'] ->
Class['redis::config'] ~>
Anchor['redis::end']
} else {
Anchor['redis::begin'] ->
Class['redis::preinstall'] ->
Class['redis::install'] ->
Class['redis::config'] ->
Class['redis::service'] ->
Anchor['redis::end']
}

include ::redis::preinstall
include ::redis::install
include ::redis::config
include ::redis::service
if $::redis::notify_service {
Anchor['redis::begin'] ->
Class['redis::preinstall'] ->
Class['redis::install'] ->
Class['redis::config'] ~>
Class['redis::service'] ~>
Anchor['redis::end']
} else {
Anchor['redis::begin'] ->
Class['redis::preinstall'] ->
Class['redis::install'] ->
Class['redis::config'] ->
Class['redis::service'] ->
Anchor['redis::end']
}
}
# Sanity check
if $::redis::slaveof {
if $::redis::bind =~ /^127.0.0./ {
Expand Down
5 changes: 3 additions & 2 deletions manifests/install.pp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@
# This class installs the application.
#
class redis::install {
unless defined(Package[$::redis::package_name]) {
unless defined(Package['$::redis::package_name']) {
ensure_resource('package', $::redis::package_name, {
'ensure' => $::redis::package_ensure
'ensure' => $::redis::package_ensure,
'provider' => $::redis::provider,
})
}
}
Expand Down
Loading

0 comments on commit 62ebea6

Please sign in to comment.