Skip to content

Latest commit

 

History

History
1737 lines (984 loc) · 42.5 KB

REFERENCE.md

File metadata and controls

1737 lines (984 loc) · 42.5 KB

Reference

Table of Contents

Classes

Public Classes

Private Classes

  • kafka::broker::config: This class handles the Kafka (broker) config.
  • kafka::broker::install: This class handles the Kafka (broker) package.
  • kafka::broker::service: This class handles the Kafka (broker) service.
  • kafka::consumer::config: This class handles the Kafka (consumer) config.
  • kafka::consumer::install: This class handles the Kafka (consumer) package.
  • kafka::consumer::service: This class handles the Kafka (consumer) service.
  • kafka::mirror::config: This class handles the Kafka (mirror) config.
  • kafka::mirror::install: This class handles the Kafka (mirror) package.
  • kafka::mirror::service: This class handles the Kafka (mirror) service.
  • kafka::producer::config: This class handles the Kafka (producer) config.
  • kafka::producer::install: This class handles the Kafka (producer) package.
  • kafka::producer::service: This class handles the Kafka (producer) service.

Defined types

  • kafka::topic: This defined type handles the creation of Kafka topics.

Classes

kafka

This class handles the Kafka requirements.

Examples

Basic usage
class { 'kafka': }

Parameters

The following parameters are available in the kafka class:

kafka_version

Data type: String[1]

The version of Kafka that should be installed.

Default value: $kafka::params::kafka_version

scala_version

Data type: String[1]

The scala version what Kafka was built with.

Default value: $kafka::params::scala_version

install_dir

Data type: Stdlib::Absolutepath

The directory to install Kafka to.

Default value: $kafka::params::install_dir

mirror_url

Data type: Stdlib::HTTPUrl

The url where the Kafka is downloaded from.

Default value: $kafka::params::mirror_url

manage_java

Data type: Boolean

Install java if it's not already installed.

Default value: $kafka::params::manage_java

package_dir

Data type: Stdlib::Absolutepath

The directory to install Kafka.

Default value: $kafka::params::package_dir

package_name

Data type: Optional[String[1]]

Package name, when installing Kafka from a package.

Default value: $kafka::params::package_name

mirror_subpath

Data type: Optional[String[1]]

The sub directory where the source is downloaded from.

Default value: $kafka::params::mirror_subpath

proxy_server

Data type: Optional[String[1]]

Set proxy server, when installing Kafka from source.

Default value: $kafka::params::proxy_server

proxy_port

Data type: Optional[String[1]]

Set proxy port, when installing Kafka from source.

Default value: $kafka::params::proxy_port

proxy_host

Data type: Optional[String[1]]

Set proxy host, when installing Kafka from source.

Default value: $kafka::params::proxy_host

proxy_type

Data type: Optional[String[1]]

Set proxy type, when installing Kafka from source.

Default value: $kafka::params::proxy_type

package_ensure

Data type: String[1]

Package version or ensure state, when installing Kafka from a package.

Default value: $kafka::params::package_ensure

user_name

Data type: String[1]

User to run Kafka as.

Default value: $kafka::params::user_name

user_shell

Data type: Stdlib::Absolutepath

Login shell of the Kafka user.

Default value: $kafka::params::user_shell

group_name

Data type: String[1]

Group to run Kafka as.

Default value: $kafka::params::group_name

system_user

Data type: Boolean

Whether the Kafka user is a system user or not.

Default value: $kafka::params::system_user

system_group

Data type: Boolean

Whether the Kafka group is a system group or not.

Default value: $kafka::params::system_group

user_id

Data type: Optional[Integer]

Create the Kafka user with this ID.

Default value: $kafka::params::user_id

group_id

Data type: Optional[Integer]

Create the Kafka group with this ID.

Default value: $kafka::params::group_id

manage_user

Data type: Boolean

Create the Kafka user if it's not already present.

Default value: $kafka::params::manage_user

manage_group

Data type: Boolean

Create the Kafka group if it's not already present.

Default value: $kafka::params::manage_group

config_dir

Data type: Stdlib::Absolutepath

The directory to create the Kafka config files to.

Default value: $kafka::params::config_dir

log_dir

Data type: Stdlib::Absolutepath

The directory for Kafka log files.

Default value: $kafka::params::log_dir

install_mode

Data type: Stdlib::Filemode

The permissions for the install directory.

Default value: $kafka::params::install_mode

kafka::broker

This class handles the Kafka (broker).

Examples

Basic usage
class { 'kafka::broker':
  config => {
    'broker.id'         => '0',
    'zookeeper.connect' => 'localhost:2181'
  }
}

Parameters

The following parameters are available in the kafka::broker class:

kafka_version

Data type: String[1]

The version of Kafka that should be installed.

Default value: $kafka::params::kafka_version

scala_version

Data type: String[1]

The scala version what Kafka was built with.

Default value: $kafka::params::scala_version

install_dir

Data type: Stdlib::Absolutepath

The directory to install Kafka to.

Default value: $kafka::params::install_dir

mirror_url

Data type: Stdlib::HTTPUrl

The url where the Kafka is downloaded from.

Default value: $kafka::params::mirror_url

manage_java

Data type: Boolean

Install java if it's not already installed.

Default value: $kafka::params::manage_java

package_dir

Data type: Stdlib::Absolutepath

The directory to install Kafka.

Default value: $kafka::params::package_dir

package_name

Data type: Optional[String[1]]

Package name, when installing Kafka from a package.

Default value: $kafka::params::package_name

package_ensure

Data type: String[1]

Package version or ensure state, when installing Kafka from a package.

Default value: $kafka::params::package_ensure

user_name

Data type: String[1]

User to run Kafka as.

Default value: $kafka::params::user_name

user_shell

Data type: Stdlib::Absolutepath

Login shell of the Kafka user.

Default value: $kafka::params::user_shell

group_name

Data type: String[1]

Group to run Kafka as.

Default value: $kafka::params::group_name

user_id

Data type: Optional[Integer]

Create the Kafka user with this ID.

Default value: $kafka::params::user_id

group_id

Data type: Optional[Integer]

Create the Kafka group with this ID.

Default value: $kafka::params::group_id

manage_user

Data type: Boolean

Create the Kafka user if it's not already present.

Default value: $kafka::params::manage_user

manage_group

Data type: Boolean

Create the Kafka group if it's not already present.

Default value: $kafka::params::manage_group

config_mode

Data type: Stdlib::Filemode

The permissions for the config files.

Default value: $kafka::params::config_mode

config_dir

Data type: Stdlib::Absolutepath

The directory to create the Kafka config files to.

Default value: $kafka::params::config_dir

log_dir

Data type: Stdlib::Absolutepath

The directory for Kafka log files.

Default value: $kafka::params::log_dir

bin_dir

Data type: Stdlib::Absolutepath

The directory where the Kafka scripts are.

Default value: $kafka::params::bin_dir

service_name

Data type: String[1]

Set the name of the service.

Default value: 'kafka'

manage_service

Data type: Boolean

Install the systemd service.

Default value: $kafka::params::manage_service

service_ensure

Data type: Enum['running', 'stopped']

Set the ensure state of the service.

Default value: $kafka::params::service_ensure

service_restart

Data type: Boolean

Whether the configuration files should trigger a service restart.

Default value: $kafka::params::service_restart

service_requires

Data type: Array[String[1]]

Set the list of services required to be running before Kafka.

Default value: $kafka::params::service_requires

limit_nofile

Data type: Optional[String[1]]

Set the 'LimitNOFILE' option of the systemd service.

Default value: $kafka::params::limit_nofile

limit_core

Data type: Optional[String[1]]

Set the 'LimitCORE' option of the systemd service.

Default value: $kafka::params::limit_core

timeout_stop

Data type: Optional[String[1]]

Set the 'TimeoutStopSec' option of the systemd service.

Default value: $kafka::params::timeout_stop

exec_stop

Data type: Boolean

Set the 'ExecStop' option of the systemd service to 'kafka-server-stop.sh'.

Default value: $kafka::params::exec_stop

daemon_start

Data type: Boolean

Use the '-daemon' option when starting Kafka with 'kafka-server-start.sh'.

Default value: $kafka::params::daemon_start

env

Data type: Hash

A hash of the environment variables to set.

Default value: {}

config

Data type: Hash[String[1], Any]

A hash of the broker configuration options.

Default value: {}

heap_opts

Data type: String[1]

Set the Java heap size.

Default value: $kafka::params::broker_heap_opts

jmx_opts

Data type: String[1]

Set the JMX options.

Default value: $kafka::params::broker_jmx_opts

log4j_opts

Data type: String[1]

Set the Log4j options.

Default value: $kafka::params::broker_log4j_opts

manage_log4j

Data type: Boolean

Enable or disable the management of the log4j configuration file.

Default value: $kafka::params::manage_log4j

log4j_content

Data type: Optional[String[1]]

Provide custom content for the log4j configuration file if manage_log4j is enabled, instead of using the content provided in this module.

Default value: undef

log_file_size

Data type: Pattern[/[1-9][0-9]*[KMG]B/]

Set the max size of the Kafka log files before they are rolled.

Default value: $kafka::params::log_file_size

log_file_count

Data type: Integer[1, 50]

Set the number of Kafka log files to keep.

Default value: $kafka::params::log_file_count

opts

Data type: String[0]

Set the Kafka options.

Default value: $kafka::params::broker_opts

kafka::consumer

This class handles the Kafka (consumer).

Examples

Basic usage
class { 'kafka::consumer':
  config => {
    'client.id'         => '0',
    'zookeeper.connect' => 'localhost:2181'
  }
}

Parameters

The following parameters are available in the kafka::consumer class:

kafka_version

Data type: String[1]

The version of Kafka that should be installed.

Default value: $kafka::params::kafka_version

scala_version

Data type: String[1]

The scala version what Kafka was built with.

Default value: $kafka::params::scala_version

install_dir

Data type: Stdlib::Absolutepath

The directory to install Kafka to.

Default value: $kafka::params::install_dir

mirror_url

Data type: Stdlib::HTTPUrl

The url where the Kafka is downloaded from.

Default value: $kafka::params::mirror_url

manage_java

Data type: Boolean

Install java if it's not already installed.

Default value: $kafka::params::manage_java

package_dir

Data type: Stdlib::Absolutepath

The directory to install Kafka.

Default value: $kafka::params::package_dir

package_name

Data type: Optional[String[1]]

Package name, when installing Kafka from a package.

Default value: $kafka::params::package_name

package_ensure

Data type: String[1]

Package version or ensure state, when installing Kafka from a package.

Default value: $kafka::params::package_ensure

user_name

Data type: String[1]

User to run Kafka as.

Default value: $kafka::params::user_name

user_shell

Data type: Stdlib::Absolutepath

Login shell of the Kafka user.

Default value: $kafka::params::user_shell

group_name

Data type: String[1]

Group to run Kafka as.

Default value: $kafka::params::group_name

user_id

Data type: Optional[Integer]

Create the Kafka user with this ID.

Default value: $kafka::params::user_id

group_id

Data type: Optional[Integer]

Create the Kafka group with this ID.

Default value: $kafka::params::group_id

manage_user

Data type: Boolean

Create the Kafka user if it's not already present.

Default value: $kafka::params::manage_user

manage_group

Data type: Boolean

Create the Kafka group if it's not already present.

Default value: $kafka::params::manage_group

config_mode

Data type: Stdlib::Filemode

The permissions for the config files.

Default value: $kafka::params::config_mode

config_dir

Data type: Stdlib::Absolutepath

The directory to create the Kafka config files to.

Default value: $kafka::params::config_dir

log_dir

Data type: Stdlib::Absolutepath

The directory for Kafka log files.

Default value: $kafka::params::log_dir

bin_dir

Data type: Stdlib::Absolutepath

The directory where the Kafka scripts are.

Default value: $kafka::params::bin_dir

service_name

Data type: String[1]

Set the name of the service.

Default value: 'kafka-consumer'

manage_service

Data type: Boolean

Install the systemd service.

Default value: $kafka::params::manage_service

service_ensure

Data type: Enum['running', 'stopped']

Set the ensure state of the service.

Default value: $kafka::params::service_ensure

service_restart

Data type: Boolean

Whether the configuration files should trigger a service restart.

Default value: $kafka::params::service_restart

service_requires

Data type: Array[String[1]]

Set the list of services required to be running before Kafka.

Default value: $kafka::params::service_requires

limit_nofile

Data type: Optional[String[1]]

Set the 'LimitNOFILE' option of the systemd service.

Default value: $kafka::params::limit_nofile

limit_core

Data type: Optional[String[1]]

Set the 'LimitCORE' option of the systemd service.

Default value: $kafka::params::limit_core

env

Data type: Hash

A hash of the environment variables to set.

Default value: {}

config

Data type: Hash[String[1], Any]

A hash of the consumer configuration options.

Default value: {}

service_config

Data type: Hash[String[1],String[1]]

A hash of the kafka-console-consumer.sh script options.

Default value: {}

jmx_opts

Data type: String[1]

Set the JMX options.

Default value: $kafka::params::consumer_jmx_opts

log4j_opts

Data type: String[1]

Set the Log4j options.

Default value: $kafka::params::consumer_log4j_opts

manage_log4j

Data type: Boolean

Enable or disable the management of the log4j configuration file.

Default value: $kafka::params::manage_log4j

log_file_size

Data type: Pattern[/[1-9][0-9]*[KMG]B/]

Set the max size of the Kafka log files before they are rolled.

Default value: $kafka::params::log_file_size

log_file_count

Data type: Integer[1, 50]

Set the number of Kafka log files to keep.

Default value: $kafka::params::log_file_count

kafka::mirror

This class handles the Kafka (mirror).

Examples

Basic usage
class { 'kafka::mirror':
  consumer_config => {
    'group.id'          => 'kafka-mirror',
    'zookeeper.connect' => 'localhost:2181'
  },
  producer_config => {
    'zookeeper.connect' => 'localhost:2181',
  },
  service_config => {
    'whitelist'         => '.*',
  }
}

Parameters

The following parameters are available in the kafka::mirror class:

kafka_version

Data type: String[1]

The version of Kafka that should be installed.

Default value: $kafka::params::kafka_version

scala_version

Data type: String[1]

The scala version what Kafka was built with.

Default value: $kafka::params::scala_version

install_dir

Data type: Stdlib::Absolutepath

The directory to install Kafka to.

Default value: $kafka::params::install_dir

mirror_url

Data type: Stdlib::HTTPUrl

The url where the Kafka is downloaded from.

Default value: $kafka::params::mirror_url

manage_java

Data type: Boolean

Install java if it's not already installed.

Default value: $kafka::params::manage_java

package_dir

Data type: Stdlib::Absolutepath

The directory to install Kafka.

Default value: $kafka::params::package_dir

package_name

Data type: Optional[String[1]]

Package name, when installing Kafka from a package.

Default value: $kafka::params::package_name

package_ensure

Data type: String[1]

Package version or ensure state, when installing Kafka from a package.

Default value: $kafka::params::package_ensure

user_name

Data type: String[1]

User to run Kafka as.

Default value: $kafka::params::user_name

user_shell

Data type: Stdlib::Absolutepath

Login shell of the Kafka user.

Default value: $kafka::params::user_shell

group_name

Data type: String[1]

Group to run Kafka as.

Default value: $kafka::params::group_name

user_id

Data type: Optional[Integer]

Create the Kafka user with this ID.

Default value: $kafka::params::user_id

group_id

Data type: Optional[Integer]

Create the Kafka group with this ID.

Default value: $kafka::params::group_id

manage_user

Data type: Boolean

Create the Kafka user if it's not already present.

Default value: $kafka::params::manage_user

manage_group

Data type: Boolean

Create the Kafka group if it's not already present.

Default value: $kafka::params::manage_group

config_mode

Data type: Stdlib::Filemode

The permissions for the config files.

Default value: $kafka::params::config_mode

config_dir

Data type: Stdlib::Absolutepath

The directory to create the Kafka config files to.

Default value: $kafka::params::config_dir

log_dir

Data type: Stdlib::Absolutepath

The directory for Kafka log files.

Default value: $kafka::params::log_dir

bin_dir

Data type: Stdlib::Absolutepath

The directory where the Kafka scripts are.

Default value: $kafka::params::bin_dir

service_name

Data type: String[1]

Set the name of the service.

Default value: 'kafka-mirror'

manage_service

Data type: Boolean

Install the systemd service.

Default value: $kafka::params::manage_service

service_ensure

Data type: Enum['running', 'stopped']

Set the ensure state of the service.

Default value: $kafka::params::service_ensure

service_restart

Data type: Boolean

Whether the configuration files should trigger a service restart.

Default value: $kafka::params::service_restart

service_requires

Data type: Array[String[1]]

Set the list of services required to be running before Kafka.

Default value: $kafka::params::service_requires

limit_nofile

Data type: Optional[String[1]]

Set the 'LimitNOFILE' option of the systemd service.

Default value: $kafka::params::limit_nofile

limit_core

Data type: Optional[String[1]]

Set the 'LimitCORE' option of the systemd service.

Default value: $kafka::params::limit_core

env

Data type: Hash

A hash of the environment variables to set.

Default value: {}

consumer_config

Data type: Hash[String[1],String[1]]

A hash of the consumer configuration options.

Default value: {}

producer_config

Data type: Hash[String[1],String[1]]

A hash of the producer configuration options.

Default value: {}

service_config

Data type: Hash[String[1],String[1]]

A hash of the mirror script options.

Default value: {}

heap_opts

Data type: String[1]

Set the Java heap size.

Default value: $kafka::params::mirror_heap_opts

jmx_opts

Data type: String[1]

Set the JMX options.

Default value: $kafka::params::mirror_jmx_opts

log4j_opts

Data type: String[1]

Set the Log4j options.

Default value: $kafka::params::mirror_log4j_opts

manage_log4j

Data type: Boolean

Enable or disable the management of the log4j configuration file.

Default value: $kafka::params::manage_log4j

log_file_size

Data type: Pattern[/[1-9][0-9]*[KMG]B/]

Set the max size of the Kafka log files before they are rolled.

Default value: $kafka::params::log_file_size

log_file_count

Data type: Integer[1, 50]

Set the number of Kafka log files to keep.

Default value: $kafka::params::log_file_count

kafka::params

This class provides default parameters.

kafka::producer

This class handles the Kafka (producer).

Examples

Basic usage
class { 'kafka::producer':
  config => {
    'client.id'         => '0',
    'zookeeper.connect' => 'localhost:2181'
  }
}

Parameters

The following parameters are available in the kafka::producer class:

input

Data type: Optional[String[1]]

Set named pipe as input.

kafka_version

Data type: String[1]

The version of Kafka that should be installed.

Default value: $kafka::params::kafka_version

scala_version

Data type: String[1]

The scala version what Kafka was built with.

Default value: $kafka::params::scala_version

install_dir

Data type: Stdlib::Absolutepath

The directory to install Kafka to.

Default value: $kafka::params::install_dir

mirror_url

Data type: Stdlib::HTTPUrl

The url where the Kafka is downloaded from.

Default value: $kafka::params::mirror_url

manage_java

Data type: Boolean

Install java if it's not already installed.

Default value: $kafka::params::manage_java

package_dir

Data type: Stdlib::Absolutepath

The directory to install Kafka.

Default value: $kafka::params::package_dir

package_name

Data type: Optional[String[1]]

Package name, when installing Kafka from a package.

Default value: $kafka::params::package_name

package_ensure

Data type: String[1]

Package version or ensure state, when installing Kafka from a package.

Default value: $kafka::params::package_ensure

user_name

Data type: String[1]

User to run Kafka as.

Default value: $kafka::params::user_name

user_shell

Data type: Stdlib::Absolutepath

Login shell of the Kafka user.

Default value: $kafka::params::user_shell

group_name

Data type: String[1]

Group to run Kafka as.

Default value: $kafka::params::group_name

user_id

Data type: Optional[Integer]

Create the Kafka user with this ID.

Default value: $kafka::params::user_id

group_id

Data type: Optional[Integer]

Create the Kafka group with this ID.

Default value: $kafka::params::group_id

manage_user

Data type: Boolean

Create the Kafka user if it's not already present.

Default value: $kafka::params::manage_user

manage_group

Data type: Boolean

Create the Kafka group if it's not already present.

Default value: $kafka::params::manage_group

config_mode

Data type: Stdlib::Filemode

The permissions for the config files.

Default value: $kafka::params::config_mode

config_dir

Data type: Stdlib::Absolutepath

The directory to create the Kafka config files to.

Default value: $kafka::params::config_dir

log_dir

Data type: Stdlib::Absolutepath

The directory for Kafka log files.

Default value: $kafka::params::log_dir

bin_dir

Data type: Stdlib::Absolutepath

The directory where the Kafka scripts are.

Default value: $kafka::params::bin_dir

service_name

Data type: String[1]

Set the name of the service.

Default value: 'kafka-producer'

manage_service

Data type: Boolean

Install systemd service.

Default value: $kafka::params::manage_service

service_ensure

Data type: Enum['running', 'stopped']

Set the ensure state of the service.

Default value: $kafka::params::service_ensure

service_restart

Data type: Boolean

Whether the configuration files should trigger a service restart.

Default value: $kafka::params::service_restart

service_requires

Data type: Array[String[1]]

Set the list of services required to be running before Kafka.

Default value: $kafka::params::service_requires

limit_nofile

Data type: Optional[String[1]]

Set the 'LimitNOFILE' option of the systemd service.

Default value: $kafka::params::limit_nofile

limit_core

Data type: Optional[String[1]]

Set the 'LimitCORE' option of the systemd service.

Default value: $kafka::params::limit_core

env

Data type: Hash

A hash of the environment variables to set.

Default value: {}

config

Data type: Hash[String[1], Any]

A hash of the producer configuration options.

Default value: {}

service_config

Data type: Hash[String[1],String[1]]

A hash of the kafka-console-producer.sh script options.

Default value: {}

jmx_opts

Data type: String[1]

Set the JMX options.

Default value: $kafka::params::producer_jmx_opts

log4j_opts

Data type: String[1]

Set the Log4j options.

Default value: $kafka::params::producer_log4j_opts

Defined types

kafka::topic

This defined type handles the creation of Kafka topics.

Examples

Basic usage
kafka::topic { 'test':
  ensure             => present,
  zookeeper          => 'localhost:2181',
  replication_factor => 1,
  partitions         => 1,
}

Parameters

The following parameters are available in the kafka::topic defined type:

ensure

Data type: Optional[String[1]]

Should the topic be created.

Default value: undef

zookeeper

Data type: Optional[String[1]]

The connection string for the ZooKeeper connection in the form host:port. Multiple hosts can be given to allow fail-over. Kafka < 3.0.0 only!

Default value: undef

bootstrap_server

Data type: Optional[String[1]]

The Kafka server to connect to in the form host:port. Kafka >= 2.2.0 only!

Default value: undef

replication_factor

Data type: Integer

The replication factor for each partition in the topic being created. If not supplied, defaults to the cluster default.

Default value: 1

partitions

Data type: Integer

The number of partitions for the topic being created or altered. If not supplied for create, defaults to the cluster default.

Default value: 1

bin_dir

Data type: String[1]

The directory where the file kafka-topics.sh is located.

Default value: '/opt/kafka/bin'

config

Data type: Optional[Hash[String[1],String[1]]]

A topic configuration override for the topic being created or altered. See the Kafka documentation for full details on the topic configs.

Default value: undef