Skip to content

@jfriesse jfriesse released this Mar 5, 2019 · 23 commits to master since this release

I am pleased to announce the latest maintenance release of Corosync
3.0.1 available immediately from our website at
http://build.clusterlabs.org/corosync/releases/.

This release contains important fix for hostname resolving which should now work much better.

Complete changelog for 3.0.1:

Fabio M. Di Nitto (1):
      [totemknet] update for libknet.so.2.0.0 init API

Ferenc Wágner (3):
      More natural error messages
      Don't declare success early
      Config version must be specified

Jan Friesse (1):
      totemip: Use AF_UNSPEC for ipv4-6 and ipv6-4

Jan Pokorný (1):
      init: Enable StopWhenUnneeded

Upgrade is highly recommended.

Thanks/congratulations to all people that contributed to achieve this
great milestone.

Assets 6

@jfriesse jfriesse released this Mar 5, 2019 · 29 commits to master since this release

I am pleased to announce the first stable release of Corosync 3.0 (codename Camelback) branch available immediately from our website at
http://build.clusterlabs.org/corosync/releases/.

You can also download RPMs for various distributions from CI https://kronosnet.org/builds/.

Corosync 3.0 is result of almost 2.5 years (counting just Corosync, there is also endless amount of work made by Kronosnet developers) of work (needle and master branch split Aug 30 2016) of almost 30 (29 to be precise ) people who created almost 300 (exactly 297) commits (we are quite conservative and usually have only one commit per feature). Complete git stats: 383 files changed, 12156 insertions(+), 45369 deletions(-). It's quite impressive how much code we were able to remove.

Corosync 3.0 contains many interesting features mostly related to usage of Kronosnet (https://kronosnet.org/) as a default (and preferred) network transport.

List of high-level changes:

  • Knet

    • Support for 8 links which can be dynamically reconfigured without restart of corosync daemon
    • MTU auto-configuration
    • Support for NSS or OpenSSL encryption of packets
    • Compression
    • Higher throughput and lower latency (yes, it's really possible to get both of these at once )
    • And many more
  • Support for RDMA and Upstart is gone

  • Enhanced statistics

  • Nodelist is now mandatory

  • UDP/UDPU transports are still present, but supports only single ring (RRP is gone in favor of Knet) and doesn't support encryption

  • Support for systemd startup notifications

  • Corosync-qdevice found new home at https://github.com/corosync/corosync-qdevice and is no longer part of corosync repository/packages. Corosync-qdevice 3.0.0 was released this week on Wed.

  • With Knet transport it's possible to remove ring0_addr from corosync.conf and as long as there is other ring (ring1_addr), everything should work as expected

  • Corosync 3 is not wire compatible with previous versions. Needle was patched so it will display warning message when receive corosync 3 packet (and vice-versa).

  • CTS was removed - it was unmaintained for long time and unused by developers, currently without replacement.

  • libtotem is no longer shared library and it's directly compiled into
    corosync binary - main idea of having libtotem.so was to allow other projects usage of libtotem and build custom "corosync" on top of it. This idea actually never got expected usage and because it was super big, it was just making corosync development much harder. This doesn't affect any corosync user. In future totemsrp.c should be made into real well testable library without network protocol handling, ... but right now, there is no replacement for libtotem.so.

  • All environment variables were removed and corosync arguments were tied uo:

    • -p, -P, -R and -r options - replaced by system.sched_rr, system.priority and system.move_to_root_cgroup options in the config file.
    • env COROSYNC_MAIN_CONFIG_FILE - Replaced by "-c" option. This also affects uidgid.d location.
    • env COROSYNC_TOTEM_AUTHKEY_FILE - Replaced by (already existing) totem.keyfile config option, which is now documented.
    • env COROSYNC_RUN_DIR - Replaced by system.state_dir and documented.
  • Usage of libcgroup removal - deprecated in most of new distributions, replaced by "short" code with even functionality.

  • NSS dependency removal - not needed anymore because crypto is now handled by knet, so no replacement needed. This change affects only cpgverify where packet format is changed.

  • Corosync config file parser updated so it's now more strict and (finally) displays line with error. Affects only broken config files.

  • With new enough LibQB, it's possible to send command to corosync and it tries reopen its log files. Used by logrotate in favor of old copytruncate method. Copytruncate still exists and is compiled/installed by default with old LibQB.

  • Timestamps are now enabled by default. With new enough LibQB hires (including milliseconds) timestamps are used by default.

  • ip_version config setting has new default ipv6-4 (resolve IPv6 address first, if it fails try IPv4). For more info please consult corosync.conf(5) man page. To achieve old behavior (IPv4 only) please set totem.ip_version setting to ipv4.

  • And finally - many people beloved way to test node split - ifdown - now finally works with Knet transport. UDPU transport was also partly fixed. Please keep in mind this method of testing is still not supported nor recommended.

For presentation about corosync/knet please see http://build.clusterlabs.org/corosync/presentations/2017-Kronosnet-The-new-face-of-corosync-communications.pdf.

For more in depth document about how to configure/use corosync 3 with knet see http://people.redhat.com/ccaulfie/docs/KnetCorosync.pdf

We did our best to fix all the know issues, and we believe this is the best Corosync release ever, but if you find some issues, please let us know.

Thanks/congratulations to all people that contributed to achieve this
great milestone.

Assets 6

@jfriesse jfriesse released this Mar 5, 2019 · 1968 commits to master since this release

I am pleased to announce the last maintenance of old stable (Flatiron branch) release of Corosync
1.4.10 available immediately from github releases.

Flatiron branch is now unsupported and no longer maintained by Corosync team. The main reason for this decision is that Camelback (v3.0.0) should be released on Thu this week and having 3 releases is simply too much.

Just a short statistics/:

  • Version 1.0.0 was released on Wed Jul 8 2009 so Flatiron was supported for a little more than 9 years
  • There were 612 commits made by 25 people during Flatiron lifetime
  • 224 files changed, 25830 insertions(+), 45350 deletions(-)
  • This version began it's life on fedorahosted (decommissioned) and source code repository was SVN
  • Flatiron was last release with LCR supported

Complete changelog for 1.4.10 (compared to v1.4.9):

Jan Friesse (1):
      lcr: Fix memleak in interface_find_and_load

If your deployment still run on top of Corosync 1.x it's really highly recommended to upgrade.

Thanks/congratulations to all people that contributed to achieve this
great milestone.

Assets 6
Pre-release

@jfriesse jfriesse released this Mar 6, 2019 · 37 commits to master since this release

I am pleased to announce the first beta version of Corosync 3.0 (codename Camelback) available immediately from our website at
http://build.clusterlabs.org/corosync/releases/ as corosync-2.99.5.

You can also download RPMs for various distributions from CI https://kronosnet.org/builds/.

This release is really nicely small so it's probably last version before GA.

Complete changelog for Beta 1 (compared to Alpha 5):

Christine Caulfield (3):
      config: Report IP addr/nodename parse errors back
      config: Disallow corosync-cmapctl updates of nodelist
      man: Add instructions for adding/removing nodes

Jan Friesse (3):
      testcpg2: Check cpg_dispatch return code
      cpghum: Check cpg_local_get return code
      coroparse: Remove unused cs_err initialization

We did our best to test this release as best as we could, but still take it as an Beta version.

For anybody who migrates from stable version please consult http://people.redhat.com/ccaulfie/docs/KnetCorosync.pdf about changes in Corosync 3.

Thanks/congratulations to all people that contributed to achieve this
great milestone.

Assets 6
Pre-release

@jfriesse jfriesse released this Mar 6, 2019 · 43 commits to master since this release

I am pleased to announce the fifth testing (Alpha 5) release of Corosync 3.0 (codename Camelback) available immediately from our website at
http://build.clusterlabs.org/corosync/releases/ as corosync-2.99.4.

You can also download RPMs for various distributions from CI https://kronosnet.org/builds/.

This release turns out to be quite bigger than I've expected, so that's the reason why it's not Beta/RC yet. It also contains quite a lot of removal of unused/unmaintained code.

List of biggest changes and backwards compatibility breakages with reasoning and replacement (if needed):

  • Removal of CTS - unmaintained for long time and unused by developers, currently without replacement.

  • libtotem is no longer shared library and it's directly compiled into corosync binary - main idea of having libtotem.so was to allow other projects usage of libtotem and build custom "corosync" on top of it. This idea actually never got expected usage and because it was super big, it was just making corosync development much harder. This doesn't affect any corosync user. In future totemsrp.c should be made into real well testable library without network protocol handling, ... but right now, there is no replacement for libtotem.so.

  • Remove usage of all environment variables and tied up corosync arguments:

    • -p, -P, -R and -r options - replaced by system.sched_rr, system.priority and system.move_to_root_cgroup options in the config file.
    • env COROSYNC_MAIN_CONFIG_FILE - Replaced by "-c" option. This also affects uidgid.d location.
    • env COROSYNC_TOTEM_AUTHKEY_FILE - Replaced by (already existing) totem.keyfile config option, which is now documented.
    • env COROSYNC_RUN_DIR - Replaced by system.run_dir and documented.
  • Usage of libcgroup removal - deprecated in most of new distributions, replaced by "short" code with even functionality.

  • NSS dependency removal - not needed anymore because crypto is now handled by knet, so no replacement needed. This change affects only cpgverify where packet format is changed.

  • Corosync config file parser updated so it's now more strict and (finally) displays line with error. Affects only broken config files.

  • With new enough LibQB, it's possible to send command to corosync and it tries reopen its log files. Used by logrotate in favor of old copytruncate method. Copytruncate still exists and is compiled/installed by default with old LibQB.

  • Timestamps are now enabled by default. With new enough LibQB hires (including milliseconds) timestamps are used by default.

Complete changelog for Alpha 5 (compared to Alpha 4):

Chris Walker (3):
      Add option for quiet operation to corosync-cmapctl
      Add token_warning configuration option
      Add option to force cluster into GATHER state

Christine Caulfield (2):
      config: Fix crash in reload if new interfaces are added
      config: Allow generated nodeis for UDP & UDPU

Ferenc Wágner (3):
      man: fix cmap key name runtime.config.totem.token
      man: Fix typo connnections -> connections
      man: Fix typo conains -> contains

Jan Friesse (40):
      spec: Add explicit gcc build requirement
      totemknet: Free instance on failure exit
      util: Fix strncpy in setcs_name_t function
      cmap: Fix strncpy warning in cmap_iter_next
      ipc_glue: Fix strncpy in pid_to_name function
      totemconfig: Enlarge error_string_response
      totemsrp: Add assert into memb_lowest_in_config
      corosync-notifyd: Rename global local_nodeid
      Remove libcgroup
      build: Support for git archive stored tags
      git-version-gen: Fail on UNKNOWN version
      notifyd: Propagate error to exit code
      coroparse: Return error if config line is too long
      coroparse: Check icmap_set results
      coroparse: Fix remove_whitespace end condition
      coroparse: Be more strict in what is parsed
      coroparse: Add file name and line to error message
      coroparse: Use key_name for error message
      coroparse: Fix newly introduced warning
      man: Fix crypto_hash and crypto_cipher defaults
      cts: Remove CTS
      build: Remove NSS dependencies
      build: Do not compile totempg as a shared library
      build: Remove totempg shared library leftovers
      man: Fix default knet_pmtud_interval to match code
      totemconfig: Replace strcpy by strncpy
      log: Implement support for reopening log files
      config example: Migrate to newer syntax
      totemconfig: Fix logging of freed string
      logsys: Support hires timestamp
      logsys: Make hires timestamp default
      configure: move to AC_COMPILE_IFELSE
      main: Move sched paramaters to config file
      main: Replace COROSYNC_MAIN_CONFIG_FILE
      main: Remove COROSYNC_TOTEM_AUTHKEY_FILE
      man: Describe nodelist.node.name properly
      main: Remove COROSYNC_RUN_DIR
      init: Fix init script to work with containers
      stats: Fix delete of track
      notifyd: Delete registered tracking keys

Jan Pokorný (1):
      systemd: prevent redundancy in journal

We did our best to test this release as best as we could, but still take it as an Alpha version .

For anybody who migrates from stable version please consult http://people.redhat.com/ccaulfie/docs/KnetCorosync.pdf about changes in Corosync 3.

Thanks/congratulations to all people that contributed to achieve this
great milestone.

Assets 6
Pre-release

@jfriesse jfriesse released this Mar 6, 2019 · 92 commits to master since this release

I am pleased to announce the fourth testing (Alpha 4) release of Corosync 3.0 (codename Camelback) available immediately from our website at
http://build.clusterlabs.org/corosync/releases/ as corosync-2.99.3.

You can also download RPMs for various distributions from CI https://kronosnet.org/builds/.

This release is quite small so it's probably last Alpha and next release is probably going to be Beta or RC.

One of the biggest feature is better handling for ifdown with UDPU transport (knet handles ifdown without any problem since Alpha 1). This doesn't mean simulating cluster failure by using ifdown is supported or recommended! (simply because it exercises different code path than real failure).

Complete changelog for Alpha 4 (compared to Alpha 3):

Bin Liu (1):
      totemudpu: Add local loop support

Christine Caulfield (5):
      knet: Fix knet log buffer size
      knet: Don't try to create loopback interface twice
      totemconfig: Check for things that cannot be changed on the fly
      config: Enforce use of 'name' node attribute in multi-link clusters
      config: Fail config validation if not all nodes have all links

Jan Friesse (4):
      upstart: Remove notifyd upstart unit
      init: Use existing env variable from sysconf
      Fix snprintf warnings
      totemudpu: Pass correct paramto totemip_nosigpipe

We did our best to test this release as best as we could, but still take it as an Alpha version .

For anybody who migrates from stable version please consult http://people.redhat.com/ccaulfie/docs/KnetCorosync.pdf about changes in Corosync 3.

Thanks/congratulations to all people that contributed to achieve this
great milestone.

Assets 6
Pre-release

@jfriesse jfriesse released this Mar 6, 2019 · 102 commits to master since this release

I am pleased to announce the second testing (Alpha 3) release of Corosync 3.0 (codename Camelback) available immediately from our website at
http://build.clusterlabs.org/corosync/releases/ as corosync-2.99.2.

You can also download RPMs for various distributions from CI https://kronosnet.org/builds/.

This release contains very important fixes so it's "must to have" for everybody with previous alpha versions.

  1. Fix Alpha-2 regression which may result in membership not created
  2. Fix long time problem with CPG where paused node may not register other nodes leave

Complete changelog for Alpha 3 (compared to Alpha 2):

Chris Lamb (1):
      man: Make the manpages reproducible

Fabio M. Di Nitto (1):
      Drop all references to SECURITY file

Ferenc Wágner (4):
      Fix typo: sucesfully -> successfully
      Fix typo: defualt -> default
      NSS_NoDB_Init: the parameter is reserved, must be NULL
      tools: don't distribute what we can easily make

Jan Friesse (7):
      totemsrp: Implement sanity checks of received msgs
      totemsrp: Check join and leave msg length
      SECURITY: Remove SECURITY file
      totemsrp: Fix srp_addr_compare
      totemsrp: Log proc/fail lists in memb_join_process
      totemsrp: Fix leave message regression
      cpg: Inform clients about left nodes during pause

Rytis Karpuška (1):
      cpg: Handle fragmented message sending interrupt

We did our best to test this release as best as we could, but still take it as an Alpha version.

For anybody who migrates from stable version please consult http://people.redhat.com/ccaulfie/docs/KnetCorosync.pdf about changes in Corosync 3.

Thanks/congratulations to all people that contributed to achieve this
great milestone.

Assets 6

@jfriesse jfriesse released this Mar 5, 2019 · 326 commits to master since this release

I am pleased to announce the latest maintenance release of Corosync
2.4.4 available immediately from github releases.

This release contains a lot of fixes, including fix for CVE-2018-1084.

Complete changelog for 2.4.4:

Andrey Ter-Zakhariants (1):
      corosync-notifyd: improve error handling

Bin Liu (7):
      man:fix in corosync-qdevice.8
      quorumtool: remove duplicated help message
      cfg: nodeid should be unsigned int
      coroparse: Use readdir instead of readdir_r
      wd: fix snprintf warnings
      Fix compile errors in qdevice on FreeBSD
      qdevice: mv free(str) after port validation

Ferenc Wágner (6):
      Fix various typos
      Fix typo: recomended -> recommended
      man: support SOURCE_DATE_EPOCH
      configure: add --with-initconfigdir option
      Use static case blocks to determine distro flavor
      Use RuntimeDirectory instead of tmpfiles.d

Jan Friesse (29):
      coroparse: Do not convert empty uid, gid to 0
      sam: Fix snprintf compiler warnings
      quorumtool: Use full buffer size in snprintf
      man: Add note about qdevice parallel cmds start
      sync: Remove unneeded determine sync code
      sync: Call sync_init of all services at once
      corosync.conf: publicize nodelist.node.name
      totemudp[u]: Drop truncated packets on receive
      logging: Make blackbox configurable
      logging: Close before and open blackbox after fork
      init: Quote subshell result properly
      blackbox: Quote subshell result properly
      qdevice: quote certutils scripts properly
      sam_test_agent: Remove unused assignment
      qdevice: Fix NULL pointer dereference
      quorumtool: Don't set our_flags without v_handle
      qdevice: Nodelist is set into string not array
      qdevice: Check if user_data can be dereferenced
      qdevice: Add safer wrapper of strtoll
      qdevice: Replace strtol by strtonum
      qnetd: Replace strtol by strtonum
      main: Set errno before calling of strtol
      totemcrypto: Implement bad crypto header guess
      cpg: Use list_del instead of qb_list_del
      totemcrypto: Check length of the packet
      totemsrp: Implement sanity checks of received msgs
      totemsrp: Check join and leave msg length
      totemudp: Check lenght of message to sent
      qdevice msgio: Fix reading of msg longer than i32

Jan Pokorný (3):
      logsys: Avoid redundant callsite section checking
      man: corosync-qdevice: fix formatting vs. punctuation
      man: corosync-qdevice: some more stylistics

Jonathan Davies (1):
      man: fix cpg_mcast_joined.3.in

Rytis Karpuška (5):
      libcpg: Fix issue with partial big packet assembly
      totempg: Fix fragmentation segfault
      totempg: use iovec[i].iov_len instead of copy_len
      totempg: Fix corrupted messages
      cpg: Handle fragmented message sending interrupt

Toki Winter (1):
      corosync.aug: Add missing options

yuusuke (1):
      systemd: Delete unnecessary soft_margin

Upgrade is (as usually) highly recommended.

Thanks/congratulations to all people that contributed to achieve this
great milestone.

Assets 6
Pre-release

@jfriesse jfriesse released this Mar 6, 2019 · 116 commits to master since this release

I am pleased to announce the second testing (Alpha 2) release of Corosync 3.0 (codename Camelback) available immediately from our website at
http://build.clusterlabs.org/corosync/releases/ as corosync-2.99.1.

Corosync is now using kronosnet CI, so you can download RPMs for various distributions from https://kronosnet.org/builds/.

There are two big changes compared to Alpha 1:

1.) With Knet transport, it's now possible to remove ring0_addr from corosync.conf and as long as there is other ring (ring1_addr), everything should work as expected.

This small feature required HUGE amount of work (thank you Chrissie) mostly in totem subsystem. As a result, Totem no longer addresses nodes by ring0_addr IP, but only by node id. This makes packet smaller (especially for bigger clusters).

Ability to remove ring0_addr also brings another problem: how to find local node in corosync.conf. We've decided to compare hostname with nodelist.node.name cmap keys. Using hostname is now preferred, but old style can still be used if ring0_addr exists.

One of the drawbacks is that we need to change on-wire protocol so Alpha 2 is wire incompatible with Alpha 1 (and any other version).

For more details and information about configuration please see updated http://people.redhat.com/ccaulfie/docs/KnetCorosync.pdf

2.) Because on-wire was already incompatible we've decided replace endian_detector with magic number and version. This allows Needle (with already included patch) to detect that other side is Corosync 3 and display informative message (now finally also with IP of sender).

Complete changelog for Alpha 2 (compared to Alpha 1):

Andrey Ter-Zakhariants (1):
      corosync-notifyd: improve error handling

Christine Caulfield (9):
      totem: Use nodeid ONLY in srp_addr
      totem: Return interface count correctly
      cfg: Fix cfg_get_node_addrs so that DLM works
      config: Update message when local host isn't found
      config: Allow use of ring0_addr
      totem: Fix debug warnings printed by knet
      knet: Always use link0 for loopback
      cfgtool: Don't assume link ID is a single char
      knet: Fix display of links with unconfigured link0

Christoph Berg (1):
      Man: Move overview mp to sections 3 and 7 from 8

Fabio M. Di Nitto (4):
      [man] fix regression introduced by 7162e75dcf81b7e475536e3060bf5e9312cd43b8
      [build] fix build with non-standard knet location
      [rpm] fixup corosync.spec.in to build on opensuse
      [rpm] use rpm macros to identify build distro

Jan Friesse (9):
      logging: Make blackbox configurable
      logging: Close before and open blackbox after fork
      init: Quote subshell result properly
      blackbox: Quote subshell result properly
      sam_test_agent: Remove unused assignment
      quorumtool: Don't set our_flags without v_handle
      main: Set errno before calling of strtol
      totemsrp: Add magic and version into header
      totem: Display IP of sender

Rytis Karpuška (3):
      totempg: Fix fragmentation segfault
      totempg: use iovec[i].iov_len instead of copy_len
      totempg: Fix corrupted messages

We did our best to test this release as best as we could, but still take it as an Alpha version.

Thanks/congratulations to all people that contributed to achieve this
great milestone.

Assets 6
Pre-release

@jfriesse jfriesse released this Mar 6, 2019 · 143 commits to master since this release

I am pleased to announce the first testing (Alpha 1) release of Corosync 3.0 (codename Camelback) available immediately from our website at
http://build.clusterlabs.org/corosync/releases/ as corosync-2.99.0.

Corosync 3.0 contains many interesting features mostly related to usage of Kronosnet (https://kronosnet.org/) as a default (and preferred) network transport.

Short list of high-level changes:

  • Knet
    • Support for 8 links which can be dynamically reconfigured without restart of corosync daemon
    • MTU auto-configuration
    • Support for NSS or OpenSSL encryption of packets
    • Compression
    • Higher throughput and lower latency (yes, it's really possible to get both of these at once )
    • And many more
  • Support for RDMA and Upstart is gone
  • Enhanced statistics
  • Nodelist is now mandatory
  • UDP/UDPU transports are still present, but supports only single ring (RRP is gone in favor of Knet) and doesn't support encryption
  • Support for systemd startup notifications
  • Corosync-qdevice found new home at https://github.com/corosync/corosync-qdevice and is no longer part of corosync repository/packages. It's going to have it's own release schedule, packages, ...

For presentation about corosync/knet please see http://build.clusterlabs.org/corosync/presentations/2017-Kronosnet-The-new-face-of-corosync-communications.pdf.

For more in depth document about how to configure/use corosync 3 with knet see http://people.redhat.com/ccaulfie/docs/KnetCorosync.pdf

We did our best to fix all the know issue, but if you find some, please let us know.

Thanks/congratulations to all people that contributed to achieve this
great milestone.

Assets 6
You can’t perform that action at this time.