0.3.8

@mreiferson mreiferson released this May 26, 2016 · 76 commits to master since this release

Upgrading from 0.3.7: Binaries contain no backwards incompatible changes.

This release fixes a critical regression in 0.3.7 that could result in message loss when attempting to cleanly shutdown nsqd by sending it a SIGTERM. The expected behavior was for it to flush messages in internal buffers to disk before exiting. See #757 and #759 for more details.

A few performance improvements landed including #743, which improves channel throughput by ~17%, and #740, which reduces garbage when reading messages from disk.

We're now stripping debug info, reducing binary size, in the official binary downloads and Windows binaries are now bundled with the appropriate .exe extension (#726 and #751).

Features:

  • #743 - nsqd: remove channel messagePump
  • #751 - strip debug info from binaries (thanks @ploxiln)
  • #740 - nsqd: reduce garbage when reading from diskqueue (thanks @dieterbe)

Bugs:

  • #757/#759 - nsqd: properly handle SIGTERM (thanks @judwhite)
  • #738 - updates for latest go-options
  • #730 - nsqd: diskqueue sync count on both read/write
  • #734 - nsqadmin: make rate column work without --proxy-graphite (thanks @ploxiln)
  • #726 - add .exe extension to Windows binaries (thanks @ploxiln)
  • #722 - nsqadmin: fix connected duration > 1hr

Downloads

0.3.7

@mreiferson mreiferson released this Feb 24, 2016 · 110 commits to master since this release

Upgrading from 0.3.6: Binaries contain no backwards incompatible changes.

This release has been built with Go 1.6.

Highlights include the various work done to reduce nsqd lock contention, significantly improving the impact of high load on the /stats endpoint, addressing issues with timeouts and failures in nsqadmin (#700, #701, #703, #709).

Thanks to @judwhite, nsqd and nsqlookupd now natively support being run as a Windows service (#718). We're also now publishing official Windows releases.

nsqd will now flock its data directory on linux, preventing two nsqd from running simultaneously pointed at the same path (#583).

On the bugfix side, the most noteworthy change is that nsqd will now correctly reset health state on a successful backend write (#671).

Features:

Bugs:

  • #672 - nsqd: fix max size accounting in diskqueue (thanks @judwhite)
  • #671 - nsqd: reset health on successful backend write (thanks @judwhite)
  • #615 - nsqd: prevent OOM when reading from nsqlookupd peer
  • #664/#666 - dist.sh/Makefile cleanup (thanks @ploxiln)

Downloads

0.3.6

@mreiferson mreiferson released this Sep 25, 2015 · 163 commits to master since this release

Upgrading from 0.3.5: Binaries contain no backwards incompatible changes.

We've adopted the Contributor Covenant 1.2 Code of Conduct (#593). Help us keep NSQ open and inclusive by reading and following this document.

We closed a few longstanding issues related to nsqadmin, namely (#323, et al.) converting it to an API and single-page app (so that it is much easier to develop), displaying fine-grained errors (#421, #657), and enabling support for --tls-required configurations (#396).

For nsqd, we added support for deferred publishing aka DPUB (#293), which allows a producer to specify a duration of time to delay initial delivery of the message. We also addressed performance issues relating to large numbers of topics/channels (#577) by removing some per-channel goroutines in favor of a centralized, periodic, garbage collection approach.

In order to provide more flexibility when deploying NSQ in dynamically orchestrated topologies, nsqd now supports the ability to configure nsqlookupd peers at runtime via HTTP (#601), eliminating the need to restart the daemon.

As part of the large nsqadmin refactoring, we took the opportunity to cleanup the internals for all of the daemon's HTTP code paths (#601, #610, #612, #641) as well as improving the test suite so that it doesn't leave around temporary files (#553).

Features:

  • #593 - add code of conduct
  • #323/#631/#632/#642/#421/#649/#650/#651/#652/#654 - nsqadmin: convert to API / single-page app
  • #653 - nsqadmin: expand notification context
  • #293 - nsqd: add deferred pub (DPUB)
  • #577 - nsqd: drop per-channel queue workers in favor of centralized queue GC
  • #584 - nsqlookupd: improve registration DB performance (thanks @xiaost)
  • #601 - nsqd: HTTP endpoints to dynamically configure nsqlookupd peers
  • #608 - nsqd: support for filtering /stats to topic/channel (thanks @chrusty)
  • #601/#610/#612/#641 - improved HTTP internal routing / log HTTP requests
  • #628 - nsqd: clarify help text for --e2e-processing-latency-percentile
  • #640 - switch --{consumer,producer}-opt to nsq.ConfigFlag

Bugs:

  • #656 - nsqadmin: update statsd prefix to stats.counters
  • #421/#657 - nsqadmin: display upstream/partial errors
  • #396 - nsqdamin/nsqd: support for --tls-required
  • #558 - don't overwrite docker root FS
  • #582 - nsqd: ignore benign EOF errors
  • #587 - nsqd: GUID error handling / catch errors if GUID goes backwards (thanks @mpe)
  • #586 - nsqd: fix valid range for --worker-id
  • #550/#602/#617/#618/#619/#620/#622 - nsqd: fix benchmarks (thanks @Dieterbe)
  • #553 - cleanup test dirs
  • #600 - nsqd: enforce diskqueue min/max message size (thanks @twmb)

Downloads

0.3.5

@mreiferson mreiferson released this Apr 26, 2015 · 322 commits to master since this release

Upgrading from 0.3.4: Binaries contain no backwards incompatible changes.

This is another quick bug fix release to address the broken nsqadmin binary in the v0.3.4 distribution (see #578).

See the release notes for v0.3.3 for more info.

Downloads

0.3.4

@mreiferson mreiferson released this Apr 26, 2015 · 326 commits to master since this release

WARNING: please upgrade to v0.3.5 to address the broken nsqadmin binary.

Upgrading from 0.3.3: Binaries contain no backwards incompatible changes.

This is a quick bug fix release to fix the outdated go-nsq dependency in v0.3.3 for the bundled utilities (see 6e8504e).

Downloads

0.3.3

@mreiferson mreiferson released this Apr 26, 2015 · 330 commits to master since this release

WARNING: please upgrade to v0.3.5 to address the outdated go-nsq dependency for the
bundled utilities and the broken nsqadmin binary.

Upgrading from 0.3.2: Binaries contain no backwards incompatible changes.

This release is primarily a bug fix release after cleaning up and reorganizing the codebase. nsqadmin is now importable, which paves the way for completing #323. The bundled utilities received a few feature additions and bug fixes (mostly from bug fixes on the go-nsq side).

Features:

Bugs:

  • #573 - nsqd: don't persist metadata upon startup (thanks @xiaost)
  • #560 - nsqd: do not print EOF error when client closes cleanly (thanks @twmb)
  • #557 - nsqd: fix --tls-required=tcp-https with --tls-client-auth-policy (thanks @twmb)
  • #545 - enable shell expansion in official Docker image (thanks @paddyforan)

NOTE: the bundled utilities are built against go-nsq v1.0.4 and include all of those features/fixes.

Downloads

0.3.2

@mreiferson mreiferson released this Feb 8, 2015 · 388 commits to master since this release

Upgrading from 0.3.1: Binaries contain no backwards incompatible changes however as of this release we've updated our official Docker images.

We now provide a single Docker image nsqio/nsq that includes all of the NSQ binaries. We did this for several reasons, primarily because the tagged versions in the previous incarnation were broken (and did not actually pin to a version!). The new image is an order of magnitude smaller, weighing in around 70mb.

In addition, the impetus for this quick release is to address a slew of reconnect related bug fixes in the utility apps (nsq_to_nsq, nsq_to_file, etc.), for details see the go-nsq v1.0.3 release notes.

Features:

Bugs:

  • #529 - nsqadmin: fix more #ephemeral topic deletion issues
  • #530 - nsqd: fix the provided sample config file (thanks @jnewmano)
  • #538 - nsqd: fix orphaned ephemeral channels (thanks @adamsathailo)

Downloads

0.3.1

@mreiferson mreiferson released this Jan 22, 2015 · 409 commits to master since this release

Upgrading from 0.3.0: No backwards incompatible changes.

This release contains minor bug fixes and feature additions.

There are a number of functionality improvements to the nsq_stat and nsq_to_file helper applications (and general support for #ephemeral topics, broken in 0.2.30).

Additionally, the TLS options continue to improve with support for setting --tls-min-version and a work-around for a bug relating to TLS_FALLBACK_SCSV (to be fixed in Go 1.5).

Features:

  • #527 - nsq_stat: deprecate --status-every in favor of --interval
  • #524 - nsq_stat: add --count option (thanks @nordicdyno)
  • #518 - nsqd: set defaults for --tls-min-version and set TLS max version to 1.2
  • #475/#513/#518 - nsqd: --tls-required can be disabled for HTTP / add --tls-min-version (thanks @twmb)
  • #496 - nsq_to_file: add <PID> to filename and rotation by size/interval (thanks @xiaost)
  • #507 - nsq_stat: add rates (thanks @xiaost)
  • #505 - nsqd: speed up failure path of BytesToBase10 (thanks @iand)

Bugs:

  • #522 - nsqadmin: fix #ephemeral topic deletion issues
  • #509 - nsqd: fix diskqueue atomic rename on Windows (thanks @allgeek)
  • #479 - nsqd: return output_buffer_* resolved settings in IDENTIFY response (thanks @tj)

Downloads

0.3.0

@mreiferson mreiferson released this Nov 18, 2014 · 444 commits to master since this release

Upgrading from 0.2.31: No backwards incompatible changes.

This release includes a slew of bug fixes and few key feature additions.

The biggest functional change is that nsqd no longer decrements its RDY count for clients. This means that client libraries no longer have to periodically re-send RDY. For some context, nsqd already provided back-pressure due to the fact that a client must respond to messages before receiving new ones. The decremented RDY count only made the implementation of the server and client more complex without additional benefit. Now the RDY command can be treated as an "on/off" switch. For details see #404 and the associated changes in nsqio/go-nsq#83 and nsqio/pynsq#98.

The second biggest change (and oft-requested feature!) is #ephemeral topics. Their behavior mirrors that of channels. This feature is incredibly useful for situations where you're using topics to "route" messages to consumers (like RPC) or when a backlog of messages is undesirable.

There are now scripts in the bench directory that automate the process of running a distributed benchmark. This is a work-in-progress, but it already provides a closer-to-production setup and therefore more accurate results. There's much work to do here!

A whole bunch of bugs were fixed - notably all were 3rd-party contributions! Thanks!

  • #305 - #ephemeral topics
  • #404/#459 - don't decr RDY / send RDY before FIN/REQ
  • #472 - improve nsqd diskqueue sync strategies
  • #488 - ability to filter topics by regex in nsq_to_file (thanks @lxfontes)
  • #438 - distributed pub-sub benchmark scripts
  • #448 - better nsqd IOLoop logging (thanks @rexposadas)
  • #458 - switch to gpm for builds

Bugs:

  • #493 - ensure all nsqd Notify() goroutines have exited prior to shutdown (thanks @allgeek)
  • #492 - ensure diskqueue syncs at end of benchmarks (thanks @Dieterbe)
  • #490 - de-flake TestPauseMetadata (thanks @allgeek)
  • #486 - require ports to be specified for daemons (thanks @jnewmano)
  • #482 - better bash in dist.sh (thanks @losinggeneration)
  • #480 - fix panic when nsqadmin checks stats for missing topic (thanks @jnewmano)
  • #469 - fix panic when misbehaving client sends corrupt command (thanks @prio)
  • #461 - fix panic when nsqd decodes corrupt message data (thanks @twmb)
  • #454/#455 - fix 32-bit atomic ops in nsq_to_nsq/nsq_to_http (thanks @leshik)
  • #451 - fix go get compatibility (thanks @adams-sarah)

Downloads

0.2.31

@mreiferson mreiferson released this Aug 26, 2014 · 499 commits to master since this release

Upgrading from 0.2.30: No backwards incompatible changes.

This release includes a few key changes. First, we improved feedback and back-pressure when nsqd writes to disk. Previously this was asynchronous and would result in clients not knowing that their PUB had failed. Interestingly, this refactoring improved performance of PUB by 41%, by removing the topic's goroutine responsible for message routing in favor of N:N Go channel communication. For details see #437.

@paddyforan contributed official Dockerfiles that are now built automatically via Docker Hub. Please begin to use (and improve these) as the various older images we had been maintaining will be deprecated. See the updated docs.

The utility apps deprecated the --reader-opt flag in favor of --consumer-opt and nsq_to_nsq and to_nsq received a --producer-opt flag, for configuring details of the connection publishing to nsqd. Additionally, it is now possible to configure client side TLS certificates via tls_cert and tls_key opts.

As usual, we fixed a few minor bugs, see below for details.

New Features / Enhancements:

  • #422/#437 - nsqd: diskqueue error feedback/backpressure (thanks @boyand)
  • #412 - official Dockerfiles for nsqd, nsqlookupd, nsqadmin (thanks @paddyforan)
  • #442 - utilities: add --consumer-opt alias for --reader-opt and add --producer-opt to nsq_to_nsq (also support configuration of tls_cert and tls_key)
  • #448 - nsqd: improve IOLoop error messages (thanks @rexposadas)

Bugs:

  • #440 - nsqd: fixed statsd GC stats reporting (thanks @jphines)
  • #434/#435 - refactored/stabilized tests and logging
  • #429 - nsqd: improve handling/documentation of --worker-id (thanks @bschwartz)
  • #428 - nsqd: IDENTIFY should respond with materialized msg_timeout (thanks @visionmedia)

Downloads