/vernemq

Latest release
Verified
This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
VerneMQ 1.7.0
Latest release

@vmq-ci vmq-ci released this Jan 30, 2019

  • Fix vmq_webhooks issue where MQTTv5 hooks where not configurable in the
    vernemq.conf file.
  • Support shared subscriptions in vmq_bridge.
  • Fix bug in Prometheus output (#923).
  • Fix max_message_rate to include MQTTv5 sessions.
  • Support new throttle modifier for the auth_on_publish and
    auth_on_publish_m5 hooks which will throttle the publishing client by
    waiting for a given number of milliseconds before reading new data from the
    client socket. Note, backpressure is not supported for websocket
    connections. This feature was kindly sponsored by Arduino SA
    (https://www.arduino.cc/).
  • Fix issue with long-running vmq-admin commands (#644).
  • Added a new HTTP module vmq_health_http exposing a /health endpoint that
    returns 200 when VerneMQ is accepting connections and joined the cluster
    (for clustered setups) or returns 503 in case any of the two conditions
    are not met (#889).
  • Fix issue where a QoS2 message would be republished if a client would resend
    the PUBLISH packet with the same message id within a non-finished QoS2 flow
    (#944).
  • Make VerneMQ build on FreeBSD (11.2-RELEASE) with gmake rel.
  • Fix issue with blocking socket commands in inter-node communication.
  • Improve error messages when parsing invalid CONNECT packets.
  • Log IP and port information on authentication failures to make it easy to
    block the client with tools like Fail2ban (#931).
  • Fix issue which would crash the session if a client would resend a (valid)
    pubrec during a Qos2 flow (#926).
  • Fix vmq_diversity error message if invalid or unknown modifiers are returned
    from a lua hook implementation.
  • Fix issue preventing MQTT 5.0 properties from being modifiable in
    auth_on_publish_m5 (#964).
  • Fix issue which causes a crash if not enough data is available while parsing
    the CONNECT frame in vmq_mqtt_pre_init (#950, #962).
  • Fix issue which could cause vmq-admin session show to crash when using
    --limit=X to limit the number of returned results (#902).
  • Handle edge case in the websocket implementation which could cause warninigs
    when the session was terminating.
  • Expose vernemq_dev_api:disconnect_by_subscriber_id/2 in lua
    vmq_api.disconnect_by_subscriber_id/2, an example looks like:
    vmq_api.disconnect_by_subscriber_id({mountpoint = "", client_id = "client-id"}, {do_cleanup = true}).
  • Improve vmq_webhooks errors so the error from the endpoint is shown as the
    error reason where relevant.
  • Optimization: optimize away read operation during client registration by
    reusing already available data.
  • Enable plugins to expose metrics via the available metric providers.
  • Add histogram metric type.
  • Fix incorrect debug log message in QoS 2 flow.
  • Expose bridge metric for messages dropped in case the outgoing queue is full.
  • Log (debug level) when a LWT is suppressed on session takeover.
  • Optimize subscribe operation by refactoring away one read from the metadata
    store.
  • Add protection mechanism for the plumtree metadata store from subscription
    floods by dropping and not acknowledging i_have messages when the mailbox
    size grows above a certain threshold. This threshold is configurable via the
    plumtree.drop_i_have_threshold hidden option. The default is 1000000. This
    work was kindly contributed by ADB (https://www.adbglobal.com).
  • Add new vmq-admin retain commands to inspect the retained message store.
  • Support for different MySQL password hashing methods in vmq_diversity,
    ensuring compatibility with MySQL 8.0.11+. The method is configurable via the
    vmq_diversity.mysql.password_hash_method option which allows:
    password (default for compatibility), md5, sha1 or sha256.
  • Fix the HTTP /status.json endpoint to have a valid JSON output (#786).
  • Fix bug where internal application plugins where shown as normal plugins.
  • Fix crash in bridge when calling vmq-admin session show by fixing the
    vmq_ql row initializer to handle plugin sessions (the bridge starts a local
    plugin session).
  • Add improvements to VerneMQ status page to have nicer UI and be readable on smaller devices.
  • Upgraded dependency sext to 1.5.0 (required for better OSX compilation).
  • Do not accept new client connections while the broker is shutting down as this
    could cause errors to be reported in the log (#1004).
  • Fix vmq_diversity PostgreSQL reconnect issue (#1008).
  • Fix vmq_webhooks so peer port is not considered when caching the
    auth_on_register to avoid cache misses.
  • Multiple bug fixes and improvements in vmq_swc.
  • Add the vmq_swc metadata plugin (using the already existing LevelDB backend) to
    the default VerneMQ release. To use vmq_swc instead of vmq_plumtree set
    metadata_plugin = vmq_swc in vernemq.conf. vmq_swc is still in Beta.
  • Add missing increments of the mqtt_connack_sent metric for CONNACK
    success(0) for MQTT 3.1.1.
  • Handle edge case with unknown task completion messages in vmq_reg_sync after
    a restart.
  • Fix bug which could cause a queue cleanup to block indefinitely and cause the
    vmq_in_order_delivery_SUITE tests to fail.
  • Add a new metric (queue_initialized_from_storage) to better monitor queue
    initialization process after a node restart.
  • Fix edge case where an extra queue process could be started when metadata
    events arrive late. Now local queue processes are only started when triggered
    via a new local MQTT session.
  • Reimplement dead queue repair mechanism.
  • Add feature to reauthorize existing client subscriptions by reapplying current auth_on_subscribe
    and auth_on_subscribe_m5 hooks. This feature is exposed as a developer API in vernemq_dev, via
    the vmq-admin session reauthorize CLI command, and as a API for vmq_diversity Lua scripts. This
    work was kindly sponsored by AppModule AG (http://www.appmodule.net).
  • Improve planned cluster leave queue migration speed significantly (#766).
  • Start metrics server before setting up queues to ensure queue metric counts
    are correct when restarting VerneMQ.
  • Let Travis CI build the VerneMQ release packages.
  • Add new metric system_process_count which is a gauge representing the
    current number of Erlang processes.
  • Add queue_started_at and session_started_at information to the vmq-admin session show command. Times are POSIX time in milliseconds and local to the
    node where the session or queue was started.
Assets 10
Verified
This commit was signed with a verified signature.
@larshesel larshesel Lars Hesel Christensen
GPG key ID: 52C18780DA266458 Learn about signing commits
VerneMQ 1.6.2

@larshesel larshesel released this Nov 29, 2018 · 83 commits to master since this release

  • Fix max_message_rate to include MQTTv5 sessions.
  • Fix issue with long-running vmq-admin commands (#644).
  • Fix issue where a QoS2 message would be republished if a client would resend
    the PUBLISH packet with the same message id within a non-finished QoS2 flow
    (#944).
  • Improve error messages when parsing invalid CONNECT packets.
  • Fix issue which would crash the session if a client would resend a (valid)
    pubrec during a Qos2 flow (#926).
  • Fix vmq_diversity error message if invalid or unknown modifiers are returned
    from a lua hook implementation.
  • Fix issue preventing MQTT 5.0 properties from being modifiable in
    auth_on_publish_m5 (#964).
  • Fix issue which causes a crash if not enough data is available while parsing
    the CONNECT frame in vmq_mqtt_pre_init (#950, #962).
  • Fix issue which could cause vmq-admin session show to crash when using
    --limit=X to limit the number of returned results (#902).
  • Handle edge case in the websocket implementation which could cause warninigs
    when the session was terminating.
Assets 2
Verified
This commit was signed with a verified signature.
@larshesel larshesel Lars Hesel Christensen
GPG key ID: 52C18780DA266458 Learn about signing commits
VerneMQ 1.6.1

@larshesel larshesel released this Nov 2, 2018 · 83 commits to master since this release

  • Fix vmq_webhooks issue where MQTTv5 hooks where not configurable in the
    vernemq.conf file.
  • Fix bug in Prometheus output (#923).
Assets 2
Verified
This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
VerneMQ 1.6.0

@dergraf dergraf released this Oct 26, 2018 · 84 commits to master since this release

  • Fix issue when calling a function in a Lua script that requires more time to complete than the default gen_server timeout (#589).

  • Silently drop messages published by the client that use a routing key starting with '$'.

  • Full MQTTv5 support

    With this release VerneMQ officially supports MQTT protocol version
    5.0. The MQTTv5 support is currently marked as in BETA and we may
    still have to change some things.

    VerneMQ has full support for the complete MQTTv5 spec, but to list a
    few of the new features:

    • Support for enhanced (re)authentication

    • User properties

    • Message Expiration

    • Last Will and Testament delay

    • Shared subscriptions

    • Retained messages

    • Request/Response flows

    • Topic aliases

      VerneMQ supports topic aliases from both the client to the broker and from
      the broker to the client.

      When a client connects to the broker, the broker will inform the client of
      the maximum allowed topic alias using the topic alias max property on the
      CONNACK packet (if it has been set to a non-zero value). The topic alias
      maximum property can be configured through the topic_alias_max
      configuration variable or overriden in a plugin in the auth_on_register
      hook. The broker will then handle topic aliases from the client as per the
      MQTTv5 spec.

    • Flow control

    • Subscription flags Retain as Published, No Local and Retain Handling.

    • Subscriber ids

    By default MQTTv5 is disabled, but can be enabled on a listener basis, for
    example listener.tcp.allowed_protocol_versions=3,4,5 would enable MQTT
    version 3.1, 3.1.1 and 5.0 on the TCP listener.

    MQTTv5 support has been added to the vmq_passwd, vmq_acl, vmq_webhooks
    plugins as well as the parts of the vmq_diversity plugin to support the
    MySQL, PostgreSQL, Redis and MongoDB authentication and authorization
    mechanisms. The Lua scripting language provided by the vmq_diversity plugin
    does not yet expose all MQTT 5.0 plugin hooks.

    The vmq_bridge plugin currently has no support for MQTTv5.

    !! Note !! that all MQTTv5 related features and plugins are in BETA and may
    still change if needed.

  • The metrics:

    mqtt_connack_not_authorized_sent
    mqtt_connack_bad_credentials_sent
    mqtt_connack_server_unavailable_sent
    mqtt_connack_identifier_rejected_sent
    mqtt_connack_unacceptable_protocol_sent
    mqtt_connack_accepted_sent

    Have been merged into a single metric mqtt_connack_sent and the various MQTT
    3.1.1 return codes have been mapped into labels. So for example
    mqtt_connack_sent with the label return_code=success replaces
    mqtt_connack_accepted_sent.

  • Added on_message_drop hook that is called for every message dropped due to
    exceeding the MQTTv5 max_packet_size property, hitting the message expiry,
    or when load shedding when enqueing messages.

  • Fix ordering bug in webhook subscribe topic authentication and topic rewrites
    (#823)

  • Fix issue when terminating the vmq_server application (#828).

  • Make VerneMQ build on SmartOS / Illumos / Solaris.

  • Ensure strings passed from Lua to the logger are escaped (#864).

  • Handle Mongo Date / ISODate datatype properly in Lua / vmq_diversity
    (#857).

  • Optimize subscribe/unsubscribe operations for large fanout cases.

  • Allow non-standard MQTT version 131 (MQTT 3.1 bridge) by default (this was
    accidentally changed in VerneMQ 1.4.0).

  • Improve error messages returned by the vmq_diversity plugin so it's easier
    to understand and debug authentication and authorisation issues.

  • Improve performance for looking up queue processes.

  • Make VerneMQ run on Erlang/OTP 21. To do this we had to upgrade rebar3 to
    version 3.6.0 and lager to version 3.6.3. We also removed the eper
    dependency and suppressed some new warnings due to erlang:get_stacktrace/0
    having been deprecated in Erlang/OTP 21.

  • The bridge plugin (vmq_bridge) now has an option to queue outgoing message
    in case the network connection disappears. This is configurable with the
    max_outgoing_buffered_messages setting on a bridge. In addition the bridge
    now has a simple cli interface under vmq-admin bridge show. This work was
    kindly sponsored by Diacare-Soft(http://diacare-soft.ru).

  • Fix multiple message retry issues in the MQTT client used by vmq_bridge.

  • Fix issue where the message ordering wasn't preserved after a client reconnect.

  • Add experimental vmq_swc plugin that provides an alternative to the existing
    vmq_plumtree for metadata storage and replication. One must compile VerneMQ
    with make swc to generate a release containing the vmq_swc plugin.

  • Remove unused vmq_bridge ssl capath config option. This was never used
    internally.

Assets 2
VerneMQ 1.5.0

@dergraf dergraf released this Aug 21, 2018 · 227 commits to master since this release

  • Fix issue in the bridge preventing it from reconnecting after a connection
    timeout (#726).
  • Fix issue with self referential cluster leaves and cluster readiness probing
    in general (#717).
  • Add preparations for MQTTv5:
    • Make it possible to extend the cluster internal message format when adding
      new features while ensuring back- and forwards compatibility.
    • Refactor the leveldb message store code to support versioned indexes as well
      as messages.
    • Refactor the retained message store to support versions.
    • Add support for subscriber data with subscription options.
  • Fix issue when validating a shared subscription topic (#756).
  • Fix issue with retried PUBREC frames (#750).
  • Make it possible to suppress the Last Will and Testament message if an
    existing session already exists for the connecting client. This is
    configurable via the hidden setting suppress_lwt_on_session_takeover in the
    VerneMQ configuration file. The default is off.
  • Fix issue with PUBREL frames retried after client reconnects (#762).
  • Refactor and cleanup retry mechanism.
  • Warn about deprecated setting message_size_limit only when it has been defined.
  • Change build script so nightly packages will have names on the form
    vernemq_1.4.0-10-gde1b1f5-1_amd64.deb, where de1b1f5 refers to the commit
    hash from which the package was built and 10 is the number of commits since
    the latest tag. This makes it much easier to reason about nightly builds and
    the features they contain.
  • Refactor and generalize the metrics systems to allow labelling metrics coming
    from different sources in order to differentiate them in the various exporters
    and the command line. Labels are added to the Prometheus exporter by
    default. To enable generating additional metrics from the labels in the
    graphite exporter the hidden setting graphite_include_labels has to be set
    to on in the vernemq.conf file. Labels are not exposed in the $SYS
    metrics.
  • Ensure the vmq_bridge is properly restarted after a crash (#785).
  • Fix issue where not calling http.body(ref) in a Lua script would not return
    the underlyning HTTP connection to the connection pool. This now happens
    automatically, irregardless of the user calling http.body(ref) or not
    (#588).
Assets 2
Verified
This commit was signed with a verified signature.
@dergraf dergraf Andre Graf
GPG key ID: 76D6BD4EBE98CAED Learn about signing commits
VerneMQ 1.4.2

@dergraf dergraf released this Aug 21, 2018 · 243 commits to master since this release

  • Fix issue with PUBREL frames retried after client reconnects (#762).
  • Refactor and cleanup retry mechanism.
  • Ensure the vmq_bridge is properly restarted after a crash (#785).
  • Fix issue where not calling http.body(ref) in a Lua script would not return
    the underlyning HTTP connection to the connection pool. This now happens
    automatically, irregardless of the user calling http.body(ref) or not
    (#588).
Assets 2
Verified
This commit was signed with a verified signature.
@dergraf dergraf Andre Graf
GPG key ID: 76D6BD4EBE98CAED Learn about signing commits
VerneMQ 1.4.1

@dergraf dergraf released this Jul 5, 2018 · 243 commits to master since this release

  • Fix issue in the bridge preventing it from reconnecting after a connection timeout (#726).
  • Fix issue with self referential cluster leaves and cluster readiness probing in general (#717).
  • Fix issue when validating a shared subscription topic (#756).
  • Fix issue with retried PUBREC frames (#750).
Assets 2
VerneMQ 1.4.0

@dergraf dergraf released this May 25, 2018 · 243 commits to master since this release

  • Fix for OSX compilation issue on vmq_passwd due to openssl headers not found.
  • Refactoring to further abstract and generalize the Plumtree metadata layer.
  • Small refactoring moving the calling of plugin hooks into the fsm code. This
    is a preparation for MQTTv5. Note, this change has an impact on the
    vmq_reg:direct_plugin_exports/1 function. Even though this function is
    internal and as such not guaranteed to be stable, we know some plugin
    developers use it and we therefore wanted to describe the changes. It works as
    before, except that the hooks auth_on_subscribe, on_subscribe or
    on_unsuscribe hooks will no longer be called when using the exported
    functions.
  • Corrected a few spelling errors in the vmq-admin help commands.
  • Added two new hidden configuration parameters for vmq_webhooks. vmq_webhooks uses
    the hackney HTTP client for HTTP requests.
    It starts its own hackney pool with a default connection pool of 100, and a
    connection timeout of 60000 milliseconds. This connection pool con now be
    configurable using the settings vmq_webhooks.pool_max_connections and
    vmq_webhooks.pool_timeout (#621).
  • Add a first version of a VerneMQ status page showing node and cluster
    information. This web-page is by default available on
    http://localhost:8888/status. The status page is implemented using jQuery,
    Twitter Bootstrap und mustache.js. As this is the first version the page
    should be considered experimental and will likely be changed in future
    releases.
  • Bugfix: Handle return values correctly when enqueuing messages to offline
    shared subscribers (#625).
  • Bugfix: Fix issue preventing messages delivered to a subscriber group from
    being delivered to the online group members first before attempting delivery
    to offline queues (#618).
  • Fix some Dialyzer issues.
  • Reduce replication load during a netsplit by making sure data is not attempted
    to be replicated to unreachable nodes.
  • Bugfix: Fix issue causing some session show options (peer_host,
    peer_port) to not work for websocket clients (#542).
  • Bugfix: Fix routing table initialization issue after restarting a node
    preventing shared subscriptions on a remote node from being included in the
    routing table (#595).
  • Bugfix: Fix race condition when fetching data from the internal query subsystem.
  • Fix build issue on Raspberry PI (make rpi-32).
  • Make it possible to specify which protocol versions are allowed on an MQTT
    listener. By default the protocol versions allowed are versions 3 and 4 (MQTT
    v3.1 and v3.1.1 respectively). To set the allowed protocol versions one can
    use listener.tcp.allowed_protocol_versions = [3,4] on the transport level or
    for a specific listener using
    listener.tcp.specific_listener.allowed_protocol_versions.
  • Fix bug causing an exception to be thrown when vmq-admin cluster leave is
    used with a timout value less than 5 seconds (#642).
  • Small refactoring enabling to store versioned message store values. This is a
    preparation for MQTTv5.
  • Bugfix: Fix a bug that prevented user plugins with an explicit path to be
    loaded.
  • Fix issue with new rebar3 upstream plugin version (the port-compiler) which
    made builds fail by pegging it to an older version (1.8.0).
  • Add xmerl and inets from the Erlang standard library to the release in order
    to allow plugin developers to have these libraries available.
  • Bugfix: Fix typo (graphie_api_key -> graphite_api_key) preventing the
    graphite api key from being set in the vernemq.conf file.
  • Bugfix: WebHooks Plugin. Close the Hackney CRef so that the socket is given
    back to the Hackney pool, for the case of non-200 HTTP OK status codes.
  • Bugfix: fix bug where queries with mountpoint and client-ids would return no
    entries causing disconnecting a client with a specific mountpoint to
    fail. Also ensure to use the default mountpoint if no mountpoint was
    passed. (#714).
  • Upgraded vernemq_dev to include the disconnect_by_subscriber_id/2 API.
Assets 2
Verified
This commit was signed with a verified signature.
@larshesel larshesel Lars Hesel Christensen
GPG key ID: 52C18780DA266458 Learn about signing commits
VerneMQ 1.3.2

@larshesel larshesel released this May 11, 2018 · 271 commits to master since this release

  • Fix bug causing an exception to be thrown when vmq-admin cluster leave is
    used with a timout value less than 5 seconds (#642).
  • Fix for OSX compilation issue on vmq_passwd due to openssl headers not found.
  • Bugfix: Fix a bug that prevented user plugins with an explicit path to be
    loaded
  • Fix build issue on Raspberry PI (make rpi-32).
  • Bugfix: Fix race condition when fetching data from the internal query subsystem.
Assets 2
Verified
This commit was signed with a verified signature.
@larshesel larshesel Lars Hesel Christensen
GPG key ID: 52C18780DA266458 Learn about signing commits
VerneMQ 1.3.1

@larshesel larshesel released this Mar 20, 2018 · 271 commits to master since this release

  • Bugfix: Handle return values correctly when enqueuing messages to offline
    shared subscribers (#625).
  • Bugfix: Fix issue preventing messages delivered to a subscriber group from
    being delivered to the online group members first before attempting delivery
    to offline queues (#618).
  • Fix some Dialyzer issues.
  • Reduce replication load during a netsplit by making sure data is not attempted
    to be replicated to unreachable nodes.
  • Bugfix: Fix issue causing some session show options (peer_host,
    peer_port) to not work for websocket clients (#542).
  • Bugfix: Fix routing table initialization issue after restarting a node
    preventing shared subscriptions on a remote node from being included in the
    routing table (#595).
Assets 2