0.9.0 (17.10.2016)
This release introduces Kafka 0.10 producer/consumer API and is no longer compatible with Kafka 0.8.x and 0.9.x deployments.
Features
(#558) Use Kafka 0.10 producer/consumer API
This change breaks backwards compatibility - Hermes will not run on 0.8.x, 0.9.x Kafka clusters
Hermes uses Kafka 0.10 APIs. The change is not big for producers in Frontend module, but it rearranged whole Consumers module.
The benefits of moving to Kafka 0.10 (except from leaving the deprecated APIs behind) are:
- decreased number of active threads: in cluster with ~600 subscriptions number of threads decreased from ~4400 to ~700
- decreased memory consumption: same cluster, memory usage dropped by 10-20%
- decreased CPU consumption: same cluster, day-to-day CPU consumption dropped by ~10%
- greatly decreased shutdown time
The change is transparent for the end users.
Upgrading note
Before upgrading, make sure that offsets are committed and stored in Kafka (option: kafka.consumer.dual.commit.enabled is set to true or
kafka.consumer.offsets.storage is set to kafka (default) in Consumers module).
When upgrading, all Consumers should be stopped at once and started with new version.
(593) Confluent Schema Registry integration
Breaking change: Support for storing and validating JSON schemas has been removed
Hermes be integrated with Confluent Schema Registry to store and read Avro schemas. We kept existing integration with schemarepo.org repository. To switch between implementations, use schema.repository.type option:
schema_repofor "old" schemarepo.orgschema_registryfor Confluent Schema Registry