Backwards Incompatible Changes
These affect users who import M3 as a Go library. Deployed binaries, on-disk formats and wire formats are unchanged.
- All: Metrics scopes now use
github.com/uber-go/tally/v4; thegithub.com/m3db/prometheus_*forks are no longer used and the instrument package exposes a single upstreamprometheus/client_golangregistry type (#4423) - All:
github.com/m3db/stackmurmur3is replaced bygithub.com/twmb/murmur3(bit-identical output, verified by a parity test) (#4420, #4422) - All: Minimum Go version is now 1.26 (#4410, #4416, #4425)
- All: etcd dependency moved from 3.5.5 to 3.6.13; the embedded etcd config uses the renamed
embed.ConfigURL fields (ListenPeerUrls,ListenClientUrls,AdvertisePeerUrls,AdvertiseClientUrls) (#4407, #4416)
Features
- M3Aggregator: Support ingestion routing policies, with a routing policy filter in M3Msg and dynamic configuration of consumer service filters via topic updates (#4297, #4378, #4381, #4386, #4388, #4389, #4390)
- M3Aggregator: TLS support for the aggregator server and client, including TLS handshake on connect and a local TLS development environment (#4283, #4376, #4377, #4379)
- M3Aggregator: Add percentage filter for consumer writers and allow multiple filters per consumer service (#4223, #4247)
- M3Aggregator: Add read timeout, custom
DialContext/ContextDialerand bytes emission metrics to the aggregator client (#4131, #4290, #4300, #4315) - M3Aggregator: Add
TimerBatchSizesdistribution metric and write success/error latency histograms (#4320, #4321, #4384, #4385) - M3Aggregator: Local Docker development environment with debug mode and a consumer service (#4374, #4375, #4380)
- M3Coordinator: Add
IncludeTagsto R2 rollup options (#4277) - M3Coordinator: Add config for forwarding a percentage of unique series via remote write, and sampled logging of write/fetch host errors (#4173, #4190)
- M3Coordinator: Sort Graphite find results (#4163)
- M3DB: Circuit breaker middleware for client host queue writes, tripping only on timeout errors (#4335, #4336, #4373, #4393)
- M3DB: Sub-cluster placement algorithm with validation for partial sub-clusters (#4355, #4391)
- M3DB: Support static topology config with more hosts than the replication factor (#4239, #4242)
- M3DB: Add
x/encoding.Clonerto shallow clone response iterators (#4271) - M3DB: Add metrics for time taken to connect to the cluster at startup (#4267)
- M3DB: Add
verify_idstool (#4244) - M3DB: Support etcd prefix KV retrievals and watches (#4333)
- M3Msg: Graceful close for message writers, abort on server close, and stop accepting new connections before disconnecting existing ones (#4328, #4329, #4330, #4387)
- M3Msg: Consumer writer flush latency metrics (#4326)
- All: Optionally log build information as JSON, and expose a Zap config from the log config (#4097, #4100)
- All: Release linux and darwin arm64 binaries and multi-arch (amd64 and arm64) Docker images (#4442)
Bug Fixes
- M3Aggregator: Fix colliding filters in
MultiCharSequenceFilterby matching the most complete pattern rather than the first (#4188) - M3Aggregator: Followers can lead before the first flush is persisted (#4169)
- M3Aggregator: Fix a memory allocation issue in
ConsumerServiceWriterMetrics(#4382) - M3Aggregator: Close the aggregator client connection asynchronously (#4327)
- M3Coordinator: Fix protobuf tag collision with Prometheus (#4256)
- M3Coordinator: Fix inverted interpretation of the Prometheus remote write shadow sampling config (#4210)
- M3Coordinator: Fix etcd config default value handling and only configure env configs when specified (#4130, #4216)
- M3Query: Fix matching of disjunctions that include the empty string (#4258)
- M3DB: Fix regexp LRU cache bug (#4250)
- M3DB: Write datapoints that share timestamp, value and annotation but differ in time unit (#4401)
- M3DB: Add index results from subsequent bootstrap runs (#4193)
- M3DB: Fix strictly increasing order of the shard entry list for
FetchBlocksMetadatapagination (#4298) - M3DB: Count write acks from paired leaving/initializing shards (#4191)
- M3DB: Fix nil pointer panic on
SyncClusterfailure and theGetmethod ofwriteBatchPooledReqPool(#4246, #4253) - M3DB: Fix wrong error variable returned in two places, found by
nilnesserr(#4430) - M3Msg: Remove data race in the producer (#4160)
- All: Satisfy the yaml
Marshalerinterface on structs with custom unmarshalers (#4213)
Performance
- M3Aggregator: Pool locked aggregations, fix an unintended allocation during element flushing, and clean up the aggregated metric writer and encoder (#4108, #4109, #4110, #4112)
- M3Coordinator: Optimize
m3.IsRollupIDto not allocate and allow iterator reuse inReverseMatch(#4304, #4305, #4307) - M3Msg: Message pooling cleanup, specialized message writer acker map, field alignment fix for
RefCountedMessage, and parallel consumer writer flushes (#4104, #4105, #4113, #4331) - M3DB: Pass the shard set as part of tick instead of storing it in namespace readers (#4323)