This repository has been archived by the owner on Feb 26, 2020. It is now read-only.
/
ChangeLog
329 lines (289 loc) · 8.97 KB
/
ChangeLog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
0.3.50
------
- Misc bugfixes, including thread factory crash
0.3.49
------
- Bump dependency snapshot to 2016 03 30 version
0.3.48
------
- Improve DL documentation
- Make write pipeline purely asynchronous
- Able to skip corrupt entry
- Don't log stream failfast issues
- Script improvements
0.3.47
------
- Add create command to service and client
- Distributed Lock improvements
- Cache finagle stats
- Allow configuring replication factors per DL stream
0.3.46
------
- Append only reader/writer fixes
- Lock improvements for reliability
- Async Ledger Allocator uses zookeeper multi
0.3.45
------
- ReadAhead Refactor
- ReadAhead Idle Detection Improvement
- Add Interface for Log Segment Entry Writer
- Add ServerSetWatcher for server set changes
- Fix stream acquire limit and add more stats for limiting
0.3.44
------
- Bulk reads with waiting time
- LogRecordSet interface
- Cleanup LogSegmentWriter logic
- DL tooling improvements
- Throw IdleReader exception if readahead becomes idle
- Check existence of log or log segments should call zookeeper#sync
- Recreate zookeeper client if it stays disconnected state for long time
0.3.43
------
- Log metadata creation handles missing log identifer
0.3.42
------
- Bump birdcage release to 20160128130022-e7bf469
- Add markEndOfStream to log writer
- Conversion of many synchronous/blocking operations to asynchronous
0.3.41
------
- Bump bkc to 4.3.2
- Shard level resource limits in service
- Dynamically configurable readahead settings
- Categorize and cleanup configuration
- Bug fixes for non-durable write mode, log enumeration, readahead cache
- Fix some flaky tests
- Add docbird docs
0.3.40
------
- Reorganize local run scripts to support full service config
- Fix emulator timeout bug
- Break service layer into Stream, Service, StreamManager
- Add some stream scope stats
0.3.39
------
- Sync Reader uses Async Reader
- Write proxy returns if failed to pre-execute the stream operation
- Write proxy checksum
- Upgrade zookeeper version
- LogSegmentSequenceNumber Assignment should consider v3 rolling back to v2
- Merge Search Release to master
- SyncLogReader should start from the fromTxnId
- ReadAhead should detect log deletion
- Docbird structure
- Unify per stream stats logging
0.3.37
------
- Remove partitions from DL
- Move DistributedLogExample to distributedlog-example package
- Add getLastLogRecordAsync to log manager
- Remove unpartitioned from log writers
- Skip ledger number validation if the ledger metadata doesn't contain ledger sequence number
- Rename LedgerSequenceNumber to LogSegmentSequenceNumber
- Using BKAsyncLogReader to implement BKSyncLogReader
- LogSegmentWriter should close ledger if the log segment isn't in error state
- Async positioning reader by transaction id
- Move log creation and initialization logic out of log handler constructors
- Don't retry invalid stream name exception on acquiring ownership
- Checksum between client and proxy for write
- Move DistributedLog Write Proxy to use TwitterServer
- Update fiangle and utils dependencies to scala 2.11
0.3.36
------
- Remove partitions from DL
- Add getLastLogRecordAsync to log manager
- Remove Unpartitioned from log writers
- Skip ledger number validation if the ledger metadata doesn't contain ledger sequence number
0.3.35
------
- Fix syncownership elapsed time issue
0.3.34
------
- Add validation on server configuration
- Remove server mode from distributedlog proxy
- Fix wrong scope for client operation stats
0.3.33
------
- Make per stream thread as daemon thread
- DistributedLog Service Layer Refactor
- Binding changes for federated namespace
- Enforce ledger sequence number validation when get list of log segments
- Graceful shutdown period for write proxies
- Handshake should detect service status
- Non decreasing transaction id support
0.3.32
------
- Dynamic Config Mechanism
- DistributedLogManagerFactory cleanup for Federated Namespace
- Abort & Close Fixes
- Delay cache removal after service timeout
- Request rate limiting in write proxy
- Client Refactor
- Periodic Proxy Handshake
0.3.31
------
- Routing context for tried host history
- Propagate flush delayed exception when minDelayBetwenFlush is enabled
- Disable failure accrual on distributedlog client
- Async readBulk interface for reader
- Update BKC to 4.3.0-TWTTR
- Metadata upgrade changes
- Get subscription states for DL stream
0.3.30
------
- Sequence ID Support
- Handle disconnect between lastTxId/lastDLSN in metadata and contents of the log segment
- Address lock race conditions
- Validate stream path
- Truncation fixes
- Region based stats on distributedlog lock
0.3.29
------
- Provide zookeeper watcher manager to manage watchers
- Tests improvements
- Write LogSegmentLedgerMetadata with specific version
- Feature provider support from bookkeeper client
- DistributedLogCluster util for run all dl components
0.3.28
------
- Update cached log segments after setting truncation status
- Remove host from hosting service if client hits channel exceptions
- Dynamic option to disable durability enforcement
- Immutable distributedlog client builder
- getAsyncLogReaderWithLock position read from last commit point
- DistributedLock handle znode left by previous expired lock
- Logging improvement and misc bug fixes
0.3.27
------
- Service Level Timeout
- Stream Ownership Filters
- Pin Stream Ownership Feature
- Heartbeat with control records
- Idle reader detection improvements
- Misc fixes for high pendings
- Lock Timeout
- Reader Lock Deadlock Fixes
0.3.26
------
- BookKeeper Version 4.2.22
- Explicit Truncation Point Support
- Compression Support
- Handle missing zk notification
- Misc changes on fixing high pendings
0.3.25
------
- BookKeeper Version 4.2.21
- AsyncLogReader stall detection
- DistributedLog self lock fix
- Ledger Allocator throttling
- DistributedLog Thrift Mux Support
- Remove blocking calls in AsyncLogReader
- Misc AsyncLogWriter fixes
0.3.24
------
- Ability to block operations to a stream on proxies
- Synchronous close ledger handle
- Backpressure via write queue length
- DistributedLog Client Changes to Support Rebalancing
- BookKeeper Version 4.2.20
- Fix transactional add and delete log segment
0.3.23
------
- Fix AppendOnlyStreamWriter empty log bug
- Add StatsFilter to track pending requests issues
- Add a tracker (exposing stats) to track readahead worker progress
0.3.21
------
- Reader Lock Implementation
- Cancel pending writes when the stream is marked in error
- Handle write bursts using mininum delay between flushes
- Scala 2.10 upgrade
- DistributedLogClient creation waits for initial binding of the Name
0.3.20
------
- Options to disable gap detection and alert on gaps
- Support for getFirstDLSNAsyc
- ConsistentHashRoutingService supports Names correctly
- NameServerSet supports WeightedSocketAddresses
0.3.19
------
- Additional stats on the reader
- Fixes for DistributedLock
- ZK ACL support
- Remove reference counting from the ZK/BK Clients
- Deferred Truncation (Truncation Support and Reader Validation)
- Do not cache control records in the read ahead cache
0.3.18
-----
- Bulk(Batched) Writes
- Idle Reader logging and Force Blocking Read
- Support For Finagle Names
- Async GetLastDLSN
- Deferred Truncation (Metadata Changes)
- Upgrade BookKeeper Client version
0.3.17
-----
- Fix deadlock on rolling log segments
- Fix Stale Mapping
- Handle log segments being delete or completed during enumeration
- Handle runtime exceptions and log un-handled exceptions in the reader
- Alert on various conditions that indicate reader missing log records
- ZK Rate limiter
- Separate ZooKeeper connections used for reads and writes
0.3.16
----
- Fix Write Filter
- Tools to fix inprogress log segments with lower sequence number
- Split DL Modules
0.3.15
----
- Merge changes from master back to AsyncLogReader
- Write Control Record for Heartbeat
- Fixes for proxy shutdown
- Fixes for duplicated log segments
0.3.12
-----
- Additional stats on readers/read ahead
- Fixes for proxy restarts
- Fixes for bookie restarts
0.3.11
-----
- Remove blocking calls in the async reader
- Additional Stats
- New Finagle Settings for the proxy
0.3.10
-----
- New BKC Release (Preliminary changes for region aware placement and reordering)
- Twitter DNS Resolver
- Initial DLSN
0.3.9
-----
- Upgrade Guava 16
- Change stats to micro seconds
- Read Last Add Confirmed Long Polling
- Piggyback one entry with LAC
0.3.8
-----
- Ledger Allocation
- BKLogPartitionWriteHandler optimizations
- Flag to enable/disable sanity checking txn id
- Flag to check and create stream if not exists
- Throttling ledger rolling speed in DL
- Resilient to zookeeper exception in writer path
- TryReadLastAddConfirmed support
- Retryable ZooKeeper Client
- Region ID support for log segment
- Namespace & LogSegment Listener
- Monitor Service
- LogSegment Filter
- Recovery in background
0.3.7
-----
- New bookkeeper release
- Async getLastDLSN
- Fix for non blocking readers
- Changes from master upto 0.2.12
0.3.6
-----
- Expose DistributedLog Client Builder in Proxy Builder