forked from hypertable/hypertable
-
Notifications
You must be signed in to change notification settings - Fork 0
/
CHANGES
1379 lines (1292 loc) · 74.6 KB
/
CHANGES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Version 0.9.5.0.pre6:
(2011-06-03)
Fixed bug in MaintenanceScheduler introduced w/ merging compactions
Fixed bug in the FileBlockCache wrt growing to accomodate
Added support for DELETE lines in .tsv files
Added check for DFSBROKER_BAD_FILENAME on skip_not_found
Added --metadata-tsv option to metalog_dump
Fixed bug whereby get_table_splits() was returning stale results for previously dropped tables.
Added MaintenanceScheduler state dump via existence of run/debug-scheduler file
Fixed FMR in TableMutator timeout
Version 0.9.5.0.pre5:
(2011-05-18)
Changed Group Commit table hash_map to ordered map and fixed memory leak
Upgraded Hive jars to CDH3U0 distribution.
Added support for Hypertable.RangeServer.CellStore.TargetSize.Minimum
Fixed compaction maintenance staging skip for non-minor compactions
Fixed split-merge-loop10 and ac-garbage-compaction tests
Set NO_LOG_SYNC in HqlInterpreter unconditionally for LOAD DATA INFILE
Cleaned up MapReduce job conf properties and Fixed ROW interval spec
Auto-increase block cache size to accomodate query load
issue 618: Fixed NULL pointer dereference in AccessGroup::run_compaction
Added Hypertable.RangeServer.BlockCache.MaxMemory config property
Strip base filename from scan files in METADATA Files column
Upgraded apache_log and freebase examples to use post namespace API.
Fixed inconsistency in timestamp parsing between LDI and Hql shell insert command.
Andy's EnsureUnused patch (disabled by default)
Fixed FindRRDtool.cmake to work on mac
Version 0.9.5.0.pre4:
(2011-05-01)
[Issue 617] Fixed Superfluous RangeServer::create_scanner() request when scanning over exact range
Removed benchmark tasks from Capfiles, added Capfile.benchmark
Added support for ScanSpec in Hadoop streaming mapred
Added hostname to AsyncComm proxy map
issue 528: Allow embedded semicolons in HQL string constants
Added optional support for jemalloc
Fixed bug in scanner which causes assert when a scanner hits row limit and receives results from a cancelled scanner
Missed checkin of auto-generated java code
Fixed CellStore GC regression; Added test
API Change: Made Table objects auto-refresh schema by default
Added more info to RS_METRICS (including disk_bytes_read)
Support for optionally including master location hash into RangeServer proxies
Added support for IP addresses to be used as Hyperspace.Replication.Host strings.
Added is_ipv4() method to InetAddr to test if a string is an ip addr in the n.n.n.n format.
Fixed MergeScanner bug which failed to use MAX_VERSIONS filter correctly when value regex was used.
Added '\0' escaping/unescaping to SELECT, DUMP TABLE, LOAD DATA INFILE
Removed trailing slash on exclude hyperspace in Capfile rsync commands
Added Hyperspace.Client.Datagram.SendPort property (default=0)
Fixed bug in scanner timeout logic. Timers were not being reset correctly in between calls.
Removing dependency on BerkeleyDb libraries in Hyperspace client library.
Fixed Capfile install/upgrade tasks to work with standalone installs
Fix for rounding error in LOAD DATA INFILE timestamp parsing
Fix for KEYS_ONLY scan
Fixed bug in MergeScanner logic.
Version 0.9.5.0.pre3:
(2011-04-12)
Modified stop-servers.sh to stop master and rangeserver iff client progs exist
Upgraded to CDH3u0
Fix for recent ensure unused physical memory patch
Version 0.9.5.0.pre2:
(2011-04-11)
Ensure amount of unused physical memory for RangeServer
Fixed scanner deadlock when RangeServer dies or throws exception in create_scanner request.
Fixed ThriftBroker timeout problem; Removed default value for ThriftBroker.Timeout
Added --nomd5 option to rpm command
Got rid of role for install tasks in Capfiles
Fixed bug which causes assert when error message length > int16_t limit.
More Capfile changes.
Changes to Capfile. Replaced "dist" task with "install" and modified "upgrade".
Lazily create RangeLocator in RangeServer
Fix for recent changes to ApplicationQueue
Fixed bugs in TimeInline parse_ts function; Added regression test
Fixed problem with recent changes to ApplicationQueue
Put shell into "batch mode" when commands supplied with --execute
Set default RangeServer work thread count back down to 50
Modified ApplicationQueue to create thread on urgent task if all busy
Added server-side timeout during initialization; Increased worker threads
Fixed RSML upgrade logic
Fixed problem with SystemInfo::DiskUsage computation
Issue 598: Common-properties and Common-init tests failing on some platforms. Fixed.
Fixed shutdown problem with TableMutatorIntervalHandler
Fixed Range load_acknowledged problem when upgrading from pre-0.9.5
Fixed release string verification regex; Added print functions for MaintenanceData
Fixed hang problem with serverup dfsbroker
Fixed some bugs with recent monitoring changes
Fixed RangeServer freeze during initialization with low memory condition
Fixed typos in VersionHelper.cmake
Fixed MasterClient-TransparentFailover test
Fixed some issues with monitoring graphs
Fixed bug in Future class.
Use existing app queue for metadata/metrics table
Added "Range Count" to range server monitoring table
Fixed low memory freeze problem
Stopped Master GC from adding unnecessary deletes into METADATA table
Made RangeServer print error and exit on listen error
Fixed bug in ScanCells interface whereby it was possibly to dereference a null pointer.
Fixed monitoring bug, changed installation dir from "0.9.4.3" to "current"
Fixed problem resulting from lost load acknowledge response
Fixed bug in future_abrupt_end_test.
Fixed Master DispatchHandler outstanding tracking
Fixed deadlock in scanners.
Fixed bug in scanner logic.
issue 585: Copy new METADATA.xml and RS_METRICS.xml in fhsize.sh
if on startup MML is empty and /sys/METADATA exists, don't try to create it
Fixed shutdown order sequence
Fixed RangeServerConnection state persistence problem
Updated CHANGES file for 0.9.5.0.pre release
Fixed scanner bug where once a scan was cancelled scanners were being set current out of order.
Version 0.9.5.0.pre:
(2011-03-24)
Master overhaul
MetaLog overhaul
Asynchronous Scanner API
Upgraded to Thrift 0.6.0
Upgraded to CDH3B4
Added sys/RS_METRICS
Fixed bug in monitoring system that was calculating buggy Cell and Byte read/write rates.
Added METADATA-split master failover tests
Added MasterClient-TransparentFailover test; fix bugs that turned up
Added VERSION_MISC_SUFFIX to version string for 0.9.5.0.pre release.
Added delete_count to CellStoreV5Trailer which stores the number of delete records in the CS.
Added list of replaced files to CellStoreV5.
Fixed soft_limit regression
Fixed intermittent test failues due to exit(); Got rid of valgrind warnings
Added two-phase master requests
Fixed warnings
Upgraded version number to 0.9.5.0
Cleaned up prune threshold limits; Got rid of warnings
Fixed deadlock in ResponseManager
Fixed data loss bug - made CommitLog close synchronous; fixed async scanner bug
[Issue 579] metalog backup verification causing intermittent test failures. Fixed
Added regression tests for stopping synchronous and asynchronous scanners abruptly before scan completes.
Fixed deadlock in TableScannerAsync code.
Added needs_compaction flag to RangeServer::load_range method.
[Issue 578] Deadlock in async scanner. Fixed
Added ht_master_client shell program with shutdown command
Fixed hyperspace-reconnect test
Fixed monitoring server initialization problem
Minor fix to dot jpg file generation.
added start_time,end_time as http query params
Close cell store file before removing directory
Create <data dir>/run folder if required
Fixed a bunch of minor issues.
[Issue 577] RangeServer::commit_log_sync should respect group commit. Fixed
Added regression tests and bug fixes for Future API.
Performance and functional bug fixes to TableScanner class.
Implemented changes to C++ and Thrift clients to support asynchronous scanners. -TODO: add tests for Php and Python
Fixed a bug that was causing the METADATA-split-recovery test to fail intermittently.
issue 552: Ensure Hyperspace handles get closed; Naming cleanup
Fixed incorrect CellStoreTrailerV5 version check caught by assert
Added CellStoreV5; Monitoring system improvements (avg. key & value size)
added new column to table stats
Added file_count to StatsTable; Fixed compression ratio computation
Got rid of read_ids flag in Schema parse API
"changes to header labels"
Monitoring UI Changes, Sorting options for stats summary
Updated clean-database.sh script to reflect new rsml backup location
added invalidate methods for table name changes
bunch of changes to Monitoring UI changes (reading from json and got rid google graphs which gives summary) Added Ta
Fixes to monitoring & stats gathering
Fixed bugs caught by Andy Thalmann (ScanContext copy ctor bug)
added table names to json , removing unnecessary code
Fixed minor monitoring/stats gathering bugs
issue 563: fixed METADATA split test
bunch of changes to Monitoring UI changes (reading from json and got rid google graphs which gives summary) Added Ta
Added Hypertable.RangeServer.CellStore.SkipNotFound
issue 559: Prevent transfer log from getting linked in twice
issue 552: Ensure Hyperspace handles get closed; Naming cleanup
[Issue 505] Client-no-log-sync regression failure
issue 537: Fixed RangeServer shutdown hang
issue 542: Only write last-dfs file if it doesn't exist
issue 544: Set default RangeServer memory limit to 50%
issue 553: Schema HQL render wrap table name in quotes if necessary
issue 545: Reduced random-write-read test to 1/5 the size
[Issue 551] Upgraded to QuickLZ 1.5
Fixed bugs related to MasterGc and live file tracking
Renamed MoveStart and MoveDone RSML operations to RelinquishStart and RelinquishDone.
Made changes to BalanceStarted/Done and RangeMoveLoaded/Acknowledged MML entries.
Added code for additional MML entries.
Changed MasterMetaLog to garbage collect entries during recovery.
Added MoveStart and MoveDone RSML entries.
Changed RangeServerMetaLog to garbage collect entries during recovery.
Fixed rare bug that caused root range corruption
issue 547: Found and fixed more race conditions
Fixed excessive maintenance scheduling during low memory condition
Fixed deadlocks uncovered recently
Fixed race cond in drop_table; Fixed RangeServer::update bug
Changed DfsBroker.Local.DirectIO default to false
Fixed some stats gathering issues discovered in sys/RS_METRICS
issue 531: Fixed bug in load_generator that caused intermittent drop of last cells
Added RangeServer::relinquish_range() (with RSML update code stubbed out)
Added Hyperspace::Session::open() method with no callback; code cleanup
Use "scanned" cells/bytes for load metrics; Added paging info to load metrics
Fixed recently introduced "bad ProxyName" problem
Do not reduce the limit below the minimum
Scan and filter rows fix and optimization
Config property "Hypertable.RangeServer.LowMemoryLimit.Percentage" has been added
Added disk_used, disk_estimate, and compression_ratio to StatsTable
Fixed revision number problem in CommitLogReader with link entries
Renamed Master::report_split() to Master::move_range()
Added MML
Fixed monitoring stats; fixed JSON output for RS summary
Monitoring overhaul part 1
new rangeserver stats
changes to use new rangeserver summary data
Destroy comm has been fixed
HQL scan and filter rows option added
Logic changed to setup row intervals in case of scan and filter rows
Optimization for scan and filter rows
Scan and filter rows has been implemented (Issue 525)
check for negative resolution
issues with config
Added resolution param to rrd page
Fixed monitoring stats; fixed JSON output for RS summary
Monitoring overhaul part 1
Check for zero-lengthed row and skip in LoadDataSource
Allow NULL strings to be passed into FlyweightString
Fixed bad memory reference in RangeServer::FillScanBlock
cleanup has been added
Added prepend_md5 tool
Redirect thrift output to HT logger
Assignment operator added
Support empty qualifier filtering
Recursive option added to the hyperspace readdirattr command
Recursivly option added to the hyperspace readdirattr command
Include sub entries for get_listing/NamespaceListing, readdir_attr/DirEntryAttr
Fixed syntax error recently intoduced into Ceph broker code
Added StrictHostKeyChecking=no to rsync
Added regexp filtering to DUMP TABLE command.
Added optimization for row and qualifier regex matching.
Added script to compare test runs times and detect potential performance regressions.
Cleaned up SELECT [CELLS] Hql command.
Removed DfsBroker.Host from default hypertable.cfg; Cleaned up DFS Port properties
Fixed bug in Hyperspace caused by Reactor thread directly calling BerkeleyDbFilesystem on disconnect.
Improved Master handling of already assigned location in register_server
Fixed performance regression in ScanContext by using set instead of hash_set for exact qualifiers
Version 0.9.4.3:
(2010-11-15)
issue 534: Made RangeLocator thread-safe
MetaKeyBuilder crash with nested namespces fixed
Got rid of spurious "Connection refused" error messages
Decreased default bucket count to 20 for DUMP TABLE
Added group commit
Added regex filtering on ROW, COLUMN FAMILY, and VALUE
Modified Capfiles to start ThriftBroker on master
Added code to catch BAD HEADER LENGTH exception in IOHandlerDatagram
Fixed unprotected access to Range stats data
Changed regexp members of ScanSpec to use memory arena
Added more documentation about how to LOAD DATA INFILE from stdin
Added support for filtering by qualified column and remove restriction of HT rowkey==first col in Hi
Added the ability to set multiple column qualifier filters for a given column family.
Added retry logic to LiveFileTracker::update_file_column. -Added regression test, commented out for
issue 247: Added methods to RangeServerClient that accept Timer
Fixed bug in AsyncComm poll() mode
Evaluate --hyperspace host:port argument
Fixed problems in MetalogRangeServer and FileBlockCache tests (Andy Thalmann)
Fixed bug in SerializedCellsReader; Added more test scripts
Toplevel directory added
Added parallel INCR test to PerformanceTest; Increased ThriftBroker timeouts
Added instrumentation to group-commit tests; Fixed TestHarness
Lazy fix for issue 530
Crash fixed (happens for MutatorNoLogSyncTest)
Close open file handles before delete file
Hypertable.DataDirectory (Hypertable data directory root) configuration parameter added to the Defau
Added RE2 dependency to PackageHelper
Added API support for row, value and column qualifier regexps. -Added API support for row, value and
Added dependency & cmake check for RE2 regex library.
Fixed error in CREATE NAMESPACE documentation
Version 0.9.4.2:
(2010-10-26)
Upgraded to Thrift 0.5.0
Added COUNTER type column family.
Fixed potential deadlock in periodic mutator code path.
Added missing rrd dependencies to packages
Pulled in Trent's fix for null values in TextTableInputFormat and InputFormat.
Support for cell flags (FLAG_INSERT, FLAG_DELETE_*) in thrift serialized cells
Make use of existing Common/endian-c.h
Read and set timeout for thriftbroker
Added optimization to increment CellCache counter in place
Updated PerformanceTest added support for increment
Fixed problem with IF EXISTS option to DROP NAMESPACE
Added clean task as dependency to jar task.
issue 518: Trim whitespace to avoid HqlParser error
issue 517: compilation problem with VERSION constant in CommHeader
issue 515: prevent PHP error reporting from being enabled unconditionally
Fixed ThriftBroker crash when opening bad namespace
Added resolution param to rrd page
Version 0.9.4.1:
(2010-09-23)
issue 516: fixed Table::get_name()
Monitoring README fix.
shell script changes
Removed config.yml.in Changes to start script.
changes to config.yml
using thin server now, fixed config.yml data dir issue, added progress bar on graphs page
Modified Monitoring UI start/stop scripts.
Fixed bugs in Hive extension introduced by thrift 0.4.0 byte[] -> ByteBuffer change.
Version 0.9.4.0:
(2010-09-08)
Added Namespaces
Added AG garbage collection logic for MAX_VERSIONS, deletes, and TTL
Hypertable Monitoring web interface using sinatra
Made changes to garbage collect unused Hyperspace BerkeleyDB logs.
Upgraded to Thrift 0.4.0
issue 501: Made Capistrano "upgrade" task first verify that upgrade is OK
issue 440: Renamed CellFlag to KeyFlag; Renamed put_ methods to offer_
issue 504: Store relative filenames in METADATA
Added RENAME TABLE feature.
issue 479: Added support for configurable toplevel directory name
issue 174: Changed all state file/dir names to include table id instead of name
Modified capfile to start and stop monitoring server on master.
Updated location of monitoring UI code.
Modified start and stop scripts for Monitoring UI
Made changes to print out ProxyMap to text file to be used for the Monitoring UI.
Fixed string encoding/decoding problem in Java code
Fixed stats gathering logic to handle variable length string table ID
issue 316: Moved CREATE TABLE options to the end after column defs
issue 487: Fixed Comm layer core dump with many client connections
issue 496: Fixed bad logic in EINTR handling
Changed namespace "SYS" to "sys"
issue 497: fixed problem of column qualifier not getting cleared in Thrift API
issue 11: Use shorter cellstore filenames. Fixed
Added hypertable shell regression test for namespaces. Updated some documentation.
Namespace Client API changes.
issue 174: Changed all state file/dir names to include table id instead of name
issue 234: Propagate better error messages from DfsBroker to RangeServer log
issue 106: Use JAVA_HOME to find java binary, if set
Got rid of verbose logging from recent AG garbage collection change - Also changed Hypertable.RangeServer.AccessGrou
issue 479: Added dual-instance regression test
Changed semantics of readdirattr and readpathattr Hyperspace commands
Added NameIdMapper class.
Made changes to store next table id in Hyperspace..
Added Hyperspace readpathattr API to list the value of an attribute for all components in a path
Added Hyperspace attr_incr API to atomically increment an attribute.
[Issue 148] hql parser doesn't like quotes inside a string even when escaped. Fixed.
[Issue 481] ThriftClient-reconnect-hyperspace failure. Fixed.
[Issue 491] core dump at get_ts64() of Time.cc. Fixed
[Issue 488] Hyperspace crash. Fixed
[Issue 492] RangeServer crash. Fixed
Fixed bug in Master stats collection.
Made changes to checkpoint BDB every time there is 1M worth on uncheckpointed log data.
Minor fix in FindRRDtool.cmake
Don't link Hyperspace.Master to tcmalloc for 32-bit
Version 0.9.3.4:
(2010-07-15)
Switched to CDH3; Made Ant and Thrift required
Updated some auto-generated files for new version of thrift
Updated HqlHelpText with CELL_LIMIT feature info.
Don't link ThriftBroker with tcmalloc on 32-bit
Added support for newer versions of Boost
Minor fixes to monitoring code.
Added cell_limit feature to scan/select queries.
Fixed link problem with serverup on Mac OSX
Added dependent RRD libs to install
Version 0.9.3.3:
(2010-06-23)
Fixed bug in fhsize.sh; Added fhsize task to Capfiles - Load libthrift jar file to CLASSPATH first
Minor fixes to namespace violations and use Table name in place of ID for monitoring.
Fixes to mapred Input/Output formats
Added upgrade task to Capfiles
Made changes to monitoring system to print Table name as well as ID.
Hypertable-Hive connector.
More tuning based on perfeval test1 results
Changed ThriftBroker threading model to threaded instead of thread pool
Version 0.9.3.2:
(2010-06-12)
Changed some defaults based on perf tests
Added range readahead in IntervalScanner - Report info about actual data read in Performance test driver
Upgraded to thrift-0.3.0-rc4 from http://people.apache.org/~bryanduxbury/thrift-0.3.0-rc4.tar.gz
Initial version of Monitoring application
Fixed Sigar library name for OSX.
Modified Thrift get_schema API to return Schema object and added get_schema_str API
Fixed TableSplit.java to lookup hostname for location since Hadoop doesn't understand IP addr
issue 356: Record DFS is run/last-dfs file and report error if mismatch
Fixed race condition in Master::get_statistics()
Fixed timeout problems with ThriftClient
Fixed compile problem in ThriftBroker; Added run-benchmark.sh script
increased transfer buffer
Added support for Zipf, report output file, etc.
Added rrd library to install; Fixed race condition in Client table caching
Increased interval scanner readhead buffer size 5x512K
Fixed bugs in ThriftBroker next_cells_serialized and next_row_serialized APIs
Added next_row_serialized API to ThriftBroker.
Added system wide stats like #cores, disk , mem utilization, load avg to monitoring stats.
Added code to blow away monitoring dir when clean-database.sh is run.
Added PerformanceTest benchmark framework - Switch ThriftBroker to TThreadPoolServer on linux - Added _serialized me
Added support for DIRECT I/O
[Issue 464] Add refresh_mutator API. Added refresh_shared_mutator API to Thrift interface
[Issue 86] Monitoring.
Added Master maintenance timer
Fixed circular dependency in build.
issue 414: added thriftbroker and spare Capistrano roles - Also included DfsBroker in the hypertable-thriftbroker pa
added the ability to specify start/end row with streaming fixed the handling of splits with regards to maps
Added Hypertable.RangeServer.Scanner.BufferSize
Set key_compression_scheme propery in CellStoreV3 trailer
issue 427: Added expiration_time and expirable_data to CellStoreTrailerV3
issue 149: Added prefix key compression to CellStore (v3)
[Issue 259] LDI should be able to load data from DFS. Updated documentation.
Version 0.9.3.1:
(2010-04-14)
added support for map reduce streaming
Fixed prune_tsv to work with DUMP TABLE format, added regression test
issue 449: Remove fhsize from RPM and DEB post-install
[Issue 448] Hypertable shell trying to connect to DFSBroker. Fixed.
issue 439: Strip duplicate cells during scan
issue 443: Check for invalid ':' character in column family names
issue 442: Escape/unescape row key and column qualifier
Added Dfs support to DUMP TABLE command.
[Issue 416] Commands taking long time to run. Fixed
[Issue 420] Incorrect checking of errno after calling FileUtils:: function. Fixed
issue 436: Fixed problem causing full table scan for each split
issue 408: Fixed problem where scanner didn't propagate revision number
issue 430: Fixed STL comparator semantics
issue 385: Made REPLICATION factor configurable per-table and access group
[Issue 175] HQL parser getting confused on INSERT of 4-digit row key. Fixed.
issue 124: Documented LDI options IGNORE_UNKNOWN_COLUMNS and DUPLICATE_KEY_COLUMNS
issue 260: Display useful error message when LDI HEADER_FILE not found
issue 419: Fixed race condition in Comm::connect HandlerMap access
issue 421: Fixed Free Memory Read bug in ProxyMap
[Issue 395] Client-no-log-sync test fails intermittently. Fixed
[Issue 381] HqlInterpreter::execute(...) pass reference to output parameter.
issue 378: Added escaping of '\' in addition to \n and \t
issue 423: Fixed MetaLogDfsBase::get_filename()
issue 424: Added destructor to MetaLogReaderDfsBase to close file handle
issue 402: Lazily load CellStore block index for faster startup time
Fixed warnings on Linux x86_64 gcc 4.1.2
[Issue 259] Checked in changes to allow LDI and SELECT to read/write from/to DFS.
issue 136: Fixed bug in DfsBroker ClientBufferReaderHandler
issue 403: fixed Coverity bugs
issue 417: Allow quoted table names in INSERT command - Added regression test (verifies issue 34 as well)
issue 34: Fixed bug in ScanContext causing minimum timestamp to be 1970-01-01
issue 422: Fixed bad memory reference after copying Managed objects
Fixed minor (innocuous) problems in Comm layer
issue 411: fixed priority_queue less comparator
issue 410: fixed reverse proxy map invalidation
Fixed double file handle close in HQL interpreter
issue 406: Check iterator against appropriate map structure
issue 404: close file handle in FileUtils::file_to_buffer
Fixed FindThrift.cmake to look in /usr/lib64
Added libstacktrace to package
Fixed Capfile rsync_installation to work with FHSized install
Version 0.9.3.0:
(2010-03-22)
Added Hyperspace replication
Added MapReduce connector
Added Query Cache
FreeBSD port
Solaris port
Added DUMP TABLE
Added LOAD DATA FORMAT input file format auto-detect
Fixed memory leak
Added shadow CellCache
Added poll() support to Comm layer for Linux 2.4 kernels
Exclude CellStores that are outside of scan spec timestamp range
Improved efficiency of KeyWritable; added comments
Fixed race condition with proxy map propagation
Updated htbuild
Added missing ssl library to CephBroker build
Added --thrift flag to ht_load_generator tool to test loads via ThriftBroker.
Changed PerformanceEvaluation tool to generate random buffer once and re-use it to lower CPU load
Increased ThriftClient timeout to 10min for MR PerformanceEvaluation task
Changed Hyperspace lock operation result log from debug to info.
Fixed recursive includes in hypertable jar files that were bloating the file size
Modified path for Java Client examples and build.xml paths for examples jar file
Added Hypertable PerformanceEvaluation class.
Fixed duplicate server location assignment problem
Added Key class to Thrift, make Cell a composition of Key and value
Plumbed the Client::get_table_splits API through the ThriftBroker
Checked in OutputFormat for Hypertable MapReduce connector.
Added 'exists_table' API to Hypertable client, Thrift interface and HQL.
Added INSTALL_EXCLUDE_DEPENDENT_LIBS cmake variable
More FreeBSD porting work
Turned off ShadowCache by default
Made poll error handling more graceful (return error instead of exit)
Added Hypertable.RangeServer.AccessGroup.ShadowCache property to enable/disable shadow cache
Workaround for xen timestamp problem; Fixed Master deadlocks
Fixed memory leak in maintenance queue
Fixed deadlock in Master
Added VERSION_ADD_COMMIT_SUFFIX cmake variable
Added address proxy
Got rid of boost spirit warnings
Fix to allow 0 column family for retured cells that are DELETE_ROW
Auto-adjust blockcache size based on workload - Maintenance scheduler re-work
Fixed progress meter for large load files and generated data sets
Fixed --max-keys handling for ht_load_generator - Also respect min_pool and max_pool for Uniform random number generator
Got bloom filter working again - Added new CellStoreV2 format with all necessary bloom filter state in trailer - Added --num-hashes and --bits-per-item bloom filter options
Cleaned up FailureInducer interface
Fixed METADATA split recovery race conditions
Cleaned up regression tests; Updated documentation
Made Master::server_left() close connection to avoid race condition
[Issue 388] Fixed issue by checking for session existence within BDB txns.
Setting replication site priority according to order in which replicas are listed in config file
Fixed bug in Hyperspace logic which was causing Hyperspace to delete a handle for an expired session.
Made changes to avoid duplicate masters among BDB replication sites.
Modified Hyperspace replication logic to wait for permanent message acks from all replicas.
Modified Capfiles with new hyperspace replication stuff
[Issue 372] Minor cleanup, changed error message to info. -This is an expected case and not really an error
Upgrading BerkeleyDB requirement to >= 4.8.x
Replaced HT_EXPECTs in BerkeleyDbFilesystem code with HT_ASSERTs
[Issue 372] Added scope guards to prevent BerkeleyDB cursor leaks. -Also changed a few asserts to HT_ASSERT
Made changes to allow Hyperspace clients to connect to any Hyperspace replica and get redirected to master.
Made changes to BerkeleyDbFilesystem so that each txn opens its own DB handle to avoid DB_REP_HANDLE_DEAD errors.
Changes to implement basic replication
Fixed problem preventing CellStore indexes from getting GC'd
Leave AccessGroup in consistent state after compaction failure
Fixed Memory Corruption in MetaLogDfsBase
Fixed infinite loop on ROW_OVERFLOW
Fixed deadlock
Apply MAX_VERSIONS unconditionally
Added RSML backups in $INSTALL/run/rsml_backup/
Fixed problem with "LOAD DATA INFILE" from STDIN where incorrect progress bar update was the load down.
Updated version number to 0.9.2.8
Fix for Hyperspace race condition; Added compaction asserts
Fixed delayed connect problem
Updated system tests; Increased BlockCache default size to 150M
Default dfsclient timeout to Hypertable.Request.Timeout
Fixed memory leak; Added support for tcmalloc HEAPCHECK
Added shadow CellCache; Re-wrote maintenance scheduler
Allow .tsv files with empty values
Issue 358: be a little smarter about default config file path.
Fixed problem with soname.sh and libsigar
Workaround for clock()/poll() incompatibility
Added support for cmake variable HT_DEPENDENCY_DIR
Modified BerkeleyDb search path to look for 4.8 release if available
Changing BDB_DEADLOCK messages to info instead of warning.
Added keys_only to Thrift ScanSpec interface
Issue 361: Allow RangeServer to reconnect with Master after connection reset
Upgraded to Thrift 0.2.0
Added put_cell* API to Thrift interface to give easy access to shared periodic flushing mutators
Removing BDB deadlock error messages. Use warnings instead.
Fixed compile error on mac
Updated htbuild for thrift r830673, kfs 0.4 and ceph 0.17 - Also added fix for python thrift installation on ubuntu
Made Cells and ScanSpec fully take advantage of custom allocator - Made clear() semantics for the containers consistent as well.
Consolidate pool allocators and their usage.
Updated doxygen doc version
Cleaned up some compiler warnings under gcc 4.3.
Fixed an fmr for valgrind
Added some test helper functions/classes/templates.
Made property accessors const where appropriate - also tweaked the default config file to use relative path if possible.
Temporarily added back thrift perl dependencies to the rpm package - probably better off zip the perl examples in the distribution.
Added thrift version ht -v
Added help/usage info for the ht wrapper
Added basic regression test init_* function. - Caught int32_t masking issues on Solaris
htbuild: update for fedora 11
Fixed unknown option error from serverup for non-default options.
Fixed hql result logging in ThriftBroker
Upgraded thrift to r830673 to fix stack smashing in ThriftBroker - added slf4j jars for the new thrift - added java-thrift.sh for easy launching jhava thrift clients
Added support for Hypertable.Network.Interface config property
Added tsv-format and column-id-map options to csdump
Added Query Cache - Added .values property to load generator spec file to indicate number of distinct values to generate
Fixed RangeServer::load_range RSML write error handling to leave system consistent
[Issue 355] Fixed and added regression for LDI timestamp issue.
Fixed load_generator to support MaxKeys for load type 'update'
Added CellCacheScanner entry cache - Also fixed some build issues on Snow Leopard
Fixed problem where load_generator not respecting MaxBytes for query
Fixed TableMutator::set_cells to throw BAD_KEY exception for cf/cell deletes without a column family.
Replaced sleep with perl select() command in random-wait.sh
Fixed bug in ConnectionManager::remove - Fixed broken regressions
Fixed trailing '\n' problem in prune_tsv
Fixed CLOSED_WAIT problem with serverup and HdfsBroker
[Issue 340] Added Thrift MutatorFlag IGNORE_UNKNOWN_CFS.
Added "--exec/-e" and "--command-file" switches to CommandShell to execute commands.
Fixed memory corruption issue triggered when the interval mutator fails on successive writes.
Added --newer switch to prune_tsv
Fixed bug in Schema rename method
Fixed SIGAR library detection on mac Snow Leopard
Fixed connection/retry problems surfaced on Solaris
Fixed int32_t overloading issue on Solaris i386.
Change all tests/integration scripts to explicitly use bash
Worked around RAND_MAX quirk on Solaris
Worked around quirks in cmake on Solaris
Added table name to error message and modified Table & TableIdentifier classes to expose name.
[Issue 340] Added "ALTER TABLE RENAME COLUMN FAMILY" feature to rename column family names.
[Issue 340] Added ignore unknown cfs feature to LDI.
Changed Hypertable client Hyperspace connection check interval to 5s instead of 3s.
Fixed CellStoreScanner to turn off readahead mode in case of single row scans.
Commited changes to allow a client to automatically refresh stale schema during certain ops.
[Issue 339] Added instrumentation to debug this issue and fixed empty bloom filter problem.
Got rid of Hypertable.Request.Timeout from default hypertable.cfg
Got Master::close() plumbed through properly - Got rid of some warnings in Hyperspace - Made performance improvements to prune_tsv
Commited changes to allow a client to automatically refresh stale schema during certain ops.
Added flag to allow Sessions to reconnect to Hyperspace instead of expiring.
Major Hyperspace overhaul to store Session, Handle, Node and Event data mostly in BerkeleyDb.
Added prune_tsv tool
Fixed sequential-load test script
htbuild: exit ASAP in case of errors
Fixed tcmalloc version check (should run every time)
Fixed FindCeph.cmake for the new location of libceph.h in 0.14
Updated boost to 1.40 in package build - added ntpdate to htbuild to avoid make timestamp issues - fixed tcmalloc build problems introduced by last refactor
Version 0.9.2.7:
(2009-09-18)
[issue 333] Fixed data loss problem after recovery
[issue 332] Fixed problem in IntervalScanner due to delayed connection establishment
[issue 331] Fixed post-recovery split problem
[issue 323] Fixed problem that caused unnecessary work leading to thread exhaustion
Upgraded to Hadoop 0.20.1
Added support to run regressions on different DFSs
Updated tcmalloc to 1.4 for package build
Added CephBroker for Ceph DFS (http://ceph.newdream.net/)
Added RS_LOG_RECOVER entry to RSML after recovery to indicate log validity
Added htpkg utility script for building binary packages
Modified RSML to create new file on load; keep only last 10
Allow rpm build on Ubuntu/Debian machines
Added bin/src-utils/htbuild for building packages
Augmented htbuild to allow easy setup of dev environment
[issue 333] Fixed data loss problem after recovery
Added instrumentation to help track down bad BloomFilter creations.
Added test for out-of-order revisions; Added rangeserver_dump to Capfiles
Added RangeServer::dump() method to dump stats to file
Added DUMP command to rsclient
Modified shutdown command to first close before shutting down
Fixed stop-servers.sh; Added close command to shell
Fixed NPE in count_stored program
More fix to packaging on Mac OS X.
Issue 328: Fixed broken implementation of next_row in Thrift interface.
Changed Hypertable.RangeServer.MemoryLimit.Percentage default to 60
Improved SELECT documentation
Added packaging and version info related to CephBroker.
Fixed problem introduced in ht-env.sh due to bash discrepancies
Cleaned up doc generating targets
Made prerm always succeed to avoid broken package states
Some font tweaks so that the pages look decent in alternative platforms
Fixed memory corruption problem in ht_load_generator
Removed obsolete demo directory and cleaned up some space errors
Only print javac version if there is one in ht -v
Made system libs id more portable with ldd.sh
Worked around a compiler bug in gcc 4.2
Made kfs version optional in ht -v
Use i386 as 32-bit x86 suffix instead of i686 from uname
Updated README for thrift and packaging
Build thrift broker package first to preserve regular build tree states
Add a build type suffix to the package if it's not a release build
Print more component version info in ht -v
Added quick version info in bin/ht
Added option for kfs broker to print kfs version
Install scripting language bindings for thrift even if thrift is not installed
Added libthrift.jar to lib (in the source tree)
Added a few convenient commands for ht for shared lib diagnosis
Added script to figure out soname on supported platforms
Made cmake generate binary package index html
Moved libthrift.jar from lib to lib/java for consistency
Renamed LICENSE to LICENSE.txt to appease the PackageMaker
Added convenient method to check ldd under ht
Added some missing components for thrift broker package
Made starting server with logging options more robust
Avoid globbing hidden files (.svn etc.) in conf directory.
Made the FHSize script idempotent
Warn about prelinked shared libraries
Fixed conf packaging for thrift broker only build
Added HypertThriftConfig back into list of targets for ThriftBroker install.
Version 0.9.2.6:
(2009-08-15)
[issue 319] Fixed problem that prevented cell stores from getting written to METADATA Files column
[issue 315] Fixed scanner creation problem on restricted range CellStores
Fixed bug introduced by recent CellStoreScanner changes
Refactored startup scripts for robustness
Added support to create binary packages with minimal dependencies
Added markdown HQL documentation
Update HQL help text
Changed stop-servers.sh to invoke 'shutdown' via rsclient
Also added --no-hyperspace option to rsclient
Re-named rsclient to ht_rsclient
Changed Hypertable.Request.Timeout from 5 to 10 minutes
Output row and column instead of rowkey and columnkey for LDI header
Translate IP address to 127.0.0.1 when host resolves to 0.0.0.0
Made sequential-load/run.sh easier to run from command line.
Check returns of some system() calls to silence some compiler warnings.
Use objdump to get the correct SONAME for packaging
Included Capfile.* and removed 3+MB obsolete demo files from packaging
Added post install scripts to FHSize the directory layout
Fixed Schema::render_hql_create_table() to not included deleted columns and empty AGs
Cleaned up some noop code recently introduced in CellStoreScanner
Added TTL and MAX_VERSIONS to CREATE TABLE options
DESCRIBE TABLE no longer displays IDs by default; Added WITH IDS option
Updated README for binary packaging
Refactored init* functions from Config.h into Init.h
FindSIGAR: search for x86_64 as well as amd64
Prefer tcmalloc_minimal over tcmalloc
Added IN_MEMORY and BLOCKSIZE=<n> to CREATE TABLE options
Fixed bug in HT_DECODE_BYTES32
Version 0.9.2.5:
(2009-07-29)
Fixed memory leak in query path
Implemented purgeable CellStore indexes and bloom filters
Added mutator options to periodically flush buffers
Fixed bug in CommitLog that was causing some fragments to get dropped incorrectly.
Fixed bug in CommitLogReader caused by fragment queue iterator invalidation when log fragments are pushed to the back of the deque.
Rewrote Cell Cache/Store Scanner
Added MaintenancePrioritizerLowMemory used in low memory condition
Implemented mutator no flag sync feature and added regression test.
Fixed performance regression in IntervalScanner
[Issue 313] Added temporary fix and instrumentation for debugging.
Updated ruby tests for mutator API changes (The rescue block was masking exceptions)
Fixed load-exception test
Setting default commit log revision to TIMESTAMP_MIN instead of 0.
Fixed commit-log-gc test
Renamed ClientException::what to message for compatibility with most languages
Added hql_*2 APIs for returning cells as arrays in HQL queries (To support find_by_sql in HyperRecord)
Allow 'row' and 'column' for LDI hyperformat header
Regenerated files with updated thrift
Added --query-delay=<millis> option to load_generator
Got rid of some java warnings
Fixed KosmosBroker::pread to not modify current offset
Thrift related changes for periodic mutator flush
Rename a test label to follow the convention of the rest of the tests
Restart test servers automatically every time when make *tests
Added script to dump backtraces of running/hanging servers.
Allow VALGRIND to be defined as environment variable in start scripts.
Added missing php files for the no_log_sync flag.
Fixing occasional segfaults upon exit in some tests
Fixed occasional garbled version string.
Added system integration test which loads 5G of data, restarts the cluster and does a scan to verify the load.
Fixed infinite in-memory AccessGroup compaction loop
Lazy load bloom filters
Added rsclient 10x split/merge loop test
Applied changes to build with GCC 4.4.0
Fixed mutator log sync regression test and cleaning up dumplog.
Added Hypertable.HqlInterpreter.Mutator.NoLogSync property
Added command line switch --no-log-sync to hypertable cli for setting the aforementioned property
Fixed a couple of rare NullPointerExceptions in HdfsBroker
Added flags argument to ThriftClient open_mutator
Modified load generator tool to support no log sync and flush interval flags.
Fixed bug in commit log garbage collection that was caused some cells to be dropped.
Print out bad line number on LOAD DATA INFILE exception
Fixed ThriftClient-perl regression test
Send KEYS_ONLY clause over to RangeServer for processing server-side
Fixed test for Python Thrift module
Changed RangeServer.MemoryLimit.Percentage default from 80 to 70
Added size of block cache to memory tracker
Got rid of hardcoded 60 second DfsBroker.Timeout
Version 0.9.2.4:
(2009-05-25)
* Upgraded to Hadoop 0.20
* Added metadata-split test; Found and fixed deadlock
* Added --no-thriftbroker to start-all-servers.sh and include this option in integration tests
* Added property Range.MaximumSize (default is 3G)
* Renamed Range.MaxBytes to Range.SplitSize
* Updated to version 0.9.2.4; fixed some warnings
* Fixed performance regression in select command.
* Added CellStoreV1 with 64-bit support
* Fix for race condition in range size throttling
* Throttling is done by forcing the RangeServer::update call to wait for maintenance
completion on ranges which indicate that they are too big.
* Added API for getting hyperspace session from client object
* Checked in changes to Thriftbroker so that is works with recent Thrift version.
* Updated README to include location of new working thrift snapshot.
* Added support for selecting data into a compressed (gzip) file.
* Added regression test for select into gzipd file
* Added exit scope guard to cmd_select and cmd_load in HqlInterpreter so that the open output file is always closed.
* Refactored LoadDataSource class to make it an interface for different sources.
* Added derived classes LoadDataSourceFileLocal and LoadDataSourceStdin
* Added LoadDataSourceFactory to create LoadDataSource objects
* Added regression test for LoadDataSourceStdin
* Modified ServerLauncher to open output files in append mode.
* Added CellStoreFactory
* Added instrumentation to TableInfo
* Added CellStoreBlockIndex abstraction; Parameterized CellStoreScanner
* Fixed overflow problem with METADATA table
* Also fixed code to respect Hypertable.RangeServer.Range.MetadataMaxBytes
* Fixed serialization bug in Hyperspace client Protocol.
* Added support for attr_exists in the Hyperspace master for improved efficiency.
* Added regression test for attr_exists in BerkeleyDb test.
* Added Hyperspace attr_list API
* Added support for "attrlist" and "attrexists" in Hyperspace shell.
* Added Hyperspace shell regression tests for "attrlist" and "attrexists".
* Made LocalBroker::rmdir a little more robust
* Fixed problem that caused spurious "Hyperspace session expiration" errors
* Fixed memory corruption in CommitLogReder error handling logic
* [issue 246] Commit log not getting garbage collected
* Set URGENT bit on Hyperspace keepalive packets
* Set URGENT bit on some other RangeServer requests such as status and shutdown
* Fixed rowkey access group imbalance problem (causing ROW OVERFLOW errors)
* Added regression test RangeServer-rowkey-ag-imbalanc
* Got KosmosBroker compiling with latest KFS code
* Modified load_generator to make DataGenerator specification opitonal with --spec-file
* Fixed "this should never happen" problem in Maintenance scheduler
* Fixed split log garbage collection
* Makes hypertable build with gcc 4.3.3 on Ubuntu
* Fixed bug in scan limit feature and added regression test.
* Created load generator tool to test performance for synthetic update/query loads.
* Created DiscreteRandomGenerator to generate integer distributions.
* Created ZipfRandomGenerator to generate random integers from a Zipf distribution
* Modified existing Random class to add method to generate samples from uniform[0,1]
* Fixed excessive split problem that caused ROW OVERFLOW errors
* Fixed trailing comma problem in SHOW CREATE TABLE
* Removed redundant error message in scanner creation.
* Added Request throttling
* Made interval scanner initial creation asynchronous.
* Maintenance scheduler overhaul
* [issue 268] RangeServer methods now verify table name
* Added DataGenerator class for generating simulated data
* Added ht_generate_data program with example configs in examples/generate_data
* Added support for gcc 4.[345] in FindBoost for user compiled boost libraries.
* CellCache.PageSize default back to 512K; Fixed CellCachePool::memory_used()
* CellCachePool::memory_used() now returns total allocated from OS minus
* Free space remaining on current page
* Instrumenting CellStore and CellCache to track down suspected memory leak.
* Changed dump_stats method to dump out commit log and per access group memory usage stats.
* Hooked up dump_stats command to rsclient program.
Version 0.9.2.3:
(2009-03-26)
* Fixed problem of range getting loaded multiple times
* Fixed deadlock in AccessGroup::create_scanner (can't hold AG lock while accessing LiveFileTracker)
* [issue 240] Fixed CellCacheScanner logic to apply deletes correctly.
* Issue 255: Removed asserts for last errors to avoid ThriftBroker crashing due to bad user input.
* Upgraded to recent version of Thrift
* Improving scanner creation error reporting.
* Fixed intermittent HyperComm-Timeout test issue
* [issue 262] Fixed race condition between DROP TABLE and range recovery
* Fixed problem with split-recovery 3 & 7
* Eliminated circular reference to Range that was preventing Range destruction
* Issue 225: Fixed/improved table identifier and schema caching mechanism.
* Also fixed an issue in IntervalScanner that was silently ignoring errors.
* Switch to openssl for md5 in tests for better portability.
* Updated thrift documentation on Cell as array interface.
* Upgraded FindBoost to support 1.38
* Changed random_read_test to use new config/property system
* Added --flush option to random_write_test to flush after every write
* Changed default # of maintenance threads in RangeServer to min(2, number-of-cores)
* Changed seek+read to pread in HdfsBroker::PositionRead
* Fixed RangeServer to return TABLE_NOT_FOUND errors when table not found
* Issue 253: Alternative thrift interface for cells as arrays.
* Got rid of hardcoded 30000 ms timeouts in Master
* Added CommitLog.PruneThreshold.Max.MemoryPercentage (default 75)
* Changed AccessGroup.MaxMemory to 1GB by default
* Got rid of various Timeout properties in favor of just Hypertable.Request.Timeout
* Set most timeouts to 180ms
* [issue 258] Minor issue with incorrect time units in Thriftbroker timeout.
* Explicitly flush output streams at a few places until we don't have to _exit.
* Modified CellCache::memory_used() to return accurate info from underlying pool
* Fixed AccessGroup recovery problems for IN_MEMORY access groups
* Made CellCache memory pool page size configurable
* Introduced Hypertable.RangeServer.AccessGroup.CellCache.PageSize, default is 100KB
Version 0.9.2.2:
(2009-03-04)
* Added support for ALTER TABLE
* Addded Bloom Filter to CellStore
* Added calls to _exit in random_read/write benchmark programs to avoid static destruction
* [issue 251] Eliminated race condition between load_range and drop_table
* Upgraded to Hadoop 0.19.1
* [issue 245] Fixed segfault problem with splits and multiple AGs
* Updated thrift support for new namespace changes in the upstream library
* Added missing columns in the thrift scan spec
* Added more scanner/mutator examples for various languages
* Updated README link to the new recommended thrift snapshot
* Made CellStore::create API more generic
* Added row_len and column_qualifier_len in Key to save repeated strlens
* Enabled Bloom filter by default
* Fixed crash in TableMutator if global config properties are not initialized
* Added floating point support for properties
* Fixed broken each_cell and with_mutator methods in ruby thrift client
* Wrapped with_* methods in ensure block
* Added tests for scanner and mutator usage
* Issue 229: fixed some warnings with gcc 4.3+
* [issue 239] Made serverup respect --wait value in all places
* [issue 246] Fixed commit log GC problem
* Upgraded to Quicklz 1.4 and made quicklz the default compressor for CommitLog
* [issue 245] Use broker-assigned strictly increasing fds in KosmosBroker
* [issue 234] Made wait for Hyperspace respect supplied timeout
* [issue 127] Made Hyperspace server detect suspension and extend leases
* [issue 231] Fixed problem where CellStoreV0::add return value was getting ignored
* [issue 238] Fixed crash problem during RangeServer::replay_update
* [issue 235] Eliminated abort after split attempts fail
* [issue 236] Fixed hang problem on CentOS 5
* Fixed a range server restart issue by overwriting tmp explicitly
* Added wait logic in Master when root METADATA range server is down
* Fixed problem where drop_table was executed during a range split causing bad RSML sequence
* Got rid of an assert in the Comm layer on bad header
* Fixed problem where 'Files' column not updated properly after split
* Fixed problem where Gc.Interval was being treated as seconds
* Changed a number of programs to exit using _exit to avoid static destruction problems
* Fixed problem caused by invalid initializations of rssstat (Hiroyuki)
* Made stopping servers faster by waiting in parallel
* Leave RSML alone in case of errors
* Refactored bash specific commands in Capfile into a separate script
* Disable thrift broker API logging by default
* Issue 190: Reverted to the METADATA sanity check since RangeLocator is fixed
Version 0.9.2.1:
(2009-01-22)
* Added ulimit -c unlimited to the top of start-thriftbroker.sh
* Fixed NULL pointer dereference in LoadDataSource on bad input
* Fixing another Hyperspace deadlock (tied up reactor thread)
* Added logic to retry open of Hyperspace file /hypertable/tables/METADATA
* Make sure RangeServer gets killed in between runs of range split recovery test.
* Fixing deadlock in Hyperspace session lease renewal logic.
* Bumped up data size for split recovery tests to ensure crashes
* Remove local fs in clean-database.sh even if dfsbroker is not running
* Fixed initialization race condition in Master
* Commented out MetaLog-Master test
* Fixed version generation for testing with unsuccessful merges
* Fixed reactor thread deadlock in Hyperspace
* Fixing race condition in RangeServer recovery that was causing Hypertable master to crash.
* Eliminated DfsBroker test dependency on /usr/share/dict/words
* Eliminated reference to deprecated class TException in ruby client test
* Changed MasterClient to throw exceptions
* Fixed deadlock in Hyperspace
* Updated thrift API documentation
* make doc (and thriftdoc) now generate thrift api doc as well
* Fixed problem with LocalBroker where it was closing the wrong descriptor because of reuse
* Fixed a CommitLog serialization problem introduced with commit a58dce91b4059b022921287a43d7afb669380d0b
* Augmented multiple-maintenance-threads test to run a "SELECT *" in the background once per second
* Cleaned up logic that manages 'Files' column contents in AccessGroup
* Cleaned up error handling logic in many places
* Fixed split retry logic after exception
* Renamed CrashTest class to FailureInducer and added support for throwing exceptions
* Added 'RangeServer-load-exception' test to validate proper retry when load_range, invoked after split, throws an exception
* Added MicrosecondLayout class in Logger.cc for adding microseconds to timestamps in log messages
* Augmented the multiple-maintenance-threads test with a second dump to see if it's a transient error if the first run comes up short
* Also added a note in README about force restart test servers
* Fixed random segfaults upon exit in random_*_test and serverup
* Coding convention cleanups
* [issue 222] Fixed race condition in RangeServer::update introduced with SplitOff=high
* Cleaned up start up scripts
* Renamed top-level 'test' directory to 'tests' for some platforms
* CMake file changes to restart test servers automatically
* Cleaned up header files to reduce dependency on Config.h hence Version.h
* Fixed a few issues with FindBoost.cmake
* Added .gitignore for *.pyc
* Made some integration tests runnable via make test
* [issue 188] Added support for URGENT CommHeader flag
* Did some work on server-side request timeout
Version 0.9.2.0:
(2009-01-05)
* Fixed initialization problem that caused ConnectionManager thread to exit immediately
* Fixed build for boost 1.35+
* Issue 209: Fixed build with gcc 4.3+
* [issue 216] Fixed problem where clock skew was causing updates to be dropped
* [issue 201] Fixed deadlock recently introduced with support for SplitOff=high
* Made commit log more robust
* Added calls to FSDataOutputStream.sync() for FLUSH for HdfsBroker
* Fixed range server metalog reader
* Made RangeServer::load_range() call in Master::report_split() synchronous
* Fix for ReactorRunner to not use handle if de-registering its fd failed
* Fixed and expanded split-recovery integration test
* Worked around compiler (gcc 4.1 vs 4.0) quirks in overloading
* Fixed a bug in converting hypertable cell to thrift cell that was setting the
wrong column qualifiers in subsequent cells that don't have column qualifiers.
* [issue 193] Added table generation to CellStoreTrailer
* [issue 206] Fixed problem where CommitLog fd gets into bad state
* Fixed race condition in RangeServer::load_range (reported by Phoenix)
* Wrapped compaction logic in try/catch block to ensure maintenance bit gets cleared on exception (reported by Phoenix)
* Added support for Hypertable.CommitLog.SkipErrors (default is now ASSERT)
* Got rid of shutdown segfault in ReactorRunner by only printing LOG message if !shutdown