This repository has been archived by the owner on Feb 9, 2021. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 158
/
CHANGES.txt
12060 lines (8170 loc) · 463 KB
/
CHANGES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Hadoop Change Log
Trunk (unreleased changes)
INCOMPATIBLE CHANGES
HADOOP-8124. Remove the deprecated FSDataOutputStream constructor,
FSDataOutputStream.sync() and Syncable.sync(). (szetszwo)
NEW FEATURES
HADOOP-8135. Add ByteBufferReadable interface to FSDataInputStream. (Henry
Robinson via atm)
IMPROVEMENTS
HADOOP-8017. Configure hadoop-main pom to get rid of M2E plugin execution
not covered (Eric Charles via bobby)
HADOOP-8015. ChRootFileSystem should extend FilterFileSystem
(Daryn Sharp via bobby)
HADOOP-7595. Upgrade dependency to Avro 1.5.3. (Alejandro Abdelnur via atm)
HADOOP-7664. Remove warmings when overriding final parameter configuration
if the override value is same as the final parameter value.
(Ravi Prakash via suresh)
HADOOP-7729. Send back valid HTTP response if user hits IPC port with
HTTP GET. (todd)
HADOOP-7792. Add verifyToken method to AbstractDelegationTokenSecretManager.
(jitendra)
HADOOP-7688. Add servlet handler check in HttpServer.start().
(Uma Maheswara Rao G via szetszwo)
HADOOP-7886. Add toString to FileStatus. (SreeHari via jghoman)
HADOOP-7808. Port HADOOP-7510 - Add configurable option to use original
hostname in token instead of IP to allow server IP change.
(Daryn Sharp via suresh)
HADOOP-7987. Support setting the run-as user in unsecure mode. (jitendra)
HADOOP-7988. Upper case in hostname part of the principals doesn't work with
kerberos. (jitendra)
HADOOP-8078. Add capability to turn on security in unit tests. (Jaimin Jetly
via jitendra)
HADOOP-7994. Remove getProtocolVersion and getProtocolSignature from the
client side translator and server side implementation. (jitendra)
BUG FIXES
HADOOP-8177. MBeans shouldn't try to register when it fails to create MBeanName.
(Devaraj K via umamahesh)
HADOOP-7900. LocalDirAllocator confChanged() accesses conf.get() twice
(Ravi Gummadi via Uma Maheswara Rao G)
HADOOP-8146. FsShell commands cannot be interrupted
(Daryn Sharp via Uma Maheswara Rao G)
HADOOP-8018. Hudson auto test for HDFS has started throwing javadoc
(Jon Eagles via bobby)
HADOOP-8001 ChecksumFileSystem's rename doesn't correctly handle checksum
files. (Daryn Sharp via bobby)
HADOOP-8006 TestFSInputChecker is failing in trunk.
(Daryn Sharp via bobby)
HADOOP-7998. CheckFileSystem does not correctly honor setVerifyChecksum
(Daryn Sharp via bobby)
HADOOP-7606. Upgrade Jackson to version 1.7.1 to match the version required
by Jersey (Alejandro Abdelnur via atm)
HADOOP-7610. Fix for hadoop debian package (Eric Yang via gkesavan)
HADOOP-7641. Add Apache License to template config files.
(Eric Yang via atm)
HADOOP-7621. alfredo config should be in a file not readable by users
(Alejandro Abdelnur via atm)
HADOOP-7669 Fix newly introduced release audit warning.
(Uma Maheswara Rao G via stevel)
HADOOP-6220. HttpServer wraps InterruptedExceptions by IOExceptions
if interrupted in startup (stevel)
HADOOP-7703. Improved exception handling of shutting down web server.
(Devaraj K via Eric Yang)
HADOOP-7704. Reduce number of object created by JMXJsonServlet.
(Devaraj K via Eric Yang)
HADOOP-7769. TestJMXJsonServlet is failing. (tomwhite)
HADOOP-7770. ViewFS getFileChecksum throws FileNotFoundException for files in
/tmp and /user. (Ravi Prakash via jitendra)
HADOOP-8141. Add method to SecurityUtil to init krb5 cipher suites.
(todd)
HADOOP-7788. Add simple HealthMonitor class to watch an HAService (todd)
OPTIMIZATIONS
HADOOP-7761. Improve the performance of raw comparisons. (todd)
Release 0.23.3 - UNRELEASED
INCOMPATIBLE CHANGES
HADOOP-7920. Remove Avro Rpc. (suresh)
NEW FEATURES
HADOOP-7773. Add support for protocol buffer based RPC engine.
(suresh)
HADOOP-7875. Add helper class to unwrap protobuf ServiceException.
(suresh)
HADOOP-7454. Common side of High Availability Framework (HDFS-1623)
Contributed by Todd Lipcon, Aaron T. Myers, Eli Collins, Uma Maheswara Rao G,
Bikas Saha, Suresh Srinivas, Jitendra Nath Pandey, Hari Mankude, Brandon Li,
Sanjay Radia, Mingjie Lai, and Gregory Chanan
HADOOP-8121. Active Directory Group Mapping Service. (Jonathan Natkins via
atm)
HADOOP-7030. Add TableMapping topology implementation to read host to rack
mapping from a file. (Patrick Angeles and tomwhite via tomwhite)
HADOOP-8206. Common portion of a ZK-based failover controller (todd)
IMPROVEMENTS
HADOOP-7524. Change RPC to allow multiple protocols including multuple
versions of the same protocol (sanjay Radia)
HADOOP-7607. Simplify the RPC proxy cleanup process. (atm)
HADOOP-7687. Make getProtocolSignature public (sanjay)
HADOOP-7693. Enhance AvroRpcEngine to support the new #addProtocol
interface introduced in HADOOP-7524. (cutting)
HADOOP-7716. RPC protocol registration on SS does not log the protocol name
(only the class which may be different) (sanjay)
HADOOP-7776. Make the Ipc-Header in a RPC-Payload an explicit header.
(sanjay)
HADOOP-7862. Move the support for multiple protocols to lower layer so
that Writable, PB and Avro can all use it (Sanjay)
HADOOP-7876. Provided access to encoded key in DelegationKey for
use in protobuf based RPCs. (suresh)
HADOOP-7899. Generate proto java files as part of the build. (tucu)
HADOOP-7957. Classes deriving GetGroupsBase should be able to override
proxy creation. (jitendra)
HADOOP-7965. Support for protocol version and signature in PB. (jitendra)
HADOOP-8070. Add a standalone benchmark for RPC call performance. (todd)
HADOOP-8084. Updates ProtoBufRpc engine to not do an unnecessary copy
for RPC request/response. (ddas)
HADOOP-8085. Add RPC metrics to ProtobufRpcEngine. (Hari Mankude via
suresh)
HADOOP-8098. KerberosAuthenticatorHandler should use _HOST replacement to
resolve principal name (tucu)
HADOOP-8118. In metrics2.util.MBeans, change log level to trace for the
stack trace of InstanceAlreadyExistsException. (szetszwo)
HADOOP-8125. make hadoop-client set of curated jars available in a
distribution tarball (rvs via tucu)
HADOOP-7717. Move handling of concurrent client fail-overs to
RetryInvocationHandler (atm)
HADOOP-7728. Enable task memory management to be configurable in hadoop
config setup script. (ramya)
HADOOP-7358. Improve log levels when exceptions caught in RPC handler
(Todd Lipcon via shv)
HADOOP-8108. Move method getHostPortString() from NameNode to NetUtils.
(Brandon Li via jitendra)
HADOOP-7557 Make IPC header be extensible (sanjay radia)
HADOOP-7806. Support binding to sub-interfaces (eli)
HADOOP-6941. Adds support for building Hadoop with IBM's JDK
(Stephen Watt, Eli and ddas)
HADOOP-8183. Stop using "mapred.used.genericoptions.parser" (harsh)
HADOOP-6924. Adds a directory to the list of directories to search
for the libjvm.so file. The new directory is found by running a 'find'
command and the first output is taken. This was done to handle the
build of Hadoop with IBM's JDK. (Stephen Watt, Guillermo Cabrera and ddas)
HADOOP-8200. Remove HADOOP_[JOBTRACKER|TASKTRACKER]_OPTS. (eli)
HADOOP-8184. ProtoBuf RPC engine uses the IPC layer reply packet.
(Sanjay Radia via szetszwo)
HADOOP-8163. Improve ActiveStandbyElector to provide hooks for
fencing old active. (todd)
HADOOP-8193. Refactor FailoverController/HAAdmin code to add an abstract
class for "target" services. (todd)
HADOOP-8212. Improve ActiveStandbyElector's behavior when session expires
(todd)
HADOOP-8216. Address log4j.properties inconsistencies btw main and
template dirs. (Patrick Hunt via eli)
OPTIMIZATIONS
BUG FIXES
HADOOP-7635. RetryInvocationHandler should release underlying resources on
close. (atm)
HADOOP-7695. RPC.stopProxy can throw unintended exception while logging
error. (atm)
HADOOP-7833. Fix findbugs warnings in protobuf generated code.
(John Lee via suresh)
HADOOP-7897. ProtobufRpcEngine client side exception mechanism is not
consistent with WritableRpcEngine. (suresh)
HADOOP-7913. Fix bug in ProtoBufRpcEngine. (sanjay)
HADOOP-7892. IPC logs too verbose after "RpcKind" introduction. (todd)
HADOOP-7968. Errant println left in RPC.getHighestSupportedProtocol. (Sho
Shimauchi via harsh)
HADOOP-7931. o.a.h.ipc.WritableRpcEngine should have a way to force
initialization. (atm)
HADOOP-8104. Inconsistent Jackson versions (tucu)
HADOOP-7940. The Text.clear() method does not clear the bytes as intended. (Csaba Miklos via harsh)
HADOOP-8119. Fix javac warnings in TestAuthenticationFilter in hadoop-auth.
(szetszwo)
HADOOP-7888. TestFailoverProxy fails intermittently on trunk. (Jason Lowe
via atm)
HADOOP-8154. DNS#getIPs shouldn't silently return the local host
IP for bogus interface names. (eli)
HADOOP-8169. javadoc generation fails with java.lang.OutOfMemoryError:
Java heap space (tgraves via bobby)
HADOOP-8167. Configuration deprecation logic breaks backwards compatibility (tucu)
HADOOP-8189. LdapGroupsMapping shouldn't throw away IOException. (Jonathan Natkins via atm)
HADOOP-8191. SshFenceByTcpPort uses netcat incorrectly (todd)
HADOOP-8157. Fix race condition in Configuration that could cause spurious
ClassNotFoundExceptions after a GC. (todd)
HADOOP-8197. Configuration logs WARNs on every use of a deprecated key (tucu)
HADOOP-8159. NetworkTopology: getLeaf should check for invalid topologies.
(Colin Patrick McCabe via eli)
HADOOP-8204. TestHealthMonitor fails occasionally (todd)
HADOOP-8202. RPC stopProxy() does not close the proxy correctly.
(Hari Mankude via suresh)
HADOOP-8218. RPC.closeProxy shouldn't throw error when closing a mock
(todd)
BREAKDOWN OF HADOOP-7454 SUBTASKS
HADOOP-7455. HA: Introduce HA Service Protocol Interface. (suresh)
HADOOP-7774. HA: Administrative CLI to control HA daemons. (todd)
HADOOP-7896. HA: if both NNs are in Standby mode, client needs to try failing
back and forth several times with sleeps. (atm)
HADOOP-7922. Improve some logging for client IPC failovers and
StandbyExceptions (todd)
HADOOP-7921. StandbyException should extend IOException (todd)
HADOOP-7928. HA: Client failover policy is incorrectly trying to fail over all
IOExceptions (atm)
HADOOP-7925. Add interface and update CLI to query current state to
HAServiceProtocol (eli via todd)
HADOOP-7932. Make client connection retries on socket time outs configurable.
(Uma Maheswara Rao G via todd)
HADOOP-7924. FailoverController for client-based configuration (eli)
HADOOP-7961. Move HA fencing to common. (eli)
HADOOP-7970. HAServiceProtocol methods must throw IOException. (Hari Mankude
via suresh).
HADOOP-7992. Add ZKClient library to facilitate leader election. (Bikas Saha
via suresh).
HADOOP-7983. HA: failover should be able to pass args to fencers. (eli)
HADOOP-7938. HA: the FailoverController should optionally fence the active
during failover. (eli)
HADOOP-7991. HA: the FailoverController should check the standby is ready
before failing over. (eli)
HADOOP-8038. Add 'ipc.client.connect.max.retries.on.timeouts' entry in
core-default.xml file. (Uma Maheswara Rao G via atm)
HADOOP-8041. Log a warning when a failover is first attempted (todd)
HADOOP-8068. void methods can swallow exceptions when going through failover
path (todd)
HADOOP-8116. RetriableCommand is using RetryPolicy incorrectly after
HADOOP-7896. (atm)
Release 0.23.2 - UNRELEASED
INCOMPATIBLE CHANGES
NEW FEATURES
IMPROVEMENTS
HADOOP-8048. Allow merging of Credentials (Daryn Sharp via tgraves)
HADOOP-8032. mvn site:stage-deploy should be able to use the scp protocol
to stage documents (Ravi Prakash via tgraves)
HADOOP-7923. Automate the updating of version numbers in the doc system.
(szetszwo)
HADOOP-8137. Added links to CLI manuals to the site. (tgraves via
acmurthy)
OPTIMIZATIONS
HADOOP-8071. Avoid an extra packet in client code when nagling is
disabled. (todd)
HADOOP-6502. Improve the performance of Configuration.getClassByName when
the class is not found by caching negative results.
(sharad, todd via todd)
BUG FIXES
HADOOP-7660. Maven generated .classpath doesnot includes
"target/generated-test-source/java" as source directory.
(Laxman via bobby)
HADOOP-8042 When copying a file out of HDFS, modifying it, and uploading
it back into HDFS, the put fails due to a CRC mismatch
(Daryn Sharp via bobby)
HADOOP-8035 Hadoop Maven site is inefficient and runs phases redundantly
(abayer via tucu)
HADOOP-8051 HttpFS documentation it is not wired to the generated site (tucu)
HADOOP-8055. Hadoop tarball distribution lacks a core-site.xml (harsh)
HADOOP-8052. Hadoop Metrics2 should emit Float.MAX_VALUE (instead of
Double.MAX_VALUE) to avoid making Ganglia's gmetad core. (Varun Kapoor
via mattf)
HADOOP-8074. Small bug in hadoop error message for unknown commands.
(Colin Patrick McCabe via eli)
HADOOP-8082 add hadoop-client and hadoop-minicluster to the
dependency-management section. (tucu)
HADOOP-8066 The full docs build intermittently fails (abayer via tucu)
HADOOP-8083 javadoc generation for some modules is not done under target/ (tucu)
HADOOP-8036. TestViewFsTrash assumes the user's home directory is
2 levels deep. (Colin Patrick McCabe via eli)
HADOOP-8046 Revert StaticMapping semantics to the existing ones, add DNS
mapping diagnostics in progress (stevel)
HADOOP-8057 hadoop-setup-conf.sh not working because of some extra spaces.
(Vinayakumar B via stevel)
HADOOP-7680 TestHardLink fails on Mac OS X, when gnu stat is in path.
(Milind Bhandarkar via stevel)
HADOOP-8050. Deadlock in metrics. (Kihwal Lee via mattf)
HADOOP-8131. FsShell put doesn't correctly handle a non-existent dir
(Daryn Sharp via bobby)
HADOOP-8123. Use java.home rather than env.JAVA_HOME for java in the
project. (Jonathan Eagles via acmurthy)
HADOOP-8064. Remove unnecessary dependency on w3c.org in document processing
(Khiwal Lee via bobby)
HADOOP-8140. dfs -getmerge should process its argments better (Daryn Sharp
via bobby)
HADOOP-8164. Back slash as path separator is handled for Windows only.
(Daryn Sharp via suresh)
HADOOP-8173. FsShell needs to handle quoted metachars. (Daryn Sharp via
szetszwo)
HADOOP-8175. Add -p option to mkdir in FsShell. (Daryn Sharp via szetszwo)
HADOOP-8176. Disambiguate the destination of FsShell copies (Daryn Sharp
via bobby)
HADOOP-8088. User-group mapping cache incorrectly does negative caching on
transient failures (Khiwal Lee via bobby)
Release 0.23.1 - 2012-02-17
INCOMPATIBLE CHANGES
NEW FEATURES
HADOOP-7777 Implement a base class for DNSToSwitchMapping implementations
that can offer extra topology information. (stevel)
HADOOP-7657. Add support for LZ4 compression. (Binglin Chang via todd)
HADOOP-7910. Add Configuration.getLongBytes to handle human readable byte size values. (Sho Shimauchi via harsh)
IMPROVEMENTS
HADOOP-7801. HADOOP_PREFIX cannot be overriden. (Bruno Mahé via tomwhite)
HADOOP-7802. Hadoop scripts unconditionally source
"$bin"/../libexec/hadoop-config.sh. (Bruno Mahé via tomwhite)
HADOOP-7858. Drop some info logging to DEBUG level in IPC,
metrics, and HTTP. (todd via eli)
HADOOP-7424. Log an error if the topology script doesn't handle multiple args.
(Uma Maheswara Rao G via eli)
HADOOP-7804. Enable hadoop config generator to set configurations to enable
short circuit read. (Arpit Gupta via jitendra)
HADOOP-7877. Update balancer CLI usage documentation to include the new
-policy option. (szetszwo)
HADOOP-6840. Support non-recursive create() in FileSystem and
SequenceFile.Writer. (jitendra and eli via eli)
HADOOP-6886. LocalFileSystem Needs createNonRecursive API.
(Nicolas Spiegelberg and eli via eli)
HADOOP-7912. test-patch should run eclipse:eclipse to verify that it does
not break again. (Robert Joseph Evans via tomwhite)
HADOOP-7890. Redirect hadoop script's deprecation message to stderr.
(Koji Knoguchi via mahadev)
HADOOP-7504. Add the missing Ganglia31 opts to hadoop-metrics.properties as a comment. (harsh)
HADOOP-7933. Add a getDelegationTokens api to FileSystem which checks
for known tokens in the passed Credentials object. (sseth)
HADOOP-7737. normalize hadoop-mapreduce & hadoop-dist dist/tar build with
common/hdfs. (tucu)
HADOOP-7743. Add Maven profile to create a full source tarball. (tucu)
HADOOP-7758. Make GlobFilter class public. (tucu)
HADOOP-7590. Mavenize streaming and MR examples. (tucu)
HADOOP-7934. Normalize dependencies versions across all modules. (tucu)
HADOOP-7348. Change 'addnl' in getmerge util to be a flag '-nl' instead.
(XieXianshan via harsh)
HADOOP-7975. Add LZ4 as an entry in the default codec list, missed by HADOOP-7657 (harsh)
HADOOP-4515. Configuration#getBoolean must not be case sensitive. (Sho Shimauchi via harsh)
HADOOP-6490. Use StringUtils over String#replace in Path#normalizePath.
(Uma Maheswara Rao G via harsh)
HADOOP-7574. Improve FSShell -stat, add user/group elements.
(XieXianshan via harsh)
HADOOP-7736. Remove duplicate Path#normalizePath call. (harsh)
HADOOP-7919. Remove the unused hadoop.logfile.* properties from the
core-default.xml file. (harsh)
HADOOP-7939. Improve Hadoop subcomponent integration in Hadoop 0.23. (rvs via tucu)
HADOOP-8002. SecurityUtil acquired token message should be a debug rather than info.
(Arpit Gupta via mahadev)
HADOOP-8009. Create hadoop-client and hadoop-minicluster artifacts for downstream
projects. (tucu)
HADOOP-7470. Move up to Jackson 1.8.8. (Enis Soztutar via szetszwo)
HADOOP-8027. Visiting /jmx on the daemon web interfaces may print
unnecessary error in logs. (atm)
OPTIMIZATIONS
BUG FIXES
HADOOP-7811. TestUserGroupInformation#testGetServerSideGroups test fails in chroot.
(Jonathan Eagles via mahadev)
HADOOP-7813. Fix test-patch to use proper numerical comparison when checking
javadoc and findbugs warning counts. (Jonathan Eagles via tlipcon)
HADOOP-7841. Run tests with non-secure random. (tlipcon)
HADOOP-7851. Configuration.getClasses() never returns the default value.
(Uma Maheswara Rao G via amarrk)
HADOOP-7787. Make source tarball use conventional name.
(Bruno Mahé via tomwhite)
HADOOP-6614. RunJar should provide more diags when it can't create
a temp file. (Jonathan Hsieh via eli)
HADOOP-7859. TestViewFsHdfs.testgetFileLinkStatus is failing an assert. (eli)
HADOOP-7864. Building mvn site with Maven < 3.0.2 causes OOM errors.
(Andrew Bayer via eli)
HADOOP-7854. UGI getCurrentUser is not synchronized. (Daryn Sharp via jitendra)
HADOOP-7870. fix SequenceFile#createWriter with boolean
createParent arg to respect createParent. (Jon Hsieh via eli)
HADOOP-7898. Fix javadoc warnings in AuthenticationToken.java. (suresh)
HADOOP-7878 Regression: HADOOP-7777 switch changes break HDFS tests when the
isSingleSwitch() predicate is used. (stevel)
HADOOP-7914. Remove the duplicated declaration of hadoop-hdfs test-jar in
hadoop-project/pom.xml. (szetszwo)
HADOOP-7837. no NullAppender in the log4j config. (eli)
HADOOP-7948. Shell scripts created by hadoop-dist/pom.xml to build tar do not
properly propagate failure. (cim_michajlomatijkiw via tucu)
HADOOP-7949. Updated maxIdleTime default in the code to match
core-default.xml (eli)
HADOOP-7853. multiple javax security configurations cause conflicts.
(daryn via tucu)
HDFS-2614. hadoop dist tarball is missing hdfs headers. (tucu)
HADOOP-7874. native libs should be under lib/native/ dir. (tucu)
HADOOP-7887. KerberosAuthenticatorHandler is not setting
KerberosName name rules from configuration. (tucu)
HADOOP-7902. skipping name rules setting (if already set) should be done
on UGI initialization only. (tucu)
HADOOP-7810. move hadoop archive to core from tools. (tucu)
HADOOP_7917. compilation of protobuf files fails in windows/cygwin. (tucu)
HADOOP-7907. hadoop-tools JARs are not part of the distro. (tucu)
HADOOP-7936. There's a Hoop README in the root dir of the tarball. (tucu)
HADOOP-7963. Fix ViewFS to catch a null canonical service-name and pass
tests TestViewFileSystem* (Siddharth Seth via vinodkv)
HADOOP-7964. Deadlock in NetUtils and SecurityUtil class initialization.
(Daryn Sharp via suresh)
HADOOP-7974. TestViewFsTrash incorrectly determines the user's home
directory. (harsh via eli)
HADOOP-7971. Adding back job/pipes/queue commands to bin/hadoop for
backward compatibility. (Prashath Sharma via acmurthy)
HADOOP-7982. UserGroupInformation fails to login if thread's context
classloader can't load HadoopLoginModule. (todd)
HADOOP-7986. Adding config for MapReduce History Server protocol in
hadoop-policy.xml for service level authorization. (Mahadev Konar via vinodkv)
HADOOP-7981. Improve documentation for org.apache.hadoop.io.compress.
Decompressor.getRemaining (Jonathan Eagles via mahadev)
HADOOP-7997. SequenceFile.createWriter(...createParent...) no
longer works on existing file. (Gregory Chanan via eli)
HADOOP-7993. Hadoop ignores old-style config options for enabling compressed
output. (Anupam Seth via mahadev)
HADOOP-8000. fetchdt command not available in bin/hadoop.
(Arpit Gupta via mahadev)
HADOOP-7999. "hadoop archive" fails with ClassNotFoundException.
(Jason Lowe via mahadev)
HADOOP-8012. hadoop-daemon.sh and yarn-daemon.sh are trying to mkdir
and chown log/pid dirs which can fail. (Roman Shaposhnik via eli)
HADOOP-8013. ViewFileSystem does not honor setVerifyChecksum
(Daryn Sharp via bobby)
HADOOP-8054 NPE with FilterFileSystem (Daryn Sharp via bobby)
Release 0.23.0 - 2011-11-01
INCOMPATIBLE CHANGES
HADOOP-6904. Support method based RPC compatiblity. (hairong)
HADOOP-6432. Add Statistics support in FileContext. (jitendra)
HADOOP-7136. Remove failmon contrib component. (nigel)
NEW FEATURES
HADOOP-7324. Ganglia plugins for metrics v2. (Priyo Mustafi via llu)
HADOOP-7342. Add an utility API in FileUtil for JDK File.list
avoid NPEs on File.list() (Bharath Mundlapudi via mattf)
HADOOP-7322. Adding a util method in FileUtil for directory listing,
avoid NPEs on File.listFiles() (Bharath Mundlapudi via mattf)
HADOOP-7023. Add listCorruptFileBlocks to Filesysem. (Patrick Kling
via hairong)
HADOOP-7096. Allow setting of end-of-record delimiter for TextInputFormat
(Ahmed Radwan via todd)
HADOOP-6994. Api to get delegation token in AbstractFileSystem. (jitendra)
HADOOP-7171. Support UGI in FileContext API. (jitendra)
HADOOP-7257 Client side mount tables (sanjay)
HADOOP-6919. New metrics2 framework. (Luke Lu via acmurthy)
HADOOP-6920. Metrics instrumentation to move new metrics2 framework.
(Luke Lu via suresh)
HADOOP-7214. Add Common functionality necessary to provide an equivalent
of /usr/bin/groups for Hadoop. (Aaron T. Myers via todd)
HADOOP-6832. Add an authentication plugin using a configurable static user
for the web UI. (Owen O'Malley and Todd Lipcon via cdouglas)
HADOOP-7144. Expose JMX metrics via JSON servlet. (Robert Joseph Evans via
cdouglas)
HADOOP-7379. Add the ability to serialize and deserialize protocol buffers
in ObjectWritable. (todd)
HADOOP-7206. Support Snappy compression. (Issei Yoshida and
Alejandro Abdelnur via eli)
HADOOP-7329. Add the capability of getting invividual attribute of a mbean
using JMXProxyServlet. (tanping)
HADOOP-7380. Add client failover functionality to o.a.h.io.(ipc|retry).
(atm via eli)
HADOOP-7460. Support pluggable trash policies. (Usman Masoon via suresh)
HADOOP-6385. dfs should support -rmdir (was HDFS-639). (Daryn Sharp
via mattf)
HADOOP-7119. add Kerberos HTTP SPNEGO authentication support to Hadoop
JT/NN/DN/TT web-consoles. (Alejandro Abdelnur via atm)
IMPROVEMENTS
HADOOP-7655. Provide a small validation script that smoke tests the installed
cluster. (Arpit Gupta via mattf)
HADOOP-7042. Updates to test-patch.sh to include failed test names and
improve other messaging. (nigel)
HADOOP-7001. Configuration changes can occur via the Reconfigurable
interface. (Patrick Kling via dhruba)
HADOOP-6764. Add number of reader threads and queue length as
configuration parameters in RPC.getServer. (Dmytro Molkov via hairong)
HADOOP-7049. TestReconfiguration should be junit v4.
(Patrick Kling via eli)
HADOOP-7054 Change NN LoadGenerator to use FileContext APIs
(Sanjay Radia)
HADOOP-7060. A more elegant FileSystem#listCorruptFileBlocks API.
(Patrick Kling via hairong)
HADOOP-7058. Expose number of bytes in FSOutputSummer buffer to
implementatins. (Todd Lipcon via hairong)
HADOOP-7061. unprecise javadoc for CompressionCodec. (Jingguo Yao via eli)
HADOOP-7059. Remove "unused" warning in native code. (Noah Watkins via eli)
HADOOP-6864. Provide a JNI-based implementation of
ShellBasedUnixGroupsNetgroupMapping
(implementation of GroupMappingServiceProvider) (Erik Seffl via boryas)
HADOOP-7078. Improve javadocs for RawComparator interface.
(Harsh J Chouraria via todd)
HADOOP-6995. Allow wildcards to be used in ProxyUsers configurations.
(todd)
HADOOP-6376. Add a comment header to conf/slaves that specifies the file
format. (Kay Kay via todd)
HADOOP-7151. Document need for stable hashCode() in WritableComparable.
(Dmitriy V. Ryaboy via todd)
HADOOP-7112. Issue a warning when GenericOptionsParser libjars are not on
local filesystem. (tomwhite)
HADOOP-7114. FsShell should dump all exceptions at DEBUG level.
(todd via tomwhite)
HADOOP-7159. RPC server should log the client hostname when read exception
happened. (Scott Chen via todd)
HADOOP-7167. Allow using a file to exclude certain tests from build. (todd)
HADOOP-7133. Batch the calls in DataStorage to FileUtil.createHardLink().
(Matt Foley via jghoman)
HADOOP-7166. Add DaemonFactory to common. (Erik Steffl & jitendra)
HADOOP-7175. Add isEnabled() to Trash. (Daryn Sharp via szetszwo)
HADOOP-7180. Better support on CommandFormat on the API and exceptions.
(Daryn Sharp via szetszwo)
HADOOP-7202. Improve shell Command base class. (Daryn Sharp via szetszwo)
HADOOP-7224. Add CommandFactory to shell. (Daryn Sharp via szetszwo)
HADOOP-7014. Generalize CLITest structure and interfaces to facilitate
upstream adoption (e.g. for web testing). (cos)
HADOOP-7230. Move "fs -help" shell command tests from HDFS to COMMOM; see
also HDFS-1844. (Daryn Sharp via szetszwo)
HADOOP-7233. Refactor ls to conform to new FsCommand class. (Daryn Sharp
via szetszwo)
HADOOP-7235. Refactor the tail command to conform to new FsCommand class.
(Daryn Sharp via szetszwo)
HADOOP-7179. Federation: Improve HDFS startup scripts. (Erik Steffl
and Tanping Wang via suresh)
HADOOP-7227. Remove protocol version check at proxy creation in Hadoop
RPC. (jitendra)
HADOOP-7236. Refactor the mkdir command to conform to new FsCommand class.
(Daryn Sharp via szetszwo)
HADOOP-7250. Refactor the setrep command to conform to new FsCommand class.
(Daryn Sharp via szetszwo)
HADOOP-7249. Refactor the chmod/chown/chgrp command to conform to new
FsCommand class. (Daryn Sharp via szetszwo)
HADOOP-7251. Refactor the getmerge command to conform to new FsCommand
class. (Daryn Sharp via szetszwo)
HADOOP-7265. Keep track of relative paths in PathData. (Daryn Sharp
via szetszwo)
HADOOP-7238. Refactor the cat and text commands to conform to new FsCommand
class. (Daryn Sharp via szetszwo)
HADOOP-7271. Standardize shell command error messages. (Daryn Sharp
via szetszwo)
HADOOP-7272. Remove unnecessary security related info logs. (suresh)
HADOOP-7275. Refactor the stat command to conform to new FsCommand
class. (Daryn Sharp via szetszwo)
HADOOP-7237. Refactor the touchz command to conform to new FsCommand
class. (Daryn Sharp via szetszwo)
HADOOP-7267. Refactor the rm/rmr/expunge commands to conform to new
FsCommand class. (Daryn Sharp via szetszwo)
HADOOP-7285. Refactor the test command to conform to new FsCommand
class. (Daryn Sharp via todd)
HADOOP-7289. In ivy.xml, test conf should not extend common conf.
(Eric Yang via szetszwo)
HADOOP-7291. Update Hudson job not to run test-contrib. (Nigel Daley via eli)
HADOOP-7286. Refactor the du/dus/df commands to conform to new FsCommand
class. (Daryn Sharp via todd)
HADOOP-7301. FSDataInputStream should expose a getWrappedStream method.
(Jonathan Hsieh via eli)
HADOOP-7306. Start metrics system even if config files are missing
(Luke Lu via todd)
HADOOP-7302. webinterface.private.actions should be renamed and moved to
the MapReduce project. (Ari Rabkin via todd)
HADOOP-7329. Improve help message for "df" to include "-h" flag.
(Xie Xianshan via todd)
HADOOP-7320. Refactor the copy and move commands to conform to new
FsCommand class. (Daryn Sharp via todd)
HADOOP-7312. Update value of hadoop.common.configuration.version.
(Harsh J Chouraria via todd)
HADOOP-7337. Change PureJavaCrc32 annotations to public stable. (szetszwo)
HADOOP-7331. Make hadoop-daemon.sh return exit code 1 if daemon processes
did not get started. (Tanping Wang via todd)
HADOOP-7316. Add public javadocs to FSDataInputStream and
FSDataOutputStream. (eli)
HADOOP-7323. Add capability to resolve compression codec based on codec
name. (Alejandro Abdelnur via tomwhite)
HADOOP-1886. Undocumented parameters in FilesSystem. (Frank Conrad via eli)
HADOOP-7375. Add resolvePath method to FileContext. (Sanjay Radia via eli)
HADOOP-7383. HDFS needs to export protobuf library dependency in pom.
(todd via eli)
HADOOP-7374. Don't add tools.jar to the classpath when running Hadoop.
(eli)
HADOOP-7106. Reorganize project SVN layout to "unsplit" the projects.
(todd, nigel)
HADOOP-6605. Add JAVA_HOME detection to hadoop-config. (eli)
HADOOP-7384. Allow test-patch to be more flexible about patch format. (todd)
HADOOP-6929. RPC should have a way to pass Security information other than
protocol annotations. (sharad and omalley via mahadev)
HADOOP-7385. Remove StringUtils.stringifyException(ie) in logger functions.
(Bharath Mundlapudi via Tanping Wang).
HADOOP-310. Additional constructor requested in BytesWritable. (Brock
Noland via atm)
HADOOP-7429. Add another IOUtils#copyBytes method. (eli)
HADOOP-7451. Generalize StringUtils#join. (Chris Douglas via mattf)
HADOOP-7449. Add Data(In,Out)putByteBuffer to work with ByteBuffer similar
to Data(In,Out)putBuffer for byte[]. Merge from yahoo-merge branch,
-r 1079163. Fix missing Apache license headers. (Chris Douglas via mattf)
HADOOP-7361. Provide an option, -overwrite/-f, in put and copyFromLocal
shell commands. (Uma Maheswara Rao G via szetszwo)
HADOOP-7430. Improve error message when moving to trash fails due to
quota issue. (Ravi Prakash via mattf)
HADOOP-7444. Add Checksum API to verify and calculate checksums "in bulk"
(todd)
HADOOP-7443. Add CRC32C as another DataChecksum implementation (todd)
HADOOP-7305. Eclipse project files are incomplete. (Niels Basjes via eli)
HADOOP-7314. Add support for throwing UnknownHostException when a host doesn't
resolve. (Jeffrey Naisbitt via jitendra)
HADOOP-7465. A several tiny improvements for the LOG format.
(Xie Xianshan via eli)
HADOOP-7434. Display error when using "daemonlog -setlevel" with
illegal level. (yanjinshuang via eli)
HADOOP-7463. Adding a configuration parameter to SecurityInfo interface.
(mahadev)
HADOOP-7298. Add test utility for writing multi-threaded tests. (todd and
Harsh J Chouraria via todd)
HADOOP-7485. Add -h option to ls to list file sizes in human readable
format. (XieXianshan via suresh)
HADOOP-7378. Add -d option to ls to not expand directories.
(Daryn Sharp via suresh)
HADOOP-7474. Refactor ClientCache out of WritableRpcEngine. (jitendra)
HADOOP-7491. hadoop command should respect HADOOP_OPTS when given
a class name. (eli)
HADOOP-7178. Add a parameter, useRawLocalFileSystem, to copyToLocalFile(..)
in FileSystem. (Uma Maheswara Rao G via szetszwo)
HADOOP-6671. Use maven for hadoop common builds. (Alejandro Abdelnur
via tomwhite)
HADOOP-7502. Make generated sources IDE friendly.
(Alejandro Abdelnur via llu)
HADOOP-7501. Publish Hadoop Common artifacts (post HADOOP-6671) to Apache
SNAPSHOTs repo. (Alejandro Abdelnur via tomwhite)
HADOOP-7525. Make arguments to test-patch optional. (tomwhite)
HADOOP-7472. RPC client should deal with IP address change.
(Kihwal Lee via suresh)
HADOOP-7499. Add method for doing a sanity check on hostnames in NetUtils.
(Jeffrey Naisbit via mahadev)
HADOOP-6158. Move CyclicIteration to HDFS. (eli)
HADOOP-7526. Add TestPath tests for URI conversion and reserved
characters. (eli)
HADOOP-7531. Add servlet util methods for handling paths in requests. (eli)
HADOOP-7493. Add ShortWritable. (Uma Maheswara Rao G via szetszwo)
HADOOP-7555. Add a eclipse-generated files to .gitignore. (atm)
HADOOP-7264. Bump avro version to at least 1.4.1. (Alejandro Abdelnur via
tomwhite)
HADOOP-7498. Remove legacy TAR layout creation. (Alejandro Abdelnur via
tomwhite)
HADOOP-7496. Break Maven TAR & bintar profiles into just LAYOUT & TAR proper.
(Alejandro Abdelnur via tomwhite)
HADOOP-7561. Make test-patch only run tests for changed modules. (tomwhite)
HADOOP-7547. Add generic type in WritableComparable subclasses.
(Uma Maheswara Rao G via szetszwo)
HADOOP-7579. Rename package names from alfredo to auth.
(Alejandro Abdelnur via szetszwo)
HADOOP-7594. Support HTTP REST in HttpServer. (szetszwo)
HADOOP-7552. FileUtil#fullyDelete doesn't throw IOE but lists it
in the throws clause. (eli)