This repository has been archived by the owner on Feb 9, 2021. It is now read-only.
/
CHANGES.txt
14507 lines (9702 loc) · 546 KB
/
CHANGES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Hadoop Change Log
Release 2.4.1 - UNRELEASED
INCOMPATIBLE CHANGES
NEW FEATURES
IMPROVEMENTS
OPTIMIZATIONS
BUG FIXES
Release 2.4.0 - 2014-04-07
INCOMPATIBLE CHANGES
HADOOP-8691. FsShell can print "Found xxx items" unnecessarily often.
(Daryn Sharp via wheat9)
NEW FEATURES
HADOOP-10184. Hadoop Common changes required to support HDFS ACLs. (See
breakdown of tasks below for features and contributors)
HADOOP-10285. Admin interface to swap callqueue at runtime. (Chris Li via
Arpit Agarwal)
IMPROVEMENTS
HADOOP-10139. Update and improve the Single Cluster Setup document.
(Akira Ajisaka via Arpit Agarwal)
HADOOP-10295. Allow distcp to automatically identify the checksum type of
source files and use it for the target. (jing9 and Laurent Goujon)
HADOOP-10333. Fix grammatical error in overview.html document.
(René Nyffenegger via suresh)
HADOOP-10343. Change info to debug log in LossyRetryInvocationHandler. (arpit)
HADOOP-10348. Deprecate hadoop.ssl.configuration in branch-2, and remove
it in trunk. (Haohui Mai via jing9)
HADOOP-9454. Support multipart uploads for s3native. (Jordan Mendelson and
Akira AJISAKA via atm)
HADOOP-10374. InterfaceAudience annotations should have
RetentionPolicy.RUNTIME (Enis Soztutar via Arpit Agarwal)
HADOOP-10278. Refactor to make CallQueue pluggable. (Chris Li via
Arpit Agarwal)
HADOOP-10314. The ls command help still shows outdated 0.16 format.
(Rushabh S Shah via kihwal)
HADOOP-10379. Protect authentication cookies with the HttpOnly and Secure
flags. (wheat9)
HADOOP-10211. Enable RPC protocol to negotiate SASL-QOP values between
clients and servers. (Benoy Antony via Arpit Agarwal)
HADOOP-10386. Log proxy hostname in various exceptions being thrown in a HA
setup. (wheat9)
HADOOP-10221. Add a plugin to specify SaslProperties for RPC protocol
based on connection properties. (Benoy Antony and Daryn Sharp via
Arpit Agarwal)
HADOOP-10015. UserGroupInformation prints out excessive warnings.
(Nicolas Liochon via szetszwo)
HADOOP-10423. Clarify compatibility policy document for combination of
new client and old server. (Chris Nauroth via kasha)
HADOOP-10280. Make Schedulables return a configurable identity of user
or group. (Chris Li via Arpit Agarwal)
OPTIMIZATIONS
BUG FIXES
HADOOP-10320. Javadoc in InterfaceStability.java lacks final </ul>.
(René Nyffenegger via cnauroth)
HADOOP-10085. CompositeService should allow adding services while being
inited. (Steve Loughran via kasha)
HADOOP-10327. Trunk windows build broken after HDFS-5746.
(Vinay via cnauroth)
HADOOP-10330. TestFrameDecoder fails if it cannot bind port 12345.
(Arpit Agarwal)
HADOOP-10326. M/R jobs can not access S3 if Kerberos is enabled. (bc Wong
via atm)
HADOOP-10338. Cannot get the FileStatus of the root inode from the new
Globber (cmccabe)
HADOOP-10249. LdapGroupsMapping should trim ldap password read from file.
(Dilli Armugam via suresh)
HADOOP-10346. Deadlock while logging tokens (jlowe)
HADOOP-10328. loadGenerator exit code is not reliable.
(Haohui Mai via cnauroth)
HADOOP-10355. Fix TestLoadGenerator#testLoadGenerator. (Haohui Mai via jing9)
HADOOP-10368. InputStream is not closed in VersionInfo ctor.
(Tsuyoshi OZAWA via szetszwo)
HADOOP-10353. FsUrlStreamHandlerFactory is not thread safe.
(Tudor Scurtu via cnauroth)
HADOOP-10393. Fix the javac warnings in hadoop-auth. (szetszwo)
HADOOP-10395. TestCallQueueManager is flaky. (Arpit Agarwal)
HADOOP-10394. TestAuthenticationFilter is flaky. (Arpit Agarwal)
HADOOP-10337 ConcurrentModificationException from
MetricsDynamicMBeanBase.createMBeanInfo() (Liang Xie via stack)
HADOOP-10407. Fix the javac warnings in org.apache.hadoop.ipc package.
(szetszwo)
HADOOP-10070. RPC client doesn't use per-connection conf to determine
server's expected Kerberos principal name. (atm)
HADOOP-10191. Missing executable permission on viewfs internal dirs.
(Gera Shegalov via cnauroth)
HADOOP-10422. Remove redundant logging of RPC retry attempts. (cnauroth)
HADOOP-10425. LocalFileSystem.getContentSummary should not count crc files.
(szetszwo)
HADOOP-10440. HarFsInputStream.read(byte[]) updates position incorrectly.
(guodongdong via szetszwo)
HADOOP-10437. Fix the javac warnings in the conf and the util package.
(szetszwo)
HADOOP-10441. Namenode metric "rpc.RetryCache/NameNodeRetryCache.CacheHit"
can't be correctly processed by Ganglia. (jing9)
HADOOP-10449. Fix the javac warnings in the security package. (szetszwo)
HADOOP-10450. Build zlib native code bindings in hadoop.dll for Windows.
(cnauroth)
HADOOP-10301. AuthenticationFilter should return Forbidden for failed
authentication. (Daryn Sharp via jing9)
HADOOP-9525. Add tests that validate winutils chmod behavior on folders
(ivanmi)
BREAKDOWN OF HADOOP-10184 SUBTASKS AND RELATED JIRAS
HADOOP-10185. FileSystem API for ACLs. (cnauroth)
HADOOP-10186. Remove AclReadFlag and AclWriteFlag in FileSystem API.
(Haohui Mai via cnauroth)
HADOOP-10187. FsShell CLI: add getfacl and setfacl with minimal support for
getting and setting ACLs. (Vinay via cnauroth)
HADOOP-10192. FileSystem#getAclStatus has incorrect JavaDocs. (cnauroth)
HADOOP-10220. Add ACL indicator bit to FsPermission. (cnauroth)
HADOOP-10241. Clean up output of FsShell getfacl. (Chris Nauroth via wheat9)
HADOOP-10213. Fix bugs parsing ACL spec in FsShell setfacl.
(Vinay via cnauroth)
HADOOP-10277. setfacl -x fails to parse ACL spec if trying to remove the
mask entry. (Vinay via cnauroth)
HADOOP-10270. getfacl does not display effective permissions of masked
entries. (cnauroth)
HADOOP-10344. Fix TestAclCommands after merging HADOOP-10338 patch.
(cnauroth)
HADOOP-10352. Recursive setfacl erroneously attempts to apply default ACL to
files. (cnauroth)
HADOOP-10354. TestWebHDFS fails after merge of HDFS-4685 to trunk. (cnauroth)
HADOOP-10361. Correct alignment in CLI output for ACLs. (cnauroth)
HADOOP-10399. FileContext API for ACLs. (Vinayakumar B via cnauroth)
HADOOP-10442. Group look-up can cause segmentation fault when certain
JNI-based mapping module is used. (Kihwal Lee via jeagles)
Release 2.3.1 - UNRELEASED
INCOMPATIBLE CHANGES
NEW FEATURES
IMPROVEMENTS
OPTIMIZATIONS
BUG FIXES
Release 2.3.0 - 2014-02-18
INCOMPATIBLE CHANGES
NEW FEATURES
HADOOP-8545. Filesystem Implementation for OpenStack Swift
(Dmitry Mezhensky, David Dobbins, Stevel via stevel)
IMPROVEMENTS
HADOOP-10046. Print a log message when SSL is enabled.
(David S. Wang via wang)
HADOOP-10079. log a warning message if group resolution takes too long.
(cmccabe)
HADOOP-9623. Update jets3t dependency to 0.9.0 (Amandeep Khurana via Colin
Patrick McCabe)
HADOOP-10132. RPC#stopProxy() should log the class of proxy when IllegalArgumentException
is encountered (Ted yu via umamahesh)
HADOOP-10248. Property name should be included in the exception where property value
is null (Akira AJISAKA via umamahesh)
HADOOP-10086. User document for authentication in secure cluster.
(Masatake Iwasaki via Arpit Agarwal)
HADOOP-10274 Lower the logging level from ERROR to WARN for UGI.doAs method
(Takeshi Miao via stack)
HADOOP-9784. Add a builder for HttpServer. (Junping Du via llu)
HADOOP 9871. Fix intermittent findbugs warnings in DefaultMetricsSystem.
(Junping Du via llu)
HADOOP-9319. Update bundled LZ4 source to r99. (Binglin Chang via llu)
HADOOP-9432 Add support for markdown .md files in site documentation (stevel)
HADOOP-9241. DU refresh interval is not configurable (harsh)
HADOOP-9417. Support for symlink resolution in LocalFileSystem /
RawLocalFileSystem. (Andrew Wang via Colin Patrick McCabe)
HADOOP-9703. org.apache.hadoop.ipc.Client leaks threads on stop.
(Tsuyoshi OZAWA vi Colin Patrick McCabe)
HADOOP-9618. Add thread which detects GC pauses.
(Todd Lipcon via Colin Patrick McCabe)
HADOOP-9848. Create a MiniKDC for use with security testing.
(ywskycn via tucu)
HADOOP-9860. Remove class HackedKeytab and HackedKeytabEncoder from
hadoop-minikdc once jira DIRSERVER-1882 solved. (ywskycn via tucu)
HADOOP-9866. convert hadoop-auth testcases requiring kerberos to
use minikdc. (ywskycn via tucu)
HADOOP-9487 Deprecation warnings in Configuration should go to their
own log or otherwise be suppressible (Chu Tong via stevel)
HADOOP-9889. Refresh the Krb5 configuration when creating a new kdc in
Hadoop-MiniKDC (Wei Yan via Sandy Ryza)
HADOOP-9915. o.a.h.fs.Stat support on Mac OS X (Binglin Chang via Colin
Patrick McCabe)
HADOOP-9998. Provide methods to clear only part of the DNSToSwitchMapping.
(Junping Du via Colin Patrick McCabe)
HADOOP-9063. enhance unit-test coverage of class
org.apache.hadoop.fs.FileUtil (Ivan A. Veselovsky via jlowe)
HADOOP-9254. Cover packages org.apache.hadoop.util.bloom,
org.apache.hadoop.util.hash (Vadim Bondarev via jlowe)
HADOOP-9225. Cover package org.apache.hadoop.compress.Snappy (Vadim
Bondarev, Andrey Klochkov and Nathan Roberts via jlowe)
HADOOP-9199. Cover package org.apache.hadoop.io with unit tests (Andrey
Klochkov via jeagles)
HADOOP-9470. eliminate duplicate FQN tests in different Hadoop modules
(Ivan A. Veselovsky via daryn)
HADOOP-9494. Excluded auto-generated and examples code from clover reports
(Andrey Klochkov via jeagles)
HADOOP-9897. Add method to get path start position without drive specifier in
o.a.h.fs.Path. (Binglin Chang via cnauroth)
HADOOP-9078. enhance unit-test coverage of class
org.apache.hadoop.fs.FileContext (Ivan A. Veselovsky via jeagles)
HDFS-5276. FileSystem.Statistics should use thread-local counters to avoid
multi-threaded performance issues on read/write. (Colin Patrick McCabe)
HADOOP-9291. enhance unit-test coverage of package o.a.h.metrics2 (Ivan A.
Veselovsky via jeagles)
HADOOP-10064. Upgrade to maven antrun plugin version 1.7 (Arpit Agarwal via
jeagles)
HADOOP-9594. Update apache commons math dependency (Timothy St. Clair via
stevel)
HADOOP-10095. In CodecPool, synchronize pool and codecList separately in
order to reduce lock contention. (Nicolas Liochon via szetszwo)
HADOOP-10067. Missing POM dependency on jsr305 (Robert Rati via stevel)
HADOOP-10103. update commons-lang to 2.6 (Akira AJISAKA via stevel)
HADOOP-10111. Allow DU to be initialized with an initial value (Kihwal Lee
via jeagles)
HADOOP-10126. LightWeightGSet log message is confusing. (Vinay via suresh)
HADOOP-10127. Add ipc.client.connect.retry.interval to control the frequency
of connection retries (Karthik Kambatla via Sandy Ryza)
HADOOP-10102. Update commons IO from 2.1 to 2.4 (Akira Ajisaka via stevel)
HADOOP-10168. fix javadoc of ReflectionUtils#copy. (Thejas Nair via suresh)
HADOOP-10164. Allow UGI to login with a known Subject (bobby)
HADOOP-10169. Remove the unnecessary synchronized in JvmMetrics class.
(Liang Xie via jing9)
HADOOP-10198. DomainSocket: add support for socketpair.
(Colin Patrick McCabe via wang)
HADOOP-10208. Remove duplicate initialization in StringUtils.getStringCollection.
(Benoy Antony via jing9)
HADOOP-9420. Add percentile or max metric for rpcQueueTime, processing time.
(Liang Xie via wang)
HADOOP-9652. Allow RawLocalFs#getFileLinkStatus to fill in the link owner
and mode if requested. (Andrew Wang via Colin Patrick McCabe)
HADOOP-10305. Add "rpc.metrics.quantile.enable" and
"rpc.metrics.percentiles.intervals" to core-default.xml.
(Akira Ajisaka via wang)
HADOOP-10317. Rename branch-2.3 release version from 2.4.0-SNAPSHOT
to 2.3.0-SNAPSHOT. (wang)
HADOOP-10313. Script and jenkins job to produce Hadoop release artifacts.
(tucu)
OPTIMIZATIONS
HADOOP-10142. Avoid groups lookup for unprivileged users such as "dr.who"
(vinay via cmccabe)
HADOOP-9748. Reduce blocking on UGI.ensureInitialized (daryn)
HADOOP-10047. Add a direct-buffer based apis for compression. (Gopal V
via acmurthy)
HADOOP-10172. Cache SASL server factories (daryn)
HADOOP-10173. Remove UGI from DIGEST-MD5 SASL server creation (daryn via
kihwal)
HADOOP-10228. FsPermission#fromShort() should cache FsAction.values().
(Haohui Mai via cnauroth)
HADOOP-10143 replace WritableFactories's hashmap with ConcurrentHashMap
(Liang Xie via stack)
BUG FIXES
HADOOP-10028. Malformed ssl-server.xml.example. (Haohui Mai via jing9)
HADOOP-10030. FsShell -put/copyFromLocal should support Windows local path.
(Chuan Liu via cnauroth)
HADOOP-10031. FsShell -get/copyToLocal/moveFromLocal should support Windows
local path. (Chuan Liu via cnauroth)
HADOOP-10039. Add Hive to the list of projects using
AbstractDelegationTokenSecretManager. (Haohui Mai via jing9)
HADOOP-10040. hadoop.cmd in UNIX format and would not run by default on
Windows. (cnauroth)
HADOOP-10055. FileSystemShell.apt.vm doc has typo "numRepicas".
(Akira Ajisaka via cnauroth)
HADOOP-10072. TestNfsExports#testMultiMatchers fails due to non-deterministic
timing around cache expiry check. (cnauroth)
HADOOP-9898. Set SO_KEEPALIVE on all our sockets. (todd via wang)
HADOOP-9478. Fix race conditions during the initialization of Configuration
related to deprecatedKeyMap (cmccabe)
HADOOP-9660. [WINDOWS] Powershell / cmd parses -Dkey=value from command line
as [-Dkey, value] which breaks GenericsOptionParser.
(Enis Soztutar via cnauroth)
HADOOP-10078. KerberosAuthenticator always does SPNEGO. (rkanter via tucu)
HADOOP-10110. hadoop-auth has a build break due to missing dependency.
(Chuan Liu via arp)
HADOOP-9114. After defined the dfs.checksum.type as the NULL, write file and hflush will
through java.lang.ArrayIndexOutOfBoundsException (Sathish via umamahesh)
HADOOP-10130. RawLocalFS::LocalFSFileInputStream.pread does not track
FS::Statistics (Binglin Chang via Colin Patrick McCabe)
HDFS-5560. Trash configuration log statements prints incorrect units.
(Josh Elser via Andrew Wang)
HADOOP-10081. Client.setupIOStreams can leak socket resources on exception
or error (Tsuyoshi OZAWA via jlowe)
HADOOP-10087. UserGroupInformation.getGroupNames() fails to return primary
group first when JniBasedUnixGroupsMappingWithFallback is used (cmccabe)
HADOOP-10175. Har files system authority should preserve userinfo.
(Chuan Liu via cnauroth)
HADOOP-10090. Jobtracker metrics not updated properly after execution
of a mapreduce job. (ivanmi)
HADOOP-10193. hadoop-auth's PseudoAuthenticationHandler can consume getInputStream.
(gchanan via tucu)
HADOOP-10178. Configuration deprecation always emit "deprecated" warnings
when a new key is used. (Shanyu Zhao via cnauroth)
HADOOP-10234. "hadoop.cmd jar" does not propagate exit code. (cnauroth)
HADOOP-10240. Windows build instructions incorrectly state requirement of
protoc 2.4.1 instead of 2.5.0. (cnauroth)
HADOOP-10110. hadoop-auth has a build break due to missing dependency
(Chuan Liu via cos)
HADOOP-10167. Mark hadoop-common source as UTF-8 in Maven pom files / refactoring
(Mikhail Antonov via cos)
HADOOP-9982. Fix dead links in hadoop site docs. (Akira Ajisaka via Arpit
Agarwal)
HADOOP-10212. Incorrect compile command in Native Library document.
(Akira Ajisaka via Arpit Agarwal)
HADOOP-9830. Fix typo at http://hadoop.apache.org/docs/current/
(Kousuke Saruta via Arpit Agarwal)
HADOOP-10255. Rename HttpServer to HttpServer2 to retain older
HttpServer in branch-2 for compatibility. (Haohui Mai via suresh)
HADOOP-10292. Restore HttpServer from branch-2.2 in branch-2.
(Haohui Mai via suresh)
HADOOP-10291. TestSecurityUtil#testSocketAddrWithIP fails due to test
order dependency. (Mit Desai via Arpit Agarwal)
HADOOP-9964. Fix deadlocks in TestHttpServer by synchronize
ReflectionUtils.printThreadInfo. (Junping Du via llu)
HADOOP-9582. Non-existent file to "hadoop fs -conf" doesn't throw error
(Ashwin Shankar via jlowe)
HADOOP-9817. FileSystem#globStatus and FileContext#globStatus need to work
with symlinks. (Colin Patrick McCabe via Andrew Wang)
HADOOP-9875. TestDoAsEffectiveUser can fail on JDK 7. (Aaron T. Myers via
Colin Patrick McCabe)
HADOOP-9865. FileContext#globStatus has a regression with respect to
relative path. (Chuan Lin via Colin Patrick McCabe)
HADOOP-9909. org.apache.hadoop.fs.Stat should permit other LANG.
(Shinichi Yamashita via Andrew Wang)
HADOOP-9908. Fix NPE when versioninfo properties file is missing (todd)
HADOOP-9929. Insufficient permissions for a path reported as File Not
Found. (Contributed by Colin Patrick McCabe)
HADOOP-9791. Add a test case covering long paths for new FileUtil access
check methods (ivanmi)
HADOOP-9981. globStatus should minimize its listStatus and getFileStatus
calls. (Contributed by Colin Patrick McCabe)
HADOOP-10006. Compilation failure in trunk for
o.a.h.fs.swift.util.JSONUtil (Junping Du via stevel)
HADOOP-9016. HarFsInputStream.skip(long) must never return negative value.
(Ivan A. Veselovsky via jeagles)
HADOOP-10088. copy-nativedistlibs.sh needs to quote snappy lib dir.
(Raja Aluri via cnauroth)
HADOOP-10093. hadoop-env.cmd sets HADOOP_CLIENT_OPTS with a max heap size
that is too small. (Shanyu Zhao via cnauroth)
HADOOP-10094. NPE in GenericOptionsParser#preProcessForWindows().
(Enis Soztutar via cnauroth)
HADOOP-10100. MiniKDC shouldn't use apacheds-all artifact. (rkanter via tucu)
HADOOP-10107. Server.getNumOpenConnections may throw NPE. (Kihwal Lee via
jing9)
HADOOP-10135 writes to swift fs over partition size leave temp files and
empty output file (David Dobbins via stevel)
HADOOP-10129. Distcp may succeed when it fails (daryn)
HADOOP-10058. TestMetricsSystemImpl#testInitFirstVerifyStopInvokedImmediately
fails on trunk (Chen He via jeagles)
HADOOP-10162. Fix symlink-related test failures in
TestFileContextResolveAfs and TestStat in branch-2 (Mit Desai via Colin
Patrick McCabe)
HADOOP-8753. LocalDirAllocator throws "ArithmeticException: / by zero" when
there is no available space on configured local dir. (Benoy Antony via hitesh)
HADOOP-10106. Incorrect thread name in RPC log messages. (Ming Ma via jing9)
HADOOP-9611 mvn-rpmbuild against google-guice > 3.0 yields missing cglib
dependency (Timothy St. Clair via stevel)
HADOOP-10171. TestRPC fails intermittently on jkd7 (Mit Desai via jeagles)
HADOOP-10147 HDFS-5678 Upgrade to commons-logging 1.1.3 to avoid potential
deadlock in MiniDFSCluster (stevel)
HADOOP-10207. TestUserGroupInformation#testLogin is flaky (jxiang via
cmccabe)
HADOOP-10214. Fix multithreaded correctness warnings in ActiveStandbyElector
(Liang Xie via kasha)
HADOOP-10223. MiniKdc#main() should close the FileReader it creates.
(Ted Yu via tucu)
HADOOP-10236. Fix typo in o.a.h.ipc.Client#checkResponse. (Akira Ajisaka
via suresh)
HADOOP-10146. Workaround JDK7 Process fd close bug (daryn)
HADOOP-10125. no need to process RPC request if the client connection
has been dropped (Ming Ma via brandonli)
HADOOP-10235. Hadoop tarball has 2 versions of stax-api JARs. (tucu)
HADOOP-10252. HttpServer can't start if hostname is not specified. (Jimmy
Xiang via atm)
HADOOP-10203. Connection leak in
Jets3tNativeFileSystemStore#retrieveMetadata. (Andrei Savu via atm)
HADOOP-10250. VersionUtil returns wrong value when comparing two versions.
(Yongjun Zhang via atm)
HADOOP-10288. Explicit reference to Log4JLogger breaks non-log4j users
(todd)
HADOOP-10310. SaslRpcServer should be initialized even when no secret
manager present. (atm)
HADOOP-10311. Cleanup vendor names from the code base. (tucu)
HADOOP-10273. Fix 'mvn site'. (Arpit Agarwal)
Release 2.2.0 - 2013-10-13
INCOMPATIBLE CHANGES
HADOOP-10020. Disable symlinks temporarily (cmccabe)
NEW FEATURES
IMPROVEMENTS
HADOOP-9948. Add a config value to CLITestHelper to skip tests on Windows.
(Chuan Liu via cnauroth)
HADOOP-9976. Different versions of avro and avro-maven-plugin (Karthik
Kambatla via Sandy Ryza)
HADOOP-9758. Provide configuration option for FileSystem/FileContext
symlink resolution. (Andrew Wang via Colin Patrick McCabe)
HADOOP-8315. Support SASL-authenticated ZooKeeper in ActiveStandbyElector
(todd)
OPTIMIZATIONS
BUG FIXES
HADOOP-9776. HarFileSystem.listStatus() returns invalid authority if port
number is empty. (Shanyu Zhao via ivanmi)
HADOOP-9761. ViewFileSystem#rename fails when using DistributedFileSystem.
(Andrew Wang via Colin Patrick McCabe)
HADOOP-10003. HarFileSystem.listLocatedStatus() fails.
(Jason Dere and suresh via suresh)
HADOOP-10017. Fix NPE in DFSClient#getDelegationToken when doing Distcp
from a secured cluster to an insecured cluster. (Haohui Mai via jing9)
Release 2.1.1-beta - 2013-09-23
INCOMPATIBLE CHANGES
HADOOP-9944. Fix RpcRequestHeaderProto.callId to be sint32 rather than
uint32 since ipc.Client.CONNECTION_CONTEXT_CALL_ID is signed (i.e. -3)
(acmurthy)
NEW FEATURES
IMPROVEMENTS
HADOOP-9910. proxy server start and stop documentation wrong
(Andre Kelpe via harsh)
HADOOP-9446. Support Kerberos SPNEGO for IBM JDK. (Yu Gao via llu)
HADOOP-9787. ShutdownHelper util to shutdown threads and threadpools.
(Karthik Kambatla via Sandy Ryza)
HADOOP-9803. Add a generic type parameter to RetryInvocationHandler.
(szetszwo)
HADOOP-9821. ClientId should have getMsb/getLsb methods.
(Tsuyoshi OZAWA via jing9)
HADOOP-9435. Support building the JNI code against the IBM JVM.
(Tian Hong Wang via Colin Patrick McCabe)
HADOOP-9833 move slf4j to version 1.7.5 (Kousuke Saruta via stevel)
HADOOP-9672. Upgrade Avro dependency to 1.7.4. (sandy via kihwal)
HADOOP-9789. Support server advertised kerberos principals (daryn)
HADOOP-8814. Replace string equals "" by String#isEmpty().
(Brandon Li via suresh)
HADOOP-9802. Support Snappy codec on Windows. (cnauroth)
HADOOP-9879. Move the version info of zookeeper dependencies to
hadoop-project/pom (Karthik Kambatla via Sandy Ryza)
HADOOP-9886. Turn warning message in RetryInvocationHandler to debug (arpit)
HADOOP-9906. Move HAZKUtil to o.a.h.util.ZKUtil and make inner-classes
public (Karthik Kambatla via Sandy Ryza)
HADOOP-9918. Add addIfService to CompositeService (Karthik Kambatla via
Sandy Ryza)
HADOOP-9945. HAServiceState should have a state for stopped services.
(Karthik Kambatla via atm)
HADOOP-9962. in order to avoid dependency divergence within Hadoop itself
lets enable DependencyConvergence. (rvs via tucu)
HADOOP-9669. Reduce the number of byte array creations and copies in
XDR data manipulation. (Haohui Mai via brandonli)
OPTIMIZATIONS
BUG FIXES
HADOOP-9916. Fix race in ipc.Client retry. (Binglin Chang via llu)
HADOOP-9768. chown and chgrp reject users and groups with spaces on platforms
where spaces are otherwise acceptable. (cnauroth)
HADOOP-9801. Configuration#writeXml uses platform defaulting encoding, which
may mishandle multi-byte characters. (cnauroth)
HADOOP-9806 PortmapInterface should check if the procedure is out-of-range
(brandonli)
HADOOP-9527. Add symlink support to LocalFileSystem on Windows.
(Arpit Agarwal via cnauroth)
HADOOP-9315. Port HADOOP-9249 hadoop-maven-plugins Clover fix to branch-2 to
fix build failures. (Dennis Y via cnauroth)
HADOOP-9831. Make checknative shell command accessible on Windows. (cnauroth)
HADOOP-9675. use svn:eol-style native for html to prevent line ending
issues (Colin Patrick McCabe)
HADOOP-9757. Har metadata cache can grow without limit (Cristina Abad via daryn)
HADOOP-9858. Remove unused private RawLocalFileSystem#execCommand method from
branch-2. (cnauroth)
HADOOP-9857. Tests block and sometimes timeout on Windows due to invalid
entropy source. (cnauroth)
HADOOP-9381. Document dfs cp -f option. (Keegan Witt, suresh via suresh)
HADOOP-9868. Server must not advertise kerberos realm. (daryn via kihwal)
HADOOP-9880. SASL changes from HADOOP-9421 breaks Secure HA NN. (daryn via
jing9)
HADOOP-9887. globStatus does not correctly handle paths starting with a drive
spec on Windows. (Chuan Liu via cnauroth)
HADOOP-9899. Remove the debug message, added by HADOOP-8855, from
KerberosAuthenticator. (szetszwo)
HADOOP-9894. Race condition in Shell leads to logged error stream handling
exceptions (Arpit Agarwal)
HADOOP-9774. RawLocalFileSystem.listStatus() return absolute paths when
input path is relative on Windows. (Shanyu Zhao via ivanmi)
HADOOP-9924. FileUtil.createJarWithClassPath() does not generate relative
classpath correctly. (Shanyu Zhao via ivanmi)
HADOOP-9932. Improper synchronization in RetryCache. (kihwal)
HADOOP-9958. Add old constructor back to DelegationTokenInformation to
unbreak downstream builds. (Andrew Wang)
HADOOP-9960. Upgrade Jersey version to 1.9. (Karthik Kambatla via atm)
HADOOP-9557. hadoop-client excludes commons-httpclient. (Lohit Vijayarenu via
cnauroth)
HADOOP-9350. Hadoop not building against Java7 on OSX
(Robert Kanter via stevel)
HADOOP-9961. versions of a few transitive dependencies diverged between hadoop
subprojects. (rvs via tucu)
HADOOP-9977. Hadoop services won't start with different keypass and
keystorepass when https is enabled. (cnauroth)
HADOOP-10005. No need to check INFO severity level is enabled or not.
(Jackie Chang via suresh)
Release 2.1.0-beta - 2013-08-22
INCOMPATIBLE CHANGES
HADOOP-8886. Remove KFS support. (eli)
HADOOP-9163. [RPC v9] The rpc msg in ProtobufRpcEngine.proto should be moved out to
avoid an extra copy (Sanjay Radia)
HADOOP-9151. [RPC v9] Include RPC error info in RpcResponseHeader instead of sending
it separately (sanjay Radia)
HADOOP-9380. [RPC v9] Add totalLength to rpc response (sanjay Radia)
HADOOP-9425. [RPC v9] Add error codes to rpc-response (sanjay Radia)
HADOOP-9194. [RPC v9] RPC support for QoS. (Junping Du via llu)
HADOOP-9630. [RPC v9] Remove IpcSerializationType. (Junping Du via llu)
HADOOP-9421. [RPC v9] Convert SASL to use ProtoBuf and provide
negotiation capabilities (daryn)
HADOOP-9688. Add globally unique Client ID to RPC requests. (suresh)
HADOOP-9683. [RPC v9] Wrap IpcConnectionContext in RPC headers (daryn)
HADOOP-9698. [RPC v9] Client must honor server's SASL negotiate response (daryn)
HADOOP-9832. [RPC v9] Add RPC header to client ping (daryn)
HADOOP-9820. [RPC v9] Wire protocol is insufficient to support multiplexing. (daryn via jitendra)
NEW FEATURES
HADOOP-9283. Add support for running the Hadoop client on AIX. (atm)
HADOOP-8415. Add getDouble() and setDouble() in
org.apache.hadoop.conf.Configuration (Jan van der Lugt via harsh)
HADOOP-9338. FsShell Copy Commands Should Optionally Preserve File
Attributes. (Nick White via atm)
HADOOP-8562. Enhancements to support Hadoop on Windows Server and Windows
Azure environments. (See breakdown of tasks below for subtasks and
contributors)
HADOOP-8469. Make NetworkTopology class pluggable. (Junping Du via
szetszwo)
HADOOP-8470. Add NetworkTopologyWithNodeGroup, a 4-layer implementation
of NetworkTopology. (Junping Du via szetszwo)
HADOOP-9763. Extends LightWeightGSet to support eviction of expired
elements. (Tsz Wo (Nicholas) SZE via jing9)
HADOOP-9762. RetryCache utility for implementing RPC retries.
(Suresh Srinivas via jing9)
HADOOP-9792. Retry the methods that are tagged @AtMostOnce along
with @Idempotent. (suresh)
HADOOP-9509. Implement ONCRPC and XDR. (brandonli)
HADOOP-9515. Add general interface for NFS and Mount. (brandonli)
IMPROVEMENTS
HADOOP-9164. Print paths of loaded native libraries in
NativeLibraryChecker. (Binglin Chang via llu)
HADOOP-9253. Capture ulimit info in the logs at service start time.
(Arpit Gupta via suresh)
HADOOP-8924. Add maven plugin alternative to shell script to save
package-info.java. (Chris Nauroth via suresh)
HADOOP-9117. replace protoc ant plugin exec with a maven plugin. (tucu)
HADOOP-9279. Document the need to build hadoop-maven-plugins for
eclipse and separate project builds. (Tsuyoshi Ozawa via suresh)
HADOOP-9334. Upgrade netty version. (Nicolas Liochon via suresh)
HADOOP-9343. Allow additional exceptions through the RPC layer. (sseth)
HADOOP-9318. When exiting on a signal, print the signal name first. (Colin
Patrick McCabe via atm)
HADOOP-9358. "Auth failed" log should include exception string (todd)
HADOOP-9401. CodecPool: Add counters for number of (de)compressors
leased out. (kkambatl via tucu)
HADOOP-9450. HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH
is PREpended instead of APpended. (Chris Nauroth and harsh via harsh)
HADOOP-9496. Bad merge of HADOOP-9450 on branch-2 breaks all bin/hadoop
calls that need HADOOP_CLASSPATH. (harsh)
HADOOP-9503. Remove sleep between IPC client connect timeouts.
(Varun Sharma via szetszwo)
HADOOP-9322. LdapGroupsMapping doesn't seem to set a timeout for
its directory search. (harsh)
HADOOP-9523. Provide a generic IBM java vendor flag in PlatformName.java
to support non-Sun JREs. (Tian Hong Wang via suresh)
HADOOP-9511. Adding support for additional input streams (FSDataInputStream
and RandomAccessFile) in SecureIOUtils so as to help YARN-578. (Omkar Vinit
Joshi via vinodkv)
HADOOP-9560. metrics2#JvmMetrics should have max memory size of JVM.
(Tsuyoshi Ozawa via suresh)
HADOOP-9140 Cleanup rpc PB protos (sanjay Radia)
HADOOP-9218 Document the Rpc-wrappers used internally (sanjay Radia)
HADOOP-9574. Added new methods in AbstractDelegationTokenSecretManager for
helping YARN ResourceManager to reuse code for RM restart. (Jian He via
vinodkv)
HADOOP-7391 Document Interface Classification from HADOOP-5073 (sanjay Radia)
HADOOP-9287. Parallel-testing hadoop-common (Andrey Klochkov via jlowe)
HADOOP-9604. Javadoc of FSDataOutputStream is slightly inaccurate. (Jingguo
Yao via atm)
HADOOP-9625. HADOOP_OPTS not picked up by hadoop command.
(Paul Han via arpit)
HADOOP-9649. Promoted YARN service life-cycle libraries into Hadoop Common
for usage across all Hadoop projects. (Zhijie Shen via vinodkv)
HADOOP-9517. Documented various aspects of compatibility for Apache
Hadoop. (Karthik Kambatla via acmurthy)
HADOOP-8608. Add Configuration API for parsing time durations. (cdouglas)
HADOOP-9619 Mark stability of .proto files (sanjay Radia)
HADOOP-9676. Make maximum RPC buffer size configurable (Colin Patrick
McCabe)
HADOOP-9691. RPC clients can generate call ID using AtomicInteger instead of
synchronizing on the Client instance. (cnauroth)
HADOOP-9661. Allow metrics sources to be extended. (sandyr via tucu)
HADOOP-9370. Write FSWrapper class to wrap FileSystem and FileContext for
better test coverage. (Andrew Wang via Colin Patrick McCabe)
HADOOP-9355. Abstract symlink tests to use either FileContext or
FileSystem. (Andrew Wang via Colin Patrick McCabe)
HADOOP-9673. NetworkTopology: when a node can't be added, print out its
location for diagnostic purposes. (Colin Patrick McCabe)
HADOOP-9414. Refactor out FSLinkResolver and relevant helper methods.
(Andrew Wang via Colin Patrick McCabe)
HADOOP-9416. Add new symlink resolution methods in FileSystem and
FileSystemLinkResolver. (Andrew Wang via Colin Patrick McCabe)
HADOOP-9720. Rename Client#uuid to Client#clientId. (Arpit Agarwal via
suresh)
HADOOP-9734. Common protobuf definitions for GetUserMappingsProtocol,
RefreshAuthorizationPolicyProtocol and RefreshUserMappingsProtocol (jlowe)
HADOOP-9716. Rpc retries should use the same call ID as the original call.
(szetszwo)
HADOOP-9717. Add retry attempt count to the RPC requests. (jing9)
HADOOP-9751. Add clientId and retryCount to RpcResponseHeaderProto.
(szetszwo)
HADOOP-9754. Remove unnecessary "throws IOException/InterruptedException",
and fix generic and other javac warnings. (szetszwo)
HADOOP-9760. Move GSet and related classes to common from HDFS.
(suresh)
HADOOP-9756. Remove the deprecated getServer(..) methods from RPC.
(Junping Du via szetszwo)
HADOOP-9770. Make RetryCache#state non volatile. (suresh)
HADOOP-9786. RetryInvocationHandler#isRpcInvocation should support
ProtocolTranslator. (suresh and jing9)
HADOOP-9847. TestGlobPath symlink tests fail to cleanup properly.
(cmccabe via wang)
OPTIMIZATIONS
HADOOP-9150. Avoid unnecessary DNS resolution attempts for logical URIs
(todd)
HADOOP-9845. Update protobuf to 2.5 from 2.4.x. (tucu)
HADOOP-9872. Improve protoc version handling and detection. (tucu)
BUG FIXES
HADOOP-9451. Fault single-layer config if node group topology is enabled.
(Junping Du via llu)
HADOOP-9294. GetGroupsTestBase fails on Windows. (Chris Nauroth via suresh)
HADOOP-9305. Add support for running the Hadoop client on 64-bit AIX. (atm)
HADOOP-9245. mvn clean without running mvn install before fails.
(Karthik Kambatla via suresh)
HADOOP-9246 Execution phase for hadoop-maven-plugin should be
process-resources (Karthik Kambatla and Chris Nauroth via jlowe)
HADOOP-9297. remove old record IO generation and tests. (tucu)
HADOOP-9154. SortedMapWritable#putAll() doesn't add key/value classes to
the map. (Karthik Kambatla via tomwhite)
HADOOP-9304. remove addition of avro genreated-sources dirs to build. (tucu)
HADOOP-9267. hadoop -help, -h, --help should show usage instructions.
(Andrew Wang via atm)