forked from apache/sqoop
/
CHANGELOG.txt
1694 lines (847 loc) · 59 KB
/
CHANGELOG.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Release 1.99.6
SUB-TASK
SQOOP-1528: Sqoop2: Kerberos support on DoAs function
SQOOP-1757: Sqoop2: Document generic jdbc connector
SQOOP-1961: Sqoop2: addDelegationTokens is not exposed in SqoopClient
SQOOP-1978: Sqoop2: Command line support
SQOOP-1979: Sqoop2: Rest API support
SQOOP-1998: Sqoop2: Kite Connector: Hive support
SQOOP-2016: Sqoop2: Create integration test for JDBC to Hive
SQOOP-2019: Sqoop2: Handle post-function for updating/deleting resource
SQOOP-2079: Sqoop2: Remove useless functions for RBAC
SQOOP-2080: Sqoop2: Combine privileges into READ and WRITE
SQOOP-2092: Support for LONG in MInputType
SQOOP-2094: Sqoop2: Update RESTAPI.rst
SQOOP-2101: Sqoop2: Add SqoopClient support to call Restful API
SQOOP-2139: Sqoop2: Change create role API from PUT to POST
SQOOP-2144: Sqoop2: Show command for CLI
SQOOP-2146: Sqoop2: Sqoop client should return model objects for RBAC methods
SQOOP-2148: Sqoop2: Fix wrong input sequence of name and type in function new Principal and Resource
SQOOP-2160: Sqoop2: Datatypes: Provide foundation for the exhaustive type checks
SQOOP-2166: Sqoop2: Add "Authorization Exception" error code in SecurityError.java
SQOOP-2183: Sqoop2: Change resource type, privilege action and principal type from String to Enum.
SQOOP-2184: Sqoop2: Add ServerName for authorization handler
SQOOP-2198: Sqoop2: Datatypes: Add boolean/char/date support to DerbyTypeList
SQOOP-2204: Sanitize issues
SQOOP-2205: Cut 1.99.6 branch
SQOOP-2206: Update change log with 1.99.6 release
SQOOP-2207: Update POM with new version
SQOOP-2208: Create release candidate
SQOOP-2209: Run vote on RC candidates
SQOOP-2210: Release artifacts
SQOOP-2211: Finish release
SQOOP-2212: Update website
SQOOP-2220: Sqoop2: Fix the privilege check on job using the link id not the connector id
SQOOP-2226: Sqoop2: Access denied in shell
SQOOP-2227: Sqoop2: Clear connector cache in ShowConnectorFunction.java
SQOOP-2246: Sqoop2: Use jdbcProperties when creating database connection in GenericJDBCExecutor
SQOOP-2248: Sqoop2: ErrorCodes: Move ServerError to server module
SQOOP-2249: Sqoop2: ErrorCodes: Move SecurityError to core module
SQOOP-2251: Sqoop2: ErrorCodes: Move RepositoryError to core module
SQOOP-2255: Sqoop2: Throw Exception when no permission to the resource
SQOOP-2256: Sqoop2: Creator should have access to its entities
SQOOP-2304: Update license file
SQOOP-2313: Update change log with 1.99.6 release for RC-1
SQOOP-2314: Create release candidate for RC-1
SQOOP-2315: Run vote on RC candidates for RC-1
BUG
SQOOP-1439: Sqoop2: Refactor validations
SQOOP-1516: Sqoop2: Config Input as a Top Level Entity - RepositoryAPI changes
SQOOP-1577: Sqoop2: Refactor repository dump and load tools
SQOOP-1579: Sqoop2: Data transfer to load into Hive does not work
SQOOP-1738: Sqoop2: HDFS Connector : Check for output directory
SQOOP-1746: Sqoop2: Mapreduce job is running in DEBUG mode by default
SQOOP-1790: Sqoop2: Upgrade configs with name conflicts
SQOOP-1808: Add Integration tests to for Incremental read / write between JDBC and HDFS
SQOOP-1841: Sqoop2: Upgrade from 1.99.1 to 1.99.4 onwards is broken
SQOOP-1868: Sqoop2: Document MySQL Fast Connector
SQOOP-1949: Sqoop2: HDFS append only support
SQOOP-2036: Sqoop2: JDBC Connector need to send BigDecimal for decimal type in Object array
SQOOP-2089: Sqoop2: Clone configuration object instead of overwriting global
SQOOP-2095: Sqoop2: Remove duplicate maven dependency org.mockito:mockito-all
SQOOP-2102: Binary tarball missing executable bit on sqoop2-* files
SQOOP-2106: Upgrade the version of Derby to 6 and Postgres to 1 in sqoop trunk
SQOOP-2117: Sqoop2: Shell command 'show link' should show connector name
SQOOP-2119: Sqoop2: Remove remaining imports from junit in PostgreSQL repository tests
SQOOP-2120: Sqoop2: Test classes using powermock fails badly when compiling with java target version 1.7
SQOOP-2122: Sqoop2: Build documentation only in binary packaging profile
SQOOP-2123: Sqoop2: Copy shell libraries only in binary package profile
SQOOP-2124: Sqoop2: Pre-Commit hook should report how many tests are executed
SQOOP-2129: Sqoop2: SQ_SUBMISSION deletes should cascade
SQOOP-2140: Sqoop2: Cast exception when try to cast an integer to Avro's Long type
SQOOP-2150: Sqoop2: Kite 1.0 don't allow dashes to be used in dataset names
SQOOP-2152: Sqoop2: Kite connector always assumes that FixedPoint is long regardless of the configured size
SQOOP-2153: Sqoop2: Ensure creation of valid Avro schema names
SQOOP-2155: Sqoop2: KiteDatasetExecutor is not cleaning up merged datasets
SQOOP-2156: Sqoop2: HdfsUtils.getOutputMapreduceFiles should ignore hidden files
SQOOP-2157: Sqoop2: Kite: Add simple integration test for TO direction
SQOOP-2158: Sqoop2: Pre-Commit hook is incorrectly counting number of executed tests
SQOOP-2159: Sqoop2: Use BeforeSuite/AfterSuite for startHadoop/stopHadoop in integration tests
SQOOP-2162: Sqoop2: InputEditable should be optional in JSON API
SQOOP-2163: Sqoop2: Remove jackson dependency
SQOOP-2167: Sqoop2: Change getUserName to getShortUserName in AuthorizationEngine.java
SQOOP-2169: Sqoop2: Kafka integration test bug
SQOOP-2171: Sqoop2: Incorrect basic type boxing with JDK1.7
SQOOP-2172: Sqoop2: Move ErrorCode's sub-class from sqoop-core to sqoop-common
SQOOP-2188: Sqoop2: Failed to merge Kite datasets
SQOOP-2190: Sqoop2: Remove hadoop200 profile from test-patch.py
SQOOP-2201: Sqoop2: Add possibility to read Hadoop configuration files to HFDS connector
SQOOP-2202: Sqoop2: Add validator to check existence of local directory on Sqoop 2 server
SQOOP-2213: Sqoop2: toCSVFixedPoint ClassCastException
SQOOP-2214: Sqoop2: Add ability to easily iterate over entries stored in Context
SQOOP-2215: Sqoop2: Remember all test logs in pre-commit hook
SQOOP-2219: Sqoop2: Skip missing input in DirectoryExistsValidator
SQOOP-2225: Sqoop2: Shell client clobbers server exceptions
SQOOP-2228: Sqoop2: HDFS Connector: Import data to temporary directory before moving them to target directory
SQOOP-2229: Sqoop2: Kite connector should handle doubles
SQOOP-2230: Sqoop2: Kite connector can use URIBuilder
SQOOP-2231: Sqoop2: Kite connector should use authority in link configuration
SQOOP-2232: Sqoop2: Test initializer against local filesystem
SQOOP-2233: Sqoop2: Move TestHdfsConnector class under test directory
SQOOP-2234: Sqoop2: Move DatasetURIValidator to Kite package as it's not generally usable validator
SQOOP-2235: Sqoop2: Move PrefixContext back to mapreduce execution engine
SQOOP-2236: Sqoop2: Drop outdated comment from SubmissionCallbacks class
SQOOP-2238: Sqoop2: Parquet jars missing
SQOOP-2239: Sqoop2: Remove Repository.updateSubmission() call from Mapreduce submission engine
SQOOP-2240: Sqoop2: Remove Hadoop 1.x example from catalina.properties
SQOOP-2241: Sqoop2: PreCommit hook is incorrectly counting number of executed tests
SQOOP-2242: Sqoop2: Add support for full table names into our DatabaseProvider infrastructure
SQOOP-2265: Sqoop2: Standardize on methods
SQOOP-2268: Sqoop2: Fix a typo in ShowLinkFunction
SQOOP-2273: Sqoop2: Move authorizationServlet into v1 package
SQOOP-2276: MapReduce execution error in Sqoop job
SQOOP-2277: Sqoop2: Add annotation for public usage classes in Sqoop Core
SQOOP-2278: Sqoop2: Fix bug for all user could read connector
SQOOP-2279: Sqoop2: Clean connector cache after grant and revoke statements
SQOOP-2307: Sqoop2: Fix a typo for hdfs-connector-config.properties and kafka-connector-config.properties
SQOOP-2308: Sqoop2: Fix a typo for hdfs-connector-config.properties and kafka-connector-config.properties
SQOOP-2309: Sqoop2: Add to link id in show job --all command
SQOOP-2310: Sqoop2: fix a typo for Hadoop conf directory
SQOOP-2327: Sqoop2: Change package name from Authorization to authorization
IMPROVEMENT
SQOOP-1805: Sqoop2: GenericJdbcConnector: Delta read support
SQOOP-2134: Update Kite dependency to 1.0.0
SQOOP-2250: Sqoop2: SQ_LINK_INPUT is missing a foreign key constraint
TASK
SQOOP-506: Create Sqoop integration test infrastructure.
Release 1.99.5
SUB-TASK
SQOOP-1348: Sqoop2: Remove Data class
SQOOP-1349: Sqoop2: Use configurable writable to get Intermediate Data Format
SQOOP-1526: Sqoop2: Kerberos support when starting service
SQOOP-1527: Sqoop2: Kerberos support (SPNEGO) in communication between server and client
SQOOP-1549: Simplifying the Configuration class concept in Connector api
SQOOP-1552: Revisit if we need a config list for the link/from/to configs in connector api
SQOOP-1588: Sqoop2: Kite connector write data to HDFS
SQOOP-1589: Sqoop2: Create common constants, error codes, and queries
SQOOP-1590: Sqoop2: PostgreSQL repository implementation
SQOOP-1591: Sqoop2: PostgreSQL integration tests
SQOOP-1592: Sqoop2: Remove Hadoop1 profile
SQOOP-1602: Sqoop2: Fix the current balancing across Loaders internal to Sqoop
SQOOP-1621: Sqoop2: Allow null as a dummy Schema
SQOOP-1624: Sqoop2: Apply repository upgrader api
SQOOP-1647: Sqoop2: Read data from HDFS in KiteConnector
SQOOP-1651: Sqoop2: Cleanup: Naming and ordering of methods in Repo relates classes
SQOOP-1681: DOC: Create an API doc for the Sqoop Execution engine ( since its extensible)
SQOOP-1702: Fix typo for SQOOP-1526
SQOOP-1703: Sqoop2: rename Type to Column Type + size to length + minor doc fixes
SQOOP-1706: Add IDF API doc/wiki for the IDF interface and Schema -> ColumnTypes
SQOOP-1707: SQOOP2: Address the validate method in Column class
SQOOP-1708: Rename Unsupported Column type to Unknown and add java doc
SQOOP-1709: Column Type enhancements for complex types
SQOOP-1715: Make name for column required ( fix the corr tests)
SQOOP-1716: Add Options as a field in the Enum object ( so it can be used for validation)
SQOOP-1717: Sqoop2: Remove Data class from docs
SQOOP-1733: Port SQOOP-1728 to sqoop2 branch
SQOOP-1734: Port SQOOP-1725 to sqoop2 branch
SQOOP-1741: Port SQOOP-1736 to sqoop2 branch
SQOOP-1749: Support List Type in CSV IDF
SQOOP-1750: Support Map Type in CSV IDF
SQOOP-1751: Sqoop2: Rearrange LinkConfig and ToJobConfig of Kite Connector
SQOOP-1752: Kite connector cannot submit CSV import job
SQOOP-1754: Sqoop2: Remove JarUtil class
SQOOP-1755: Sqoop2: Security guide
SQOOP-1761: Sqoop2: Unit tests for different Column sub classes Array/Set and Map types
SQOOP-1765: Sqoop2: Time/Timestamp format support for CSV IDF
SQOOP-1769: Fix Enum to no inherit from list
SQOOP-1771: Investigation CSV IDF FORMAT of the Array/NestedArray/ Set/ Map in Postgres and HIVE.
SQOOP-1776: Sqoop2: Delegation Token support for Authentication
SQOOP-1791: Sqoop2: Add simple authentication parameters to documentation
SQOOP-1796: Support Incremental Semantics in command line tool for Job
SQOOP-1797: Support Incremental semantics in the REST apis
SQOOP-1798: Support a API/ command line utility to retrieve the last value or any JOB_OUTPUT
SQOOP-1799: Connector API : Ability for connector to indicate if its FROM and TO support incremental reading/ writing
SQOOP-1801: Delta Fetch Merge Config API : Validations for the Delta Fetch/Merge Configs
SQOOP-1804: Add editable and override attribute to inputs
SQOOP-1811: Sqoop2: IDF API changes
SQOOP-1813: Sqoop2: Add SqoopIDFUtils class and unit tests
SQOOP-1815: Sqoop2: Date and DateTime is not encoded in Single Quotes
SQOOP-1817: Sqoop2: Update CSVIntermediate BIT data type
SQOOP-1819: Support Enum in CSVIDF ( + add unit tests)
SQOOP-1825: Sqoop2: Handle NULLs for all types in CSV Intermediate Data Format
SQOOP-1829: Sqoop2: Define IDF object model
SQOOP-1834: Sqoop2: RBAC pluggable framework
SQOOP-1842: Sqoop2: Fix timestamp type casting
SQOOP-1845: Sqoop2: Make DateTime Column type support datetime with and without timezone
SQOOP-1846: Sqoop2: DateTime support in CSV IDF and iso8601
SQOOP-1849: Using JODA for datetime means we only have 3 digit millisecond representation for fraction
SQOOP-1870: Sqoop2: Merge SPI and connector-sdk packages
SQOOP-1876: Update the submission JSON in the doc (restAPI.rst)
SQOOP-1879: Sqoop2: Submission Engine does not set all details on SubmissionRecord in Local mode
SQOOP-1881: Randomize ports for Kafka integration tests
SQOOP-1897: Sqoop2: Submission Engine API change for better performance
SQOOP-1898: Change the API of getJars to use SET than LIST to avoid duplicate jars
SQOOP-1899: Minor change to CSVIDF to use Set instead of Lists
SQOOP-1900: Fix the SqoopWritable read/ write method to delegate the toIDF methods
SQOOP-1901: Sqoop2: Support DRY code in IDF impementations and add JSONIDF
SQOOP-1902: Sqoop2: Avro IDF class and unit tests
SQOOP-1903: Wrap up the design doc once implementation is done
SQOOP-1908: Sqoop2: Document external connector support
SQOOP-1909: Sqoop2: Connect security guide to main docs
SQOOP-1913: Sqoop2: Add jackson as a dependency
SQOOP-1915: sqoop2: parseCSV in CSVIDF doesn't correctly parse strings with commas
SQOOP-1925: Sqoop2: Clean up duplicate constants in DerbySchemaConstants, use CommonRepoConstants
SQOOP-1930: Sqoop2: Enforce a non empty schema name and column names
SQOOP-1931: Tech debt : Alternative to isTest flag in the SqoopOutputFormatExecutor
SQOOP-1935: Sqoop2: Fix TestSqoopWritable test and make getString and setString package private
SQOOP-1936: Sqoop2: Sort by comparing IDF data in shuffle phase
SQOOP-1937: Why need SqoopWritable, ? why not just Text?
SQOOP-1938: DOC:update the sqoop MR engine implementation details
SQOOP-1940: Add hashcode and equals methods to SqoopWritable
SQOOP-1950: Sqoop2: Use TestNG
SQOOP-1956: Sqoop2: Cleanup IDF implementations
SQOOP-1960: Sqoop2: Share auth token across ResourceRequest in SqoopClient
SQOOP-1962: Sqoop2: Start sqoop2 minicluster on random port
SQOOP-1974: Sqoop2: parseCSVString in SQOOPIDFUtils add more tests
SQOOP-1980: Sqoop2: Rule based rest API protection
SQOOP-1981: Sqoop2: Default implementation of RBAC in Sqoop
SQOOP-1982: Sqoop2: Provide username globally via AuthenticationProvider
SQOOP-1984: Sqoop2: Add error summary and details to PostgreSQL repository
SQOOP-1985: Sqoop2: On job delete cascade to submission
SQOOP-1988: Sqoop2: isNull handling should be moved to CSVIntermediateDataFormat
SQOOP-1991: Sqoop2: Define slow category in integration tests
SQOOP-1994: Sqoop2: Use common repository constants
SQOOP-2002: Sqoop2: Refactor existing security component
SQOOP-2004: Sqoop2: Make annotations run always
SQOOP-2012: DOC:Explain the connector project structure and relevant modules in sqoop to depend on
SQOOP-2013: What is the story for External Connectors to write unit and integration tests?
SQOOP-2015: Sqoop2: Create HBase code path in Kite connector
SQOOP-2026: Sqoop2: Make getUserName function in RequestContext support Kerberos
SQOOP-2028: add license header to TestConnectorManagerUtils
SQOOP-2064: Sqoop2: Update POM version on new 1.99.5 branch
SQOOP-2072: Sqoop2: Add editable and override attributes for postgres
SQOOP-2074: Sqoop2: Add input attributes documentation
SQOOP-2084: Sqoop2: Fix Indentation in input attributes documentation
SQOOP-2085: Sqoop2: Null handling in AuthorizationEngine
SQOOP-2096: Sqoop2: Fix typos in comments in SecurityConstants
BUG
SQOOP-1452: Sqoop2: Upgrade to Hadoop 2.5.1
SQOOP-1523: Sqoop2: Support for PostgreSQL database for hosting the sqoop entity repository
SQOOP-1572: Sqoop2: Duplicate Column Name in Multiple Tables Import
SQOOP-1604: Base/ Marker class for Config and Configuration
SQOOP-1636: Sqoop2: Add unique constraint to the SQI_INPUT name per type per config
SQOOP-1640: Sqoop2: Pull DatabaseProvider into sqoop-common-test module
SQOOP-1672: Sqoop2 import inserting null strings('null') for empty values in Oracle
SQOOP-1680: DOC: Create an Public API doc for the Sqoop repository.
SQOOP-1695: Sqoop2: Removed hardcoded version in pom.xml of connector-sdk
SQOOP-1705: SQOOP2: Support AVRO as a an implementation for the IDF interface
SQOOP-1713: Sqoop2: Remove SQOOP-1348.patch file
SQOOP-1730: Sqoop2: httpcore and httpclient version has not upgrade after hadoop upgraded
SQOOP-1740: Sqoop2:The sqoop.properties should say driver.autoupgrade
SQOOP-1747: Sqoop2: Provide more information for Kerberos initialization exception
SQOOP-1748: Sqoop2: Client throws NPE when misconfigured for Kerberos
SQOOP-1773: Sqoop2: ClassAvailable validator null handling
SQOOP-1774: Sqoop2: Allow anonymous authentication
SQOOP-1775: Sqoop2: Migrate to URL parameter user.name instead of using header
SQOOP-1777: Sqoop2: Add integration tests to Pre-Commit build
SQOOP-1778: Sqoop2: Increase default memory allocations for tests
SQOOP-1782: Sqoop2: Get submissions per job
SQOOP-1783: Sqoop2: Create derby integration upgrade tests
SQOOP-1784: Sqoop2: Sending POST request to link or job end-point might end up with NPE
SQOOP-1785: Sqoop2: Send driver validation results back to client when creating job
SQOOP-1786: Sqoop2: Stop using JSONValue.parse method call
SQOOP-1787: Sqoop2: Fix dual guava dependency warnings in maven
SQOOP-1788: Sqoop2: Add missing version for build-helper-maven-plugin
SQOOP-1789: Sqoop2: Add validation for partition column when using query based import
SQOOP-1792: Sqoop2: Default to simple authentication type
SQOOP-1806: Sqoop2: Remove the duplicate connector-config.properties file in Kite connector
SQOOP-1812: Sqoop2: Repository upgrade issue will prevent server startup
SQOOP-1816: Sqoop2: Add logging to the test DerbyProvider implementation
SQOOP-1818: Sqoop2: Allow transfer between 2 schema less data sources
SQOOP-1821: Sqoop2: External connector loading
SQOOP-1823: Sqoop2: Fix Derby schema javadocs
SQOOP-1827: Sqoop2: Add pattern SQOOP-XYZ-vX.patch to our pre-commit hook
SQOOP-1830: GenericJdBcExtractor does not create java date objects when extracting
SQOOP-1831: Sqoop2: Log filenames of detected Mapreduce files in HdfsUtils
SQOOP-1832: Sqoop2: Enforce that integration test working directory is always in target/ directory
SQOOP-1833: Sqoop2: Add elapsed time information to pre-commit hook
SQOOP-1836: Sqoop2: Convert the List in Schema Columns to Array for better lookup time
SQOOP-1837: Sqoop2: Verify that executing job has been successful in ConnectorTestCase
SQOOP-1838: Sqoop2: While running Integration tests in IDE ( we get Derby Start Error)
SQOOP-1839: Sqoop2: Mapreduce submission record should propogate failure info as well
SQOOP-1840: Sqoop2: Guard test DerbyProvider from infinite waits in Derby implementation
SQOOP-1843: Sqoop2: Fix anonymous configuration bug
SQOOP-1844: Sqoop2: Start Derby server in DerbyProvider on random port
SQOOP-1847: Sqoop2: Move rat configuration from special execution to the main execution
SQOOP-1850: Sqoop2: DerbyProvider retry ping
SQOOP-1857: Sqoop2: Add rat check to pre-commit hook
SQOOP-1858: Sqoop2: Print out warnings from git apply on output of precommit hook
SQOOP-1861: Sqoop2: ResourceRequest clobbers uncaught server exceptions
SQOOP-1862: Sqoop2: JDBC Connector To side needs to handle converting JODA objects to sql date
SQOOP-1871: Sqoop2: Upgrade to Hadoop 2.6.0
SQOOP-1872: IDF API should expose a method to add dependent jars
SQOOP-1874: Sqoop2: ConfigurableError should be in the configurable package
SQOOP-1875: Sqoop2: Sync MSubmission with repository and add tests for the max limit handling for submission fields
SQOOP-1882: JobManager currently ignores the TO connector IDF and assumed all IDF use String for the generic T
SQOOP-1885: Sqoop2: KafkaConnector pom.xml needs license header
SQOOP-1893: add license header to TestSqoopIDFUtils.
SQOOP-1894: Sqoop2: Print out files that failed license check in pre-commit hook
SQOOP-1895: Sqoop2: Add links to output reports to pre-commit hook
SQOOP-1896: Upgrade the Derby repo version to 5 for 1.99.5 release
SQOOP-1910: Sqoop2: Add test categories
SQOOP-1911: Sqoop2: Derby1_99_4UpgradeTest is constantly failing on pre-commit hook
SQOOP-1917: Sqoop2: "show connector --all" fails
SQOOP-1927: Sqoop2: Move CSVURIValidator into validators package
SQOOP-1929: Sqoop2: Track number of records written in Loader
SQOOP-1934: SqoopWritable and MR engine documentation
SQOOP-1943: Sqoop2: PreCommit should continue if license check fails
SQOOP-1945: Sqoop2: Add table escaping to JDBC Repository
SQOOP-1963: Generated files such as .project and .classpath files are included in source distribution
SQOOP-1971: Sqoop2: strURL may not be initialized when calling addDelegationToken
SQOOP-1972: Sqoop2: PreCommit hook have incorrect link for test reports
SQOOP-1973: Sqoop2: Remove maven module spi completely after merging it with connector-sdk
SQOOP-1977: Sqoop2: Pre-commit hook does not work for Python 2.6 environment
SQOOP-1995: Sqoop2: Allow nulls only if the column for that field has IsNullable to be true
SQOOP-1996: Sqoop2: Change getNullable to isNullable
SQOOP-2018: GenericJDBCConnector SqlTypes INTEGER should be set 4 bytes
SQOOP-2021: Sqoop2: Generic JDBC Connector extractor uses wrong nulls
SQOOP-2022: Sqoop2: SqoopIDFUtils uses bit size instead of byteSize to check for INT/LONG/FLOAT/Double
SQOOP-2023: GenericJDBCConnector Handling unsigned Ints is broken
SQOOP-2027: Sqoop2: SqoopIDFUtils handling of decimal need to be fixed
SQOOP-2029: Sqoop2: Get jobs by connector
SQOOP-2030: Should we allow creating a job with both from and to to id been the same
SQOOP-2032: Command line exception message says class not found
SQOOP-2034: Sqoop2: Clean up docstring and names in repository API
SQOOP-2041: Sqoop2: Escape identifier names
SQOOP-2043: Sqoop2: Server errors don't look informative enough in the client
SQOOP-2054: Sqoop2: Client cannot understand server exceptions
SQOOP-2077: Sqoop2: Enclose missing/extra lines in output of HdfsAsserts.assertMapreduceOutput
SQOOP-2078: Sqoop2: Add ParametrizedUtils.toArrayOfArrays utils method
SQOOP-2081: Sqoop2: Provide ability to dump content of testing table to log
SQOOP-2086: Sqoop2: Use named constraints
SQOOP-2087: Sqoop2: Named constraints should be quoted
SQOOP-2088: Sqoop2: Rename constants for named constraints to match convention
SQOOP-2099: Sqoop2: Postgresql's detectRepositoryVersion uses wrong casing
SQOOP-2100: Sqoop2: Connectors documentation is malformed
SQOOP-2109: Sqoop2: Shell module is including test dependencies when building binary artifact
SQOOP-2116: Sqoop2: RequestContext should use user short name
IMPROVEMENT
SQOOP-1678: Sqoop2: [HDFS Connector] Configurable null values
SQOOP-1727: Sqoop2: Log server initialization errors to log4j
SQOOP-1767: Sqoop2: commands in 5 minute demo docs are out-of-sync
SQOOP-1869: Sqoop2: Expand schema matching to support two schemaless connectors
SQOOP-1944: Sqoop2: No (or wrong) log message when Derby server startup/shutdown unexpectedly
SQOOP-1966: Turn off maven's source distribution profile by default
SQOOP-1968: Optimize schema operation in getMatchingData of NameMatcher
SQOOP-1969: Support getColumnSize method for Schema class
SQOOP-2006: bad smell: NPE check in SQOOP-1995 is not obvious
NEW FEATURE
SQOOP-1763: Sqoop2: Role Based Access Control
SQOOP-1926: Sqoop2: Add annotations for visibilty and compatibility guarantees
TASK
SQOOP-1350: Sqoop2: Support all supported data types in the CSV Intermediate Data Format implementation
SQOOP-1525: Kerberos support on Sqoop 2
SQOOP-2033: Technical Guide of the Sqoop Server
Release 1.99.4
NEW FEATURE
SQOOP-777 Sqoop2: Implement intermediate data format representation policy
SQOOP-1232 Sqoop2: Provide tooling infrastructure for Sqoop2
SQOOP-1367 Sqoop2: From/To
IMPROVEMENT
SQOOP-773 Sqoop2: Batch execution support for client commands
SQOOP-1144 Sqoop2: Add fixVersion to PreCommit branch detection
SQOOP-1189 Sqoop2: Ensure that clone methods will correctly copy over all values from all parents
SQOOP-1196 Sqoop2: Add support for arbitrary compression codecs
SQOOP-1211 Sqoop2: Derby repo: Sync maximal length of versions
SQOOP-1225 Sqoop 2 documentation for connector development
SQOOP-1290 Sqoop2: Kill Tomcat in case that Sqoop Server fails to load
SQOOP-1509 Sqoop2: Sqoop2 Rest API refactoring
SQOOP-1547 Sqoop2: Connector API stabilization
SQOOP-1557 Sqoop2: SQ_CONFIGURABLE ( for entities who own configs)
SQOOP-1566 Sqoop2: Fix the upgrade logic for SQOOP-1498
SQOOP-1585 Sqoop2: Prefix mapreduce classes with MR ( no functionality change)
SQOOP-1586 Sqoop2: Rename leftovers from the SQOOP2 merge of 1367
SQOOP-1597 Sqoop2: Refactor DerbySchemaQuery into one for create/ update/ and then CRUD operarations
SQOOP-1620 Sqoop2: FileSystem should be configurable in HDFS connector
BUG
SQOOP-1208 Sqoop2: job ID column is too small
SQOOP-1218 Sqoop2: Upgrade Hadoop 2 to GA release 2.2.0
SQOOP-1233 Sqoop2: Remove unnecessary explicit call to MessageFormat
SQOOP-1247 Sqoop2: Distribution module is missing dependency on Tomcat module
SQOOP-1258 Sqoop2: FrameworkMetadataUpgrade can't handle new inputs
SQOOP-1263 Sqoop2: Using CATALINA_HOME as CATALINA_BASE
SQOOP-1264 Sqoop2: Remove unnecessary cd command calls from sqoop.sh
SQOOP-1285 Sqoop2: ToolRunner is not exiting with proper exit value on error
SQOOP-1288 Add mengwei to commiter list in our root pom.xml file
SQOOP-1291 Sqoop2: Binary package do not have top level commands executable
SQOOP-1296 Sqoop2: Revert SQOOP-1233
SQOOP-1308 Sqoop2: Database export with the Generic-JDBC-Connector loses rows
SQOOP-1320 Sqoop2: Log entire SQLExceptions chain in GenericJdbcExecutor
SQOOP-1331 Sqoop2: Add method "findConnectors" to repository API
SQOOP-1338 Sqoop2: Exclude intellij idea project files from git
SQOOP-1339 Synchronize .gitignore files
SQOOP-1354 Sqoop2: wrong maven-site-plugin version
SQOOP-1383 Sqoop2: MSSQL Export not working
SQOOP-1424 Sqoop2: Simplify SqoopCommand in shell package
SQOOP-1425 Sqoop2: Improve ClassUtils to enable use of primitive types and subclasses
SQOOP-1426 Sqoop2: ThrowableBean should correct reconstruct SqoopException
SQOOP-1436 Sqoop2: Support custom form name via the Form Annotation
SQOOP-1537 Sqoop2: File TestJobManager is missing proper license header
SQOOP-1578 Sqoop2: getJars should take from and to configurations
SQOOP-1581 Sqoop2: LoaderContext has the wrong Schema
SQOOP-1594 Sqoop2: DestroyerContext has the wrong Schema
SQOOP-1605 Sqoop2: Misc From/to cleanups
SQOOP-1619 Sqoop2: Enforce Unique constraint for name on configurable table
SQOOP-1623 Sqoop2: Derive the correct Client version for sqoop
SQOOP-1625 Repository upgrade issue
SQOOP-1629 Sqoop2: Add unique constraint on the Config table for name and type
SQOOP-1636 Sqoop2: Add unique constraint to the SQI_INPUT name per type per config
SQOOP-1639 Sqoop2: CLOB data type wrong type
SQOOP-1641 DOC: Sqoop 5 minute demo.rst update to reflect latest in code
SQOOP-1655 SQOOP2 DOC: Document getSchema() and its use in the connector dev guide
SQOOP-1660 DOC: Connector SDK docs + validation to be updated
SQOOP-1661 Sqoop2: Intermediate data format text null handling
SQOOP-1662 Sqoop2: allowNullValueInPartitionColumn should be partitionColumnNull
SQOOP-1664 Sqoop2: Remove the unused SqoopFileOutputFormat class
SQOOP-1665 Sqoop2: Misc Cleanup / rename lingering connection to link
SQOOP-1668 Sqoop2: Update existing job meets NPE ( regression from SQOOP 1510)
SQOOP-1669 Sqoop2: JDBC connector does not understand iso8610 representation using " " as separator
SQOOP-1671 Fix incorrect merge from SQOOP-1645
SQOOP-1673 DOC: Update Upgrade.rst to reflect the latest in the code
SQOOP-1674 DOC: index.rst updates for Sqoop
SQOOP-1675 Sqoop2: Update Upgrade.rst to to address Rb comments
SQOOP-1676 Sqoop2: clientAPI.rst changes to reflect latest code
SQOOP-1677 DOC: CommandLineClient.rst update to reflect latest in code
SQOOP-1679 Sqoop2: Fix inconsistent naming in shell for entity driver in sqoop2
SQOOP-1694 Sqoop2: Fix broken Hadoop100 build
SQOOP-1696 Sqoop2: TestMapReduce.testOutputFormat is failing on Hadoop 1 profile
SQOOP-1697 Sqoop2: HDFS Connector TestLoader.verifyOutput have invalid default value for Hadoop 1
SQOOP-1718 Sqoop2: CommandLineClient doc error
SQOOP-1721 Sqoop2: common jars missing for MR job
SQOOP-1723 Sqoop2: findJobsByConnector fails
SQOOP-1724 Sqoop2: Support old config types for backward compatibility in connector data upgrade
SQOOP-1725 Sqoop2: Connector upgrade in 1.99.4 fails with delete configs
SQOOP-1728 Sqoop2: Force HDFS connector as a configurable fixes
SQOOP-1729 Sqoop2: The existing genericJDBC connector code upgrade logic is incomplete.
SQOOP-1731 Sqoop2: Generic JDBC upgrade from 1.99.3 to 1.99.4
SQOOP-1736 Sqoop2: Driver missing during upgrade
SQOOP-1740 Sqoop2:The sqoop.properties should say driver.autoupgrade
SQOOP-1742 Sqoop2: Add type to connectors during upgrade
TASK
SQOOP-1219 Sqoop2: Bump up Tomcat version to 6.0.37
SQOOP-1220 Sqoop2: Improve default catalina.properties file to work out of the box on both Hadoop 1 and 2
SQOOP-1234 Sqoop2: Clean up comments in MSubmission class
SQOOP-1235 Sqoop2: Repository should be immutable by default
SQOOP-1236 Sqoop2: Classpath generated by Submission engine should contain only unique elements
SQOOP-1266 Sqoop2: Introduce top level commands
SUB-TASK
SQOOP-927 Sqoop2: Integration: Mapreduce specific tests should be running on MiniCluster
SQOOP-1254 Sqoop2: Tool: Upgrade tool
SQOOP-1255 Sqoop2: Tool: Dump repository content to a repository independent format
SQOOP-1262 Sqoop2: Tool: Create user guide entry for the Tooling infrastructure
SQOOP-1376 Sqoop2: From/To: Refactor connector interface
SQOOP-1377 Sqoop2: From/To: Improve client/server communication
SQOOP-1379 Sqoop2: From/To: Disable tests
SQOOP-1380 Sqoop2: From/To: Refactor models
SQOOP-1381 Sqoop2: From/To: Refactor submission/execution engine
SQOOP-1444 Sqoop2: Validations: Remove all references to the original Validation infastructure
SQOOP-1469 Sqoop2: Validations: Rename constants in Status class
SQOOP-1499 Sqoop2: Refactor MJob to hold fromConfig, ToConfig and DriverConfig ( mandatory to create a job)
SQOOP-1500 Sqoop2: Rename Forms to Config, Create FromConfig and ToConfig
SQOOP-1501 Sqoop2: Refactor MConnector
SQOOP-1503 Sqoop2: Refactor MFramework (rename to MDriver) and add MDriverConfig
SQOOP-1504 Sqoop2: Refactor MConnection (rename to MLink)
SQOOP-1505 Sqoop2: Rename the repository schema objects to adhere to the model objects
SQOOP-1510 Sqoop2: Refactor JobRequestHandler for submit/abort job and SubmissionHandler for get operation only
SQOOP-1512 Refactor ConnectionRequestHandler and add ConfigurableRequestHandler
SQOOP-1513 Sqoop2: Refactor LinkRequestHandler
SQOOP-1514 Refactor FrameworkRequestHandler to DriverRequestHandler
SQOOP-1515 Sqoop2: VersionRequestHandler - fixes
SQOOP-1517 Test CommandLineClient to accomodate the JSON API refactoring
SQOOP-1518 Sqoop2: Update the Rest API documentation
SQOOP-1542 Sqoop2: From/To: MConnector should handle null forms
SQOOP-1550 Sqoop2: Remove the indirection in MDriverConfig, let it extend the MConfig instead of has-a MConfig
SQOOP-1551 Sqoop2: Repository Upgrader api - Extensibility
SQOOP-1554 Sqoop2: Add NullConfigurationClass/ EmptyConfigurationClass to support use cases that do not have a particular type of config
SQOOP-1575 Sqoop2: Validations: Migrate HDFS connector to new validators
SQOOP-1576 Sqoop2: Validations: Migrate Repository upgrade to the new validator infrastructure
SQOOP-1628 Sqoop2: Ensure Connector/Driver/Version only support GET method
SQOOP-1637 Sqoop2: Validations: Update documentation with the new Validator infrastructure
SQOOP-1645 Sqoop2: Update label text and (optional) required info for the generic jdbc connector config properties
SQOOP-1653 Sqoop2: Link and connector handler minor fixes
SQOOP-1686 Sqoop2: Fix the JSON names for config and config values
SQOOP-1687 Sqoop2: Single resource in JSON should not be a list
SQOOP-1688 Sqoop2: Fix the Validation Response JSON for configs
Release 1.99.3
SUB-TASK
SQOOP-659: Design metadata upgrade procedure
SQOOP-924: Sqoop2: Devguide: Describe wire protocol
SQOOP-945: Sqoop2: Integration: Auxiliary methods that will support export functionality
SQOOP-994: Sqoop2: Upgrade: Add calling validation to the upgrade method
SQOOP-996: Sqoop2: Upgrade: Create upgrade tests
SQOOP-997: Sqoop2: Upgrade: Provide ability to disable the automatic upgrade
SQOOP-998: Sqoop2: Upgrade: Add framework upgrader
SQOOP-1009: Sqoop2: Integration: Create Teradata provider
SQOOP-1014: Sqoop2: Splitter: Add Bit/Boolean based implementation
SQOOP-1015: Sqoop2: Splitter: Add Datetime based implementation
SQOOP-1016: Sqoop2: Splitter: Add text char implementation
SQOOP-1022: Metadata upgrade fails with NPE getConnector method
SQOOP-1023: STMT_DELETE_FORMS_FOR_CONNECTOR query in DerbySchemaSquery looks for incorrect column
SQOOP-1024: GenericJdbcConnectorUpgrader must ignore inputs which may not exist in old version