Skip to content
This repository was archived by the owner on May 12, 2021. It is now read-only.

TAJO-1465: Add ORCFileAppender to write into ORCFile table#652

Closed
eminency wants to merge 156 commits intoapache:masterfrom
eminency:TAJO-1465
Closed

TAJO-1465: Add ORCFileAppender to write into ORCFile table#652
eminency wants to merge 156 commits intoapache:masterfrom
eminency:TAJO-1465

Conversation

@eminency
Copy link
Copy Markdown
Contributor

@eminency eminency commented Jul 24, 2015

Hi all.

This is for writer code for ORC files.

Many files are added, but most of them are from Hive. See following notes :

  1. Core working is in ORCAppender.java and WriterImpl.java
  2. Files in org.apache.tajo.storage.orc.objectinspector are trivial. Those are for checking data type.
  3. Files in org.apache.tajo.storage.thirdparty are from Hive. WriterImpl.java is also from Hive, but it is modified to support Tajo types and Datum.

Please review and advise.
Thanks.

Review on Reviewable

jihoonson and others added 30 commits March 21, 2015 18:38
@jihoonson
TAJO-1439: Some method name is written wrongly. (Contributed by Jongy…
aacc3b7 
…oung Park. Committed by jihoon)
@navis @jinossy
TAJO-1396 Unexpected IllegalMonitorStateException can be thrown in Qu…
e563db2 
…eryInProgress

Closes apache#416

Signed-off-by: Jinho Kim <jhkim@apache.org>
@dongjoon-hyun @jinossy
TAJO-1434: Fix supporting version of Hadoop.
e854f83 
Closes apache#443

Signed-off-by: Jinho Kim <jhkim@apache.org>
@eminency @blrunner
TAJO-1147: Simple query doesn't work in Web UI
a535025 
Signed-off-by: JaeHwa Jung <blrunner@apache.org>
@eminency @jinossy
TAJO-1440: Some tests fail in parallel test environment in TestKillQu…
becf85b 
…ery.

Closes apache#472

Signed-off-by: Jinho Kim <jhkim@apache.org>
@dongjoon-hyun @jihoonson
TAJO-1437: Resolve findbug warnings on Tajo JDBC Module.
8b00e41 
Closes apache#447

Signed-off-by: Jihoon Son <jihoonson@apache.org>
@eminency @blrunner
TAJO-1501: Too many log message of HashShuffleAppenderManager.
c3b7d13 
Signed-off-by: JaeHwa Jung <blrunner@apache.org>
@charsyam @blrunner
TAJO-1360: VALUES_ field in OPTIONS table of catalog store should be …
b729a49 
…longer.

Signed-off-by: JaeHwa Jung <blrunner@apache.org>
@dongjoon-hyun @jihoonson
TAJO-1492: Replace CSV examples into TEXT examples in docs.
8173bc1 
Signed-off-by: Jihoon Son <jihoonson@apache.org>
@hys9958 @jihoonson
TAJO-1400: Add TajoStatement::setMaxRows method support.
cb69659 
Signed-off-by: Jihoon Son <jihoonson@apache.org>
@jinossy
TAJO-1529: Implement json_extract_path_text(string, string) function.
7f7b132
@navis @jinossy
TAJO-1538: TajoWorkerResourceManager.allocatedResourceMap is increasi…
e75b928 
…ng forever.

Signed-off-by: Jinho Kim <jhkim@apache.org>
@jinossy
TAJO-1564: TestFetcher fails occasionally. (jinho)
75e985e
@dongjoon-hyun @jinossy
TAJO-1567: Update old license in some pom.xml files.
db3bbc9 
Signed-off-by: Jinho Kim <jhkim@apache.org>
@jinossy
TAJO-1560: HashShuffle report should be ignored when a succeed tasks …
fbe24e8 
…are not included. (jinho)
@navis @jinossy
TAJO-1522: NPE making stage history before task scheduler is initiali…
4a02456 
…zed.

Signed-off-by: Jinho Kim <jhkim@apache.org>
@jinossy
TAJO-1568: Apply UnpooledByteBufAllocator when a tajo.test.enabled is…
5e1fa93 
… set to enable.
@jinossy
TAJO-1571: Merge TAJO-1497 and TAJO-1569 to 0.10.1. (jinho)
47008c5 
Closes apache#544
@navis @jihoonson
TAJO-1481: Numeric conversion of Inet4 type should be considered as u…
3722599 
…nsigned.

Signed-off-by: Jihoon Son <jihoonson@apache.org>
@charsyam @jihoonson
TAJO-1419: Tsql session command doesn't work.
8d79884 
Signed-off-by: Jihoon Son <jihoonson@apache.org>
@eminency @jihoonson
TAJO-1575: HBASE_HOME guidance is duplicated in tajo-env.sh.
c33b863 
Signed-off-by: Jihoon Son <jihoonson@apache.org>
@dongjoon-hyun @jihoonson
TAJO-1559: Fix data model description (tinyint, smallint).
b488a5e 
Signed-off-by: Jihoon Son <jihoonson@apache.org>
@eminency @blrunner
TAJO-1580: Error line number is incorrect.
9ca7688 
Signed-off-by: JaeHwa Jung <blrunner@apache.org>
@jinossy
TAJO-1581: Does not update last state of query stage in non-hash shuf…
c9b5e11 
…fle. (jinho)
@dongjoon-hyun @jihoonson
TAJO-1574: Fix NPE on natural join.
7ccd834 
Signed-off-by: Jihoon Son <jihoonson@apache.org>
@navis @jinossy
TAJO-1374: Support multi-bytes delimiter for CSV file.
4db8ca0 
Signed-off-by: Jinho Kim <jhkim@apache.org>
@navis @jinossy
TAJO-1381: Support multi-bytes delimiter for Text file.
b0ab7ea 
Signed-off-by: Jinho Kim <jhkim@apache.org>
@jinossy
TAJO-1534: DelimitedTextFile return null instead of a NullDatum. (jinho)
f9a531c
@superpiggy @jihoonson
TAJO-1556: "insert into select" with reordered column list does not w…
9d331f9 
…ork.

Signed-off-by: Jihoon Son <jihoonson@apache.org>
@eminency @jihoonson
TAJO-1576: Sometimes DefaultTajoCliOutputFormatter.parseErrorMessage(…
556498f 
…) eliminates an important kind of information.

Signed-off-by: Jihoon Son <jihoonson@apache.org>
@eminency
Merge branch 'master' into TAJO-1465
379190e 
* master:
  TAJO-1821: Temporary data is not cleared after TestCatalog.
  TAJO-1805: In the 'Execute Query' page of web UI, default database should be set as 'default'. (Contributed by Dongkyu Hwangbo, committed by hyunsik)
  TAJO-1814: Add some positive/negative tests for CREATE TABLE IF NOT EXISTS.
  TAJO-1823: Can't start TajoMaster.
  TAJO-1820: Fix wrong case sensitivity rules of non-reserved keywords.
  TAJO-1610: Cannot find column when the same name is used for table and database.
  TAJO-1641: Add window function documentation.
  TAJO-1818: Separate sql parser into an independent maven module.
hyunsik
hyunsik reviewed Sep 11, 2015
View reviewed changes
Comment thread tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java Outdated
Copy link
Copy Markdown
Member

@hyunsik hyunsik Sep 11, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code will make a directory in a resource directory. It seems to be not good. Could you remove this change?

@hyunsik
Copy link
Copy Markdown
Member

hyunsik commented Sep 11, 2015

I leaved comments.

@eminency
revert empty table handling
194a190
@eminency
Merge branch 'master' into TAJO-1465
b5182d4 
* master:
  TAJO-1831: Add a shutdown hook manager in order to set priorities.
  TAJO-1830: Fix race condition in HdfsServiceTracker.
  TAJO-1817: Improve SQL parser error message.
  TAJO-1739: Add a statement for adding partition to TajoDump.
  TAJO-1727: Avoid to create external table using TableSpace.
  TAJO-1833: Refine LogicalPlanPreprocessor to add new rules easily.
  TAJO-1600: Invalid query planning for distinct group-by.
  TAJO-1825: Remove zero length fragments when file length is zero.
  TAJO-1826: Revert "refine code for Parquet 1.8.1."
  TAJO-1782: Check ON_ERROR_STOP flag in TSQL when error is occured.
  TAJO-1829: Fix DelimitedTextFileAppender NPE in negative tests.
  TAJO-1828: tajo-daemon scripts should kill process after process can not stop gracefully.
  TAJO-1674: Validation of CTAS schema mismatch.
  TAJO-1729: No handling of default case in DDLExecutor. (Contributed by Dongkyu Hwangbo, committed by Jinho)
  TAJO-1819: Cannot find existing tables when pgsql catalog starts up.
@eminency
Error handling code refined
4531315
@eminency
delete TestOrc
9d14828
@eminency
Merge master into TAJO-1465
358924b
@eminency
Test code changed by exception refining
404c46f
@eminency
Error handling added
39f6fc1
@eminency
orc scanner init logic changed
12854be
@eminency
path to local variable
e201b74
@hyunsik
Copy link
Copy Markdown
Member

hyunsik commented Sep 16, 2015

+1

Before commiting the patch, I slightly changed as follows

  • INVALID_CAST to INVALID_VALUE_FOR_CAST
  • Add SQLState for INVALID_VALUE_FOR_CAST to SQLExceptionUtil
  • Add an map entry for INVALID_VALUE_FOR_CAST to ExceptionUtil

asfgit pushed a commit that referenced this pull request Sep 16, 2015
@eminency @hyunsik
TAJO-1465: Add ORCFileAppender to write into ORCFile table.
c3c78fc 
Signed-off-by: Hyunsik Choi <hyunsik@apache.org>

Closes #652
@hyunsik
Copy link
Copy Markdown
Member

hyunsik commented Sep 16, 2015

Thank you for your work. I committed as c3c78fc. But, I missed to add github command in the commit log. Could you close this ticket?

@eminency
Copy link
Copy Markdown
Contributor Author

eminency commented Sep 17, 2015

Thank you very much for your working.
I close this issue at final.

@eminency eminency closed this Sep 17, 2015
@eminency eminency deleted the TAJO-1465 branch September 22, 2015 02:29
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

10 participants

@eminency @hyunsik @jihoonson @navis @dongjoon-hyun @charsyam @hys9958 @jinossy @superpiggy @blrunner