Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "Wrapper the input and oupput stream from HadoopExtendedFileSy… #17435

Merged
merged 1 commit into from Mar 9, 2022

Conversation

varungajjala
Copy link
Contributor

…stem"

This reverts commit 195ce18.

== NO RELEASE NOTE ==

@varungajjala
Copy link
Contributor Author

This resolves some errors we are seeing in our tests such as:

Failure for format PARQUET with properties {}
Stacktrace
java.lang.AssertionError: Failure for format PARQUET with properties {}
at org.testng.Assert.fail(Assert.java:83)
at com.facebook.presto.hive.TestHiveIntegrationSmokeTest.testWithStorageFormat(TestHiveIntegrationSmokeTest.java:5932)
at com.facebook.presto.hive.TestHiveIntegrationSmokeTest.testWithAllStorageFormats(TestHiveIntegrationSmokeTest.java:5919)
at com.facebook.presto.hive.TestHiveIntegrationSmokeTest.testReadNoColumns(TestHiveIntegrationSmokeTest.java:315)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1177)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.AssertionError: Execution of 'actual' query failed: SELECT count(*) FROM test_read_no_columns
at org.testng.Assert.fail(Assert.java:83)
at com.facebook.presto.tests.QueryAssertions.assertQuery(QueryAssertions.java:150)
at com.facebook.presto.tests.QueryAssertions.assertQuery(QueryAssertions.java:106)
at com.facebook.presto.tests.AbstractTestQueryFramework.assertQuery(AbstractTestQueryFramework.java:148)
at com.facebook.presto.hive.TestHiveIntegrationSmokeTest.testReadNoColumns(TestHiveIntegrationSmokeTest.java:324)
at com.facebook.presto.hive.TestHiveIntegrationSmokeTest.testWithStorageFormat(TestHiveIntegrationSmokeTest.java:5929)
... 15 more
Caused by: java.lang.RuntimeException: Byte-buffer read unsupported by input stream
at com.facebook.presto.tests.AbstractTestingPrestoClient.execute(AbstractTestingPrestoClient.java:124)
at com.facebook.presto.tests.DistributedQueryRunner.execute(DistributedQueryRunner.java:593)
at com.facebook.presto.tests.DistributedQueryRunner.execute(DistributedQueryRunner.java:561)
at com.facebook.presto.tests.QueryAssertions.assertQuery(QueryAssertions.java:147)
... 19 more
Caused by: java.lang.UnsupportedOperationException: Byte-buffer read unsupported by input stream

Copy link
Contributor

@aweisberg aweisberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The important part of the stack is here, seems to be truncated in the comment.

Caused by: java.lang.UnsupportedOperationException: Byte-buffer read unsupported by input stream
	at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:146)
	at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:143)
	at org.apache.parquet.hadoop.util.H2SeekableInputStream$H2Reader.read(H2SeekableInputStream.java:81)
	at org.apache.parquet.hadoop.util.H2SeekableInputStream.readFully(H2SeekableInputStream.java:90)
	at org.apache.parquet.hadoop.util.H2SeekableInputStream.readFully(H2SeekableInputStream.java:75)
	at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:542)
	```

Comment on lines 152 to 153
return fs.create(f, permission,
overwrite, bufferSize, replication, blockSize, progress);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

put into the same line

Comment on lines 167 to 168
return fs.create(f, permission,
flags, bufferSize, replication, blockSize, progress, checksumOpt);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

put into the same line

@highker highker self-assigned this Mar 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants