Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-8118] [SQL] Mutes noisy Parquet log output reappeared after upgrading Parquet to 1.7.0 #6670

Closed
wants to merge 2 commits into from
Closed

[SPARK-8118] [SQL] Mutes noisy Parquet log output reappeared after upgrading Parquet to 1.7.0 #6670

wants to merge 2 commits into from

Conversation

liancheng
Copy link
Contributor

No description provided.

@SparkQA
Copy link

SparkQA commented Jun 5, 2015

Test build #34269 has finished for PR 6670 at commit 40ba1fd.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jun 5, 2015

Test build #34276 has finished for PR 6670 at commit 385603c.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@JoshRosen
Copy link
Contributor

Looks like this covered most of the logs, but there's still one message that's appearing:

04:34:49.075 WARN org.apache.parquet.hadoop.ParquetRecordReader: Can not initialize counter due to context is not a instance of TaskInputOutputContext, but is org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl

Do we need to pass a different type of context or something to fix this?

@kostya-sh
Copy link

@JoshRosen, this warning is logged mistakenly by Parquet:
https://issues.apache.org/jira/browse/PARQUET-220

@liancheng
Copy link
Contributor Author

@JoshRosen Yeah, this log line had been there even before bumping Parquet version.

@kostya-sh Since this line is the only log message ParquetRecordReader issues, I'm going to turn off ParquetRecordReader log output here, until Parquet fixes this issue (probably in 1.8.0).

@SparkQA
Copy link

SparkQA commented Jun 6, 2015

Test build #34354 has finished for PR 6670 at commit b6e85a6.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@liancheng
Copy link
Contributor Author

Merging to master. Thanks all for reviewing this!

liancheng added a commit that referenced this pull request Jun 7, 2015
@liancheng
[SPARK-8118] [SQL] Mutes noisy Parquet log output reappeared after up…
8c321d6 
…grading Parquet to 1.7.0

Author: Cheng Lian <lian@databricks.com>

Closes #6670 from liancheng/spark-8118 and squashes the following commits:

b6e85a6 [Cheng Lian] Suppresses unnecesary ParquetRecordReader log message (PARQUET-220)
385603c [Cheng Lian] Mutes noisy Parquet log output reappeared after upgrading Parquet to 1.7.0
@rxin
Copy link
Contributor

rxin commented Jun 7, 2015

Somehow github is no longer closing pull requests.

@liancheng
Copy link
Contributor Author

Closing this manually.

@liancheng liancheng closed this Jun 8, 2015
@liancheng liancheng deleted the spark-8118 branch June 8, 2015 06:23
nemccarthy pushed a commit to nemccarthy/spark that referenced this pull request Jun 19, 2015
@liancheng @nemccarthy
[SPARK-8118] [SQL] Mutes noisy Parquet log output reappeared after up…
c4a5050 
…grading Parquet to 1.7.0

Author: Cheng Lian <lian@databricks.com>

Closes apache#6670 from liancheng/spark-8118 and squashes the following commits:

b6e85a6 [Cheng Lian] Suppresses unnecesary ParquetRecordReader log message (PARQUET-220)
385603c [Cheng Lian] Mutes noisy Parquet log output reappeared after upgrading Parquet to 1.7.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
5 participants
@liancheng @SparkQA @JoshRosen @kostya-sh @rxin