Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

StackOverflowError when reading from BigQuery with SDK 1.5.0 #61

Closed
nevillelyh opened this issue Mar 24, 2016 · 2 comments
Closed

StackOverflowError when reading from BigQuery with SDK 1.5.0 #61

nevillelyh opened this issue Mar 24, 2016 · 2 comments
Assignees
@nevillelyh
Labels
bug Something isn't working

Comments

@nevillelyh
Copy link
Contributor

nevillelyh commented Mar 24, 2016 •
edited

Could be related to the change to import/export BQ as Avro introduced in 1.5.0.

java.lang.StackOverflowError
at org.apache.avro.io.parsing.Symbol$Sequence.flattenedSize(Symbol.java:323)
at org.apache.avro.io.parsing.Symbol.flattenedSize(Symbol.java:216)
at org.apache.avro.io.parsing.Symbol$Sequence.flattenedSize(Symbol.java:323)
at org.apache.avro.io.parsing.Symbol.flattenedSize(Symbol.java:216)
at org.apache.avro.io.parsing.Symbol$Sequence.flattenedSize(Symbol.java:323)
at org.apache.avro.io.parsing.Symbol.flattenedSize(Symbol.java:216)

SDK 1.5.0 exports BQ as Avro instead of JSON:
GoogleCloudPlatform/DataflowJavaSDK@4dce3c2

Related:
http://stackoverflow.com/questions/24130615/circular-references-not-handled-in-avro
https://code.google.com/p/google-bigquery/issues/detail?id=381
GoogleCloudDataproc/hadoop-connectors#16
GoogleCloudPlatform/DataflowJavaSDK#152
@nevillelyh nevillelyh added the bug Something isn't working label Mar 24, 2016
@nevillelyh nevillelyh reopened this Mar 24, 2016
@nevillelyh
Copy link
Contributor Author

So Dataflow Java SDK actually uses Avro library to read the files exported from BQ so there might be a way we can work around this in the SDK.

@nevillelyh nevillelyh mentioned this issue Mar 24, 2016
Closed
@nevillelyh
Copy link
Contributor Author

Just verified that this is fixed.

@nevillelyh nevillelyh self-assigned this Oct 12, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nevillelyh nevillelyh

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@nevillelyh