Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KryoException: java.lang.IndexOutOfBoundsException: Index: x, Size: y #1136

Closed
ravwojdyla opened this issue Apr 30, 2018 · 1 comment
Closed

KryoException: java.lang.IndexOutOfBoundsException: Index: x, Size: y #1136

ravwojdyla opened this issue Apr 30, 2018 · 1 comment
Assignees
@albhaf
Labels
bug Something isn't working

Comments

@ravwojdyla
Copy link
Contributor

There has been at least two reports on this error on large "join" jobs, this error seems to be triggered on some data - at least in once case, a pipeline would run fine one day, but consistently fail on other day.

Contact: lanx, cpast

Job id example: 2018-04-24_07_44_16-11213165782653977475
@ravwojdyla ravwojdyla added bug Something isn't working help wanted labels Apr 30, 2018
@lanxxnal
Copy link

lanxxnal commented Apr 30, 2018 •
edited

It sometimes fails with this similar but slightly error:
java.lang.RuntimeException: org.apache.beam.sdk.util.UserCodeException: com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 120

albhaf pushed a commit to albhaf/scio that referenced this issue May 2, 2018
core: fix serialization of large ByteString
3a55960 
Input#read() and Output#write() only operate on single bytes.
Any ByteString larger than supported by that would corrupt the
output and cause issues when reading.

Updates spotify#1136
albhaf pushed a commit to albhaf/scio that referenced this issue May 2, 2018
core: fix serialization of large ByteString
b504f9d 
Input#read() and Output#write() only operate on single bytes.
Any ByteString larger than supported by that would corrupt the
output and cause issues when reading.

Updates spotify#1136
@albhaf albhaf mentioned this issue May 2, 2018
Merged
albhaf pushed a commit to albhaf/scio that referenced this issue May 2, 2018
core: fix serialization of large ByteString
1a01329 
Input#read() and Output#write() only operate on single bytes.
Any ByteString larger than supported by that would corrupt the
output and cause issues when reading.

Updates spotify#1136
nevillelyh pushed a commit that referenced this issue May 2, 2018
@albhaf @nevillelyh
core: fix serialization of large ByteString (#1140)
e20bdc4 
Input#read() and Output#write() only operate on single bytes.
Any ByteString larger than supported by that would corrupt the
output and cause issues when reading.

Updates #1136
@nevillelyh nevillelyh mentioned this issue May 2, 2018
Closed
nevillelyh pushed a commit that referenced this issue May 2, 2018
@albhaf @nevillelyh
core: fix serialization of large ByteString (#1140)
d2a4dbe 
Input#read() and Output#write() only operate on single bytes.
Any ByteString larger than supported by that would corrupt the
output and cause issues when reading.

Updates #1136
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@albhaf albhaf

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@albhaf @nevillelyh @ravwojdyla @lanxxnal