Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KryoException: Buffer underflow #4

Open
pilgr opened this issue Jul 4, 2015 · 19 comments
Open

KryoException: Buffer underflow #4

pilgr opened this issue Jul 4, 2015 · 19 comments

Comments

@pilgr
Copy link
Owner

pilgr commented Jul 4, 2015

Very rarely crash happening in AppDialer app using Paper, affects only 0,05% sessions.

Caused by: com.esotericsoftware.kryo.KryoException: Buffer underflow.
Serialization trace:
a (io.paperdb.PaperTable)
       at com.esotericsoftware.kryo.io.Input.require(Input.java:199)
       at com.esotericsoftware.kryo.io.Input.readVarInt(Input.java:373)
       at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:127)
       at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:667)
       at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:118)
       at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:557)
       at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:686)
       at io.paperdb.DbStoragePlainFile.readTableFile(DbStoragePlainFile.java:204)
       at io.paperdb.DbStoragePlainFile.select(DbStoragePlainFile.java:128)
       at io.paperdb.Paper.get(Paper.java:52)
       at name.pilgr.appdialer.Storage.getHistory(Storage.java:54)
       at name.pilgr.appdialer.search.StatsManager.init(StatsManager.java:18)
       at name.pilgr.appdialer.Engine.setFire(Engine.java:197)
       at name.pilgr.appdialer.Engine.doStartUp(Engine.java:235)
       at name.pilgr.appdialer.Engine$1.run(Engine.java:70)
       at name.pilgr.appdialer.util.CrashableTask.run(CrashableTask.java:18)
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:422)
       at java.util.concurrent.FutureTask.run(FutureTask.java:237)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
       at java.lang.Thread.run(Thread.java:841)
@pilgr
Copy link
Owner Author

pilgr commented Jul 4, 2015

Looks like happens only once per affected user. All further data read attempts are successful.

@raviteja06
Copy link

@pilgr Hi, I was using this library as an experiment in one of my projects. But I am having this issue every time I tried to retrieve an object for the first time. It is a specific object only.

@pilgr
Copy link
Owner Author

pilgr commented Dec 22, 2015

@raviteja06 Is it 100% reproducible in your case? Could you write a unit test to reproduce it?

@raviteja06
Copy link

@pilgr I guess it's mostly because of different things I added in the development process. I cleared the cache and tried it, Never got that again.

@martinbonnin
Copy link
Contributor

martinbonnin commented Sep 19, 2016

I got this too on 0.1% of the sessions. I have never been able to reproduce on one of my devices so far but it is a significant cause of crashes. Also most of these crashes happen at startup which is not good.

@martinbonnin
Copy link
Contributor

martinbonnin commented Sep 19, 2016

Could it be linked to EsotericSoftware/kryo#128 ? In all cases, can you put a section about Thread Safety in the README.md ? I understand it's ok to call Paper.book().read/write() from multiple threads, am I correct ?

@pilgr
Copy link
Owner Author

pilgr commented Sep 19, 2016

@martinbonnin yes, you can call read/write from different threads.

@fingertricks
Copy link

fingertricks commented Oct 3, 2016

@pilgr I'm also seeing this crash on a client's app, which uses paperdb v1.1:

Fatal Exception: io.paperdb.PaperDbException: Couldn't read/deserialize file /data/user/0/com.justeat.app.uk/files/log/counter.pt for table counter
       at io.paperdb.DbStoragePlainFile.readTableFile(SourceFile:241)
       at io.paperdb.DbStoragePlainFile.select(SourceFile:130)
       at io.paperdb.Book.read(SourceFile:65)
...
Caused by com.esotericsoftware.kryo.KryoException: Buffer underflow.
Serialization trace:
mContent (io.paperdb.PaperTable)
       at com.esotericsoftware.kryo.io.Input.require(SourceFile:199)
       at com.esotericsoftware.kryo.io.Input.readVarInt(SourceFile:373)
       at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(SourceFile:127)
       at com.esotericsoftware.kryo.Kryo.readClass(SourceFile:667)
       at com.esotericsoftware.kryo.serializers.ObjectField.read(SourceFile:118)
       at com.esotericsoftware.kryo.serializers.FieldSerializer.read(SourceFile:551)
       at com.esotericsoftware.kryo.Kryo.readObject(SourceFile:686)
       at io.paperdb.DbStoragePlainFile.readTableFile(SourceFile:222)
       at io.paperdb.DbStoragePlainFile.select(SourceFile:130)
       at io.paperdb.Book.read(SourceFile:65)
...

We're also seeing it in a very small number of crashes per session, but according to Crashlytics the number of crashes is greater than the number of users impacted (each affected user has seen the crash 1.3x on average).

Crashlytics device insights indicate that the crash has only been seen on HTC and OnePlus devices running Android 6.0 or 6.0.1. Does that correspond with what you're seeing with AppDialer?

@pilgr pilgr changed the title Very rarely crash on read KryoException: Buffer underflow Oct 23, 2017
@hwangjr
Copy link

hwangjr commented Jan 4, 2018

I face this error too:

device: Mi 2
Android version: 5.0.2

+--- io.paperdb:paperdb:2.6
|    +--- com.esotericsoftware:kryo:4.0.1
|    |    +--- com.esotericsoftware:reflectasm:1.11.3
|    |    |    \--- org.ow2.asm:asm:5.0.4
|    |    +--- com.esotericsoftware:minlog:1.3.0
|    |    \--- org.objenesis:objenesis:2.5.1
|    \--- de.javakaffee:kryo-serializers:0.40
|         \--- com.esotericsoftware:kryo:4.0.0 -> 4.0.1 (*)
Process: com.angelia.wancai, PID: 26251

io.paperdb.PaperDbException: Couldn't read/deserialize file /data/data/com.angelia.wancai/files/io.paperdb/key_app_configuration.pt for table key_app_configuration

at io.paperdb.DbStoragePlainFile.readTableFile(DbStoragePlainFile.java:298)

at io.paperdb.DbStoragePlainFile.select(DbStoragePlainFile.java:158)

at io.paperdb.Book.read(Book.java:73)

at io.paperdb.Book.read(Book.java:58)

at com.angelia.wancai.MainApplication.config(MainApplication.java:98)

at com.angelia.wancai.utils.AppConfigurationUtils$2.onResponse(AppConfigurationUtils.java:74)

at okhttp3.RealCall$AsyncCall.execute(RealCall.java:135)

at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)

at java.lang.Thread.run(Thread.java:818)

Caused by: com.esotericsoftware.kryo.KryoException: Buffer underflow.

Serialization trace:

mContent (io.paperdb.PaperTable)

at com.esotericsoftware.kryo.io.Input.require(Input.java:199)

at com.esotericsoftware.kryo.io.Input.readVarInt(Input.java:373)

at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:127)

at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:693)

at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:118)

at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543)

at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:712)

at io.paperdb.DbStoragePlainFile.readContent(DbStoragePlainFile.java:307)

at io.paperdb.DbStoragePlainFile.readTableFile(DbStoragePlainFile.java:291)

... 10 more

@steven274
Copy link

I am facing this error when .pt file show 0 bytes

@Manikandan92
Copy link

Manikandan92 commented Dec 12, 2018

I have updated the latest version of 2.6. While crash occurring on the latest version also.
Please check the below track.

Caused by io.paperdb.PaperDbException: Couldn't read/deserialize file /data/user/0/xxx.xx.xxxxxxxxx/files/mallow/TableName.pt for table TableName.pt
at io.paperdb.DbStoragePlainFile.readTableFile(DbStoragePlainFile.java:298)
at io.paperdb.DbStoragePlainFile.select(DbStoragePlainFile.java:158)
at io.paperdb.Book.read(Book.java:73)
at xxx.xx.xxxxxxxxx(PaperDB.java:110)
at xxx.xx.xxxxxxxxx(MainActivity.java:382)
at xxx.xx.xxxxxxxxx(MainActivity.java:347)
at android.app.Activity.performCreate(Activity.java:6701)
at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1118)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2677)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2789)
at android.app.ActivityThread.-wrap12(ActivityThread.java)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1527)
at android.os.Handler.dispatchMessage(Handler.java:110)
at android.os.Looper.loop(Looper.java:208)
at android.app.ActivityThread.main(ActivityThread.java:6267)
at java.lang.reflect.Method.invoke(Method.java)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1063)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:924)

Caused by com.esotericsoftware.kryo.KryoException: Buffer underflow.
Serialization trace:
journeyState (com.fleetrover.datamodel.TrackCurrentLocation)
mContent (io.paperdb.PaperTable)
at com.esotericsoftware.kryo.io.Input.require(Input.java:199)
at com.esotericsoftware.kryo.io.Input.readAscii_slow(Input.java:616)
at com.esotericsoftware.kryo.io.Input.readAscii(Input.java:594)
at com.esotericsoftware.kryo.io.Input.readString(Input.java:472)
at com.esotericsoftware.kryo.serializers.DefaultSerializers$StringSerializer.read(DefaultSerializers.java:199)
at com.esotericsoftware.kryo.serializers.DefaultSerializers$StringSerializer.read(DefaultSerializers.java:188)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:789)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
at com.esotericsoftware.kryo.serializers.CompatibleFieldSerializer.read(CompatibleFieldSerializer.java:145)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:734)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:712)
at io.paperdb.DbStoragePlainFile.readContent(DbStoragePlainFile.java:307)
at io.paperdb.DbStoragePlainFile.readTableFile(DbStoragePlainFile.java:291)
at io.paperdb.DbStoragePlainFile.select(DbStoragePlainFile.java:158)
at io.paperdb.Book.read(Book.java:73)
at xxx.xx.xxxxxxxxx(PaperDB.java:110)
at xxx.xx.xxxxxxxxx(MainActivity.java:382)
at xxx.xx.xxxxxxxxx(MainActivity.java:347)
at android.app.Activity.performCreate(Activity.java:6701)
at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1118)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2677)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2789)
at android.app.ActivityThread.-wrap12(ActivityThread.java)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1527)
at android.os.Handler.dispatchMessage(Handler.java:110)
at android.os.Looper.loop(Looper.java:208)
at android.app.ActivityThread.main(ActivityThread.java:6267)
at java.lang.reflect.Method.invoke(Method.java)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1063)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:924)

@vzts
Copy link

vzts commented Jan 10, 2019

same here as @Manikandan92

@condesales
Copy link

I can reproduce the problem 100% of the times.
For me, it's happening because I moved the class I was saving to a different package.

@Joseph82
Copy link

We went in production recently (3 days ago) and we just registered two episodes: both for the same user.

Still no idea about what can cause it?

@brandonpage
Copy link

This issue is still present in version 2.7.1. It is rare and not immediately reproducible like 2.6 but it still happens. Please catch the exception within Paper.

The crash occurs within our SDK at this read line.

com.esotericsoftware.kryo.KryoException: Buffer underflow. Serialization trace: mContent (io.paperdb.PaperTable)

com.esotericsoftware.kryo.io.Input.readVarInt Input.java:373
com.esotericsoftware.kryo.util.DefaultClassResolver.readClass DefaultClassResolver.java:127
com.esotericsoftware.kryo.Kryo.readClass Kryo.java:693
com.esotericsoftware.kryo.serializers.ObjectField.read ObjectField.java:118
com.esotericsoftware.kryo.serializers.FieldSerializer.read FieldSerializer.java:543
com.esotericsoftware.kryo.Kryo.readObject Kryo.java:712
io.paperdb.DbStoragePlainFile.readContent DbStoragePlainFile.java:332
io.paperdb.DbStoragePlainFile.readTableFile DbStoragePlainFile.java:316
io.paperdb.DbStoragePlainFile.select DbStoragePlainFile.java:166
io.paperdb.Book.read Book.java:73
com.salesforce.androidsdk.analytics.store.EventStoreManager.fetchEvent EventStoreManager.java:123
com.salesforce.androidsdk.analytics.store.EventStoreManager.fetchAllEvents EventStoreManager.java:147
com.salesforce.androidsdk.analytics.SalesforceAnalyticsManager.publishAllEvents SalesforceAnalyticsManager.java:279
com.salesforce.mobile.analytics.ept.SalesforceAILTNEvent$PublishTask.doInBackground SalesforceAILTNEvent.java:605
com.salesforce.mobile.analytics.ept.SalesforceAILTNEvent$PublishTask.doInBackground SalesforceAILTNEvent.java:591
android.os.AsyncTask$2.call AsyncTask.java:333
java.util.concurrent.FutureTask.run FutureTask.java:266
android.os.AsyncTask$SerialExecutor$1.run AsyncTask.java:245
java.util.concurrent.ThreadPoolExecutor.runWorker ThreadPoolExecutor.java:1167
java.util.concurrent.ThreadPoolExecutor$Worker.run ThreadPoolExecutor.java:641
java.lang.Thread.run Thread.java:764

io.paperdb.DbStoragePlainFile.readTableFile DbStoragePlainFile.java:323
io.paperdb.DbStoragePlainFile.select DbStoragePlainFile.java:166
io.paperdb.Book.read Book.java:73
com.salesforce.androidsdk.analytics.store.EventStoreManager.fetchEvent EventStoreManager.java:123
com.salesforce.androidsdk.analytics.store.EventStoreManager.fetchAllEvents EventStoreManager.java:147
com.salesforce.androidsdk.analytics.SalesforceAnalyticsManager.publishAllEvents SalesforceAnalyticsManager.java:279
com.salesforce.mobile.analytics.ept.SalesforceAILTNEvent$PublishTask.doInBackground SalesforceAILTNEvent.java:605
com.salesforce.mobile.analytics.ept.SalesforceAILTNEvent$PublishTask.doInBackground SalesforceAILTNEvent.java:591
android.os.AsyncTask$2.call AsyncTask.java:333
java.util.concurrent.FutureTask.run FutureTask.java:266
android.os.AsyncTask$SerialExecutor$1.run AsyncTask.java:245
java.util.concurrent.ThreadPoolExecutor.runWorker ThreadPoolExecutor.java:1167
java.util.concurrent.ThreadPoolExecutor$Worker.run ThreadPoolExecutor.java:641
java.lang.Thread.run Thread.java:764

@brandonpage
Copy link

Hello @pilgr, do you have any intention of fixing or catching this issue within Paper?

@pilgr
Copy link
Owner Author

pilgr commented Dec 7, 2020

@brandonpage since this is incredibly rare and hardly to be reproduced, I'm not sure how to fix it. We may add some workarounds like second read attempts though. I'm wondering how often does it happen comparing to the total number of the user sessions?

@rufuszhu
Copy link

rufuszhu commented Dec 7, 2020

@pilgr This buffer underflow crash is also happening in our app as we are using SalesforceMobileSDK, which use version 2.7.1 of paper db.
This issue only affected a few of our users but once it happens it will crash consistently (crash 200+ times for some of our users). Also because this crash is happening to those users consistently, the buffer underflow crash has pushed our crash per session rate to almost 4%

@pilgr
Copy link
Owner Author

pilgr commented Dec 7, 2020

@rufuszhu my understanding is if issue happens all the time for particular user, that might be caused by app update which used new package name for the class holding data. Please check out this message mentioned above.

I can reproduce the problem 100% of the times.
For me, it's happening because I moved the class I was saving to a different package.

If the class has been stored with one package name and then on read the different class is expected (even if only package name has been changed) that may cause the issue.

For the same exception which happens very rarely I assume some other root cause takes place, that is still unknown.

XMKhalid added a commit to xmedius/Paper that referenced this issue Nov 17, 2022
=> error: pilgr#4
They said it was fixed in paperdb 2.7.2, but not really!
fix: pilgr#194 (upgrade kryo)

=> why (kryo.setRegistrationRequired(false);)
EsotericSoftware/kryo#398 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests