OOM during ToMany update

[motornyimaksym]

Issue Basics

• ObjectBox version: 1.4.4
• Reproducibility: always

Reproducing the bug

Description

Important: this bug related only to Many-to-Many relation type.

1. During update entity ToMany field with some amount of existing data I got OOM.
2. Updating complexity increase non linear. More existing relations need more time and memory consumption.

Code

Project available at: https://github.com/Yazon2006/ObjectBoxIssue

class MainActivity : AppCompatActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
        val boxStore = MyObjectBox.builder().androidContext(applicationContext).build()
        boxStore.startObjectBrowser()
        boxStore.boxFor(Teacher::class.java).put(Teacher())

        while (true) {
            val students = ArrayList<Student>()
            for (i in 0 until 1000) {
                students.add(Student())
            }

            val start = System.nanoTime()
            val teacher = boxStore.boxFor(Teacher::class.java)[777]
            teacher.students.addAll(students)
            Log.d("DBLog", "Students list size: ${teacher.students.size} ")
            boxStore.boxFor(Teacher::class.java).put(teacher)
            val tookTime = ((System.nanoTime() - start) / 1000000)
            Log.d("DBLog", "Updating students took: $tookTime ms")
        }
    }
}

Logs & stackstraces

17:28:58 D/DBLog: Students list size: 1000 
17:28:58 D/DBLog: Updating students took: 27 ms
17:28:58 D/DBLog: Students list size: 2000 
17:28:58 D/DBLog: Updating students took: 337 ms
17:28:59 D/DBLog: Students list size: 3000 
17:28:59 D/DBLog: Updating students took: 653 ms
17:29:00 D/DBLog: Students list size: 4000 
17:29:00 D/DBLog: Updating students took: 835 ms
17:29:02 D/DBLog: Students list size: 5000 
17:29:02 D/DBLog: Updating students took: 1170 ms
17:29:03 D/DBLog: Students list size: 6000 
17:29:04 D/DBLog: Updating students took: 1489 ms
17:29:05 D/DBLog: Students list size: 7000 
17:29:05 D/DBLog: Updating students took: 1712 ms
17:29:08 D/DBLog: Students list size: 8000 
17:29:08 D/DBLog: Updating students took: 1969 ms
17:29:10 D/DBLog: Students list size: 9000 
17:29:10 D/DBLog: Updating students took: 2179 ms
17:29:13 D/DBLog: Students list size: 10000 
17:29:13 D/DBLog: Updating students took: 2545 ms
17:29:16 D/DBLog: Students list size: 11000 
17:29:16 D/DBLog: Updating students took: 2793 ms
17:29:19 D/DBLog: Students list size: 12000 
17:29:19 D/DBLog: Updating students took: 3065 ms
17:29:23 D/DBLog: Students list size: 13000 
17:29:23 D/DBLog: Updating students took: 3421 ms
17:29:27 D/DBLog: Students list size: 14000 
17:29:27 D/DBLog: Updating students took: 3610 ms
17:29:31 D/DBLog: Students list size: 15000 
17:29:31 D/DBLog: Updating students took: 3831 ms
17:29:35 D/DBLog: Students list size: 16000 
17:29:35 D/DBLog: Updating students took: 4122 ms
17:29:40 D/DBLog: Students list size: 17000 
17:29:40 D/DBLog: Updating students took: 4624 ms
17:29:45 D/DBLog: Students list size: 18000 
17:29:45 D/DBLog: Updating students took: 4757 ms
17:29:51 D/DBLog: Students list size: 19000 
17:29:51 D/DBLog: Updating students took: 5135 ms
17:29:56 D/DBLog: Students list size: 20000 
17:29:56 D/DBLog: Updating students took: 5267 ms
17:30:02 D/DBLog: Students list size: 21000 
17:30:02 D/DBLog: Updating students took: 5738 ms
17:30:09 D/DBLog: Students list size: 22000 
17:30:09 D/DBLog: Updating students took: 5923 ms
17:30:15 D/DBLog: Students list size: 23000 
17:30:15 D/DBLog: Updating students took: 6211 ms
17:30:22 D/DBLog: Students list size: 24000 
17:30:22 D/DBLog: Updating students took: 6660 ms
17:30:29 D/DBLog: Students list size: 25000 
17:30:29 D/DBLog: Updating students took: 6859 ms
17:30:37 D/DBLog: Students list size: 26000 
17:30:37 D/DBLog: Updating students took: 7293 ms
17:30:45 E/art: Throwing OutOfMemoryError "Failed to allocate a 262156 byte allocation with 4194208 free bytes and 168MB until OOM; failed due to fragmentation (required continguous free 266240 bytes where largest contiguous free 196608 bytes)"
17:30:45 D/AndroidRuntime: Shutting down VM

17:30:45 E/AndroidRuntime: FATAL EXCEPTION: main
    Process: ua.motorny.objectbox.objectboxissue, PID: 6805
    java.lang.OutOfMemoryError: Failed to allocate a 262156 byte allocation with 4194208 free bytes and 168MB until OOM; failed due to fragmentation (required continguous free 266240 bytes where largest contiguous free 196608 bytes)
        at java.util.HashMap.makeTable(HashMap.java:550)
        at java.util.HashMap.doubleCapacity(HashMap.java:570)
        at java.util.HashMap.put(HashMap.java:400)
        at io.objectbox.relation.ToMany.ensureEntitiesWithTrackingLists(ToMany.java:163)
        at io.objectbox.relation.ToMany.trackAdd(ToMany.java:230)
        at io.objectbox.relation.ToMany.addAll(ToMany.java:264)
        at ua.motorny.objectbox.objectboxissue.MainActivity.onCreate(MainActivity.kt:28)
        at android.app.Activity.performCreate(Activity.java:5990)
        at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1106)
        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2278)
        at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2387)
        at android.app.ActivityThread.access$800(ActivityThread.java:151)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1303)
        at android.os.Handler.dispatchMessage(Handler.java:102)
        at android.os.Looper.loop(Looper.java:135)
        at android.app.ActivityThread.main(ActivityThread.java:5254)
        at java.lang.reflect.Method.invoke(Native Method)
        at java.lang.reflect.Method.invoke(Method.java:372)
        at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:903)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:698)

Entities

@Entity
class Teacher(
        @Id(assignable = true)
        var id: Long = 777
) {
    lateinit var students: ToMany<Student>
}

@Entity
class Student(
        @Id
        var id: Long = 0
) {
    var field: String

    init {
        val stringBuilder = StringBuilder()
        for (i in 0 until 100) {
            stringBuilder.append(UUID.randomUUID().toString())
        }
        field = stringBuilder.toString()
    }
}

Misc

My main concern that it is necessary to get all entity with all related data to update its ToMany field. That's why each time when I want to add even only one entity in collection It takes a lot of time and memory.

Maybe @Backlink for Many-to-Many relations will partially solve my issue? Or something like adding relation without getting parent entity?

[motornyimaksym]

@greenrobot Can you take a look and say if it possible to improve ToMany update process? It loos like a major bug.

[greenrobot]

We'll look into it.

[greenrobot-team]

Related to #281 -ut

[motornyimaksym]

@greenrobot-team Yes, I mentioned it in #281 but @greenrobot suggested to create standalone ticket =)
Looking forward!

[motornyimaksym]

Any updates since March 15? Can you type on the ObjectBox main web page "DO NOT USE MANY-TO-MANY RELATIONS" until you solve this critical bug?

[greenrobot]

Maybe @Backlink for Many-to-Many relations will partially solve my issue?

If it would, please check version 2.0.0-beta. Will put this on the list for 2.0 final.

[motornyimaksym]

@greenrobot Yes, it works!

[greenrobot]

2.0.0 released; Off-loaded further improvements to #510