Benchmarks for SQLite ORM

There are many competitor as SQLite ORM for Android platform. To compare them, i reused an existing open source benchmark, kindly offered by greenrobot, the authors of greendao, one the Kripton's competitor.

You can found my fork here. The original benchmark is android-database-performance.

Kind of tests

For benchmarking i used some of my devices. The operations benchmarked are almost the same of original benchmarking tool:

• Batch create: insert rows into database within transaction
• Batch update: update rows in database within transation
• Batch read: read rows from database within transation
• Batch access: simply access from a collection of bean, all single elements. This kind of test is more or less useless, because in most case (except requery library) it's quite an immediate operation.
• Batch delete: delete rows from database with transaction
• Query indexed: execute a query on a table using a column with index
• One by one: insert rows into database without transaction
• One by one update: update rows in database within transation

Compared ORM Library

I try to compare Kripton with all big competitor. All library is open source and freely available:

• greenDao available here.
• dbflow available here
• room: available here. source code
• requery: available here.
• sqlite: try to use directly SQLite without any ORM library.
• ormlite: available here

Used devices

I used the following device, each one with different Android version:

• Xiaomi Mipad with Android 4.4.4
• Nexus 10 with Android 5.0.1
• Galaxy J5 with Android 6.0.1
• Nexus 6 with Android 7.1.1

Before run tests, airplane mode is selected for devices.

Results

Results are misured in milliseconds. Lower results are better. Every row contains result for a specific library. Every columns rapresents time misured for each kind of benchmark. For greenDAO in the query indexed i can not found the benchmark. For every device except the Mipad (Android 4.4.4) i was able to monitor memory usage too. I report min and max value in the last two columns of each tables.

Numbers does not be read as absolute values: it is better to compare, on specific device and Android version, number of one library to other ones.

For each device i create some graphs.

Mipad (android 4.4.4)

mipad (android 4.4.4)	BATCH CREATE	BATCH UPDATE	BATCH READ	BATCH ACCESS	BATCH DELETE	QUERY INDEXED	ONE BY ONE CREATE	ONE BY ONE UPDATE
greenDAO	603	817	318	4.5	72		4458	4809
dbflow	588	6581	533	3	52	5050	4506	5212
kripton	716	784	447	6.5	7.5	226	1415	1094
room	639	846	352	6	124.5	346	4661.5	4887
requery	1470	6209	819	142	73	592	5796	5383
sqlite	1860	2635	348	6	62	289	4612	4563
ormlite	1233	1699	718	13	69	528	5635	4503

Nexus 10 (android 5.0.1)

nexus 10 (android 5.0.1)	BATCH CREATE	BATCH UPDATE	BATCH READ	BATCH ACCESS	BATCH DELETE	QUERY INDEXED	ONE BY ONE CREATE	ONE BY ONE UPDATE	Mem min	Mem max
greenDAO	466	641	357	29	30		8149	7940	12	13
dbflow	472	4790	570	43	25	4716	7914	8375	23	37
kripton	576	606	460	25	15	273	1277	1145	12	24
room	475	632	361	25	30	429	8356	8335	13	30
requery	868	2560	697	116	31	603	8552	8429	18	37
sqlite	819	1549	391	5	25	424	7850	7821	12	31
ormlite	936	1154	656	3	23	596	9119	7361	12	38

Galaxy J5 (android 6.0.1)

Galaxy J5 (android 6.0.1)	BATCH CREATE	BATCH UPDATE	BATCH READ	BATCH ACCESS	BATCH DELETE	QUERY INDEXED	ONE BY ONE CREATE	ONE BY ONE UPDATE	Mem min	Mem max
greenDAO	549	784	413	36	16		1698	1756	9
dbflow	569	6603	658	43	20	7272	1585	2498	16	22
kripton	731	731	567	30	8	324	1445	1156	12	19
room	539	776	432	31	18	563	2476	2364	12	24
requery	1263	4458	969	133	16	974	2648	2409	13	29
sqlite	1103	2308	425	7	14	531	1668	1722	13	24
ormlite	1170	1563	857	6	17	909	5178	1537	11	29

Nexus 6 (Android 7.1.1)

This device has kripted file system. This is the reason, i think, because thid device has worst benchmark results respect Galaxy J5 that is a lower performant device.

Nexus 6 (android 7.1.1)	BATCH CREATE	BATCH UPDATE	BATCH READ	BATCH ACCESS	BATCH DELETE	QUERY INDEXED	ONE BY ONE CREATE	ONE BY ONE UPDATE	Mem min	Mem max
greenDAO	1284	1639	615	1	35		4950	4754	8	13
dbflow	1060	10860	1323	2	33	5840	4525	5679	9	24
kripton	1244	1272	1004	6	19	428	1160	898	10	19
room	1074	1290	707	6	21	660	5103	5004	11	22
requery	1770	5486	1592	80	35	1031	5438	5419	15	36
sqlite	1975	3170	721	2	19	629	4935	5082	11	12
ormlite	2169	3183	1867	2	27	954	5219	4712	10	39

In the following subsection there are some graphs about this numbers:

• Android 7.1.1
• Android 6.0.1
• Android 5.0.1
• Android 4.4.4