Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add some benchmarks for odbv1 memory consumption #11

Merged
merged 7 commits into from Feb 20, 2023
Merged

add some benchmarks for odbv1 memory consumption #11

merged 7 commits into from Feb 20, 2023

Conversation

bbrehm
Copy link
Contributor

@bbrehm bbrehm commented Feb 17, 2023

This adds some benchmarks for joern/odbv1 in order to get a somewhat reproducible baseline.

@bbrehm bbrehm requested a review from ml86 February 17, 2023 16:50
@bbrehm
Copy link
Contributor Author

bbrehm commented Feb 17, 2023 •
edited

Example run below (joern importCode from current master of vlc player). We already have our first performance riddle @mpollmeier : We have twice as many NodeRef instances as NodeDb instances (correct number of NodeDb instances, incorrect number of NodeRef instances). Do we understand why that is the case?

[bruhns@bruhnsWS overflowdbv2]$ ./benchJoern/target/universal/stage/bin/bench-joern -J-Xmx20G  -Djdk.attach.allowAttachSelf ./cpg.bin 

VM is version 19.0.2+7 with max heap 20480 mb.


SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
Graph with 2387850 nodes (709958 calls) and 20220777 edges at ./cpg.bin

<=========
Running complete benchmark in 17489655192 ns  == 17489.655192 ms == 7.3244362887116035 us/node and costing 1156858400 bytes == 1103.2661437988281 MB == 484.47699813639883 bytes/node.
Top 20 account for 92.34579167165143%:
Class, #instances, total size, instances/node, bytes/instance, bytes/node
Object[], 2389516, 425952024, 1.000697698766673,  178.25870343617703, 178.38307431371317
byte[], 3031234, 189729688, 1.26944071026237,  62.59156765858393, 79.45628410494797
overflowdb.AdjacentNodes, 2387850, 57308400, 1.0,  24.0, 24.0
io.shiftleft.codepropertygraph.generated.nodes.Call, 1419916, 56796640, 0.5946420420043135,  40.0, 23.78568168017254
io.shiftleft.codepropertygraph.generated.nodes.CallDb, 709958, 51116976, 0.29732102100215674,  72.0, 21.407113512155288
io.shiftleft.codepropertygraph.generated.nodes.Identifier, 1097266, 43890640, 0.45952048914295285,  40.0, 18.380819565718113
long[], 5, 42029408, 2.0939338735682726E-6,  8405881.6, 17.60136021944427
io.shiftleft.codepropertygraph.generated.nodes.IdentifierDb, 548633, 35112512, 0.22976024457147642,  64.0, 14.704655652574491
Integer, 1799008, 28784128, 0.7534007580040623,  16.0, 12.054412128064996
int[], 452, 21087936, 1.8929162217057186E-4,  46654.72566371682, 8.831348702807965
String, 681085, 16346040, 0.2852293904558494,  24.0, 6.845505370940386
io.shiftleft.codepropertygraph.generated.nodes.FieldIdentifier, 401658, 16066320, 0.16820905835793706,  40.0, 6.728362334317482
io.shiftleft.codepropertygraph.generated.nodes.Literal, 358342, 14333680, 0.1500688904244404,  40.0, 6.002755616977616
io.shiftleft.codepropertygraph.generated.nodes.Block, 303730, 12149200, 0.1271981070837783,  40.0, 5.087924283351132
overflowdb.Node[], 1, 11676976, 4.1878677471365454E-7,  1.1676976E7, 4.8901631174487505
io.shiftleft.codepropertygraph.generated.nodes.FieldIdentifierDb, 200829, 11246424, 0.08410452917896853,  56.0, 4.709853634022237
io.shiftleft.codepropertygraph.generated.nodes.LiteralDb, 179171, 10033576, 0.0750344452122202,  56.0, 4.201928931884331
java.util.concurrent.ConcurrentHashMap$Node[], 50716, 8842880, 0.021239190066377703,  174.3607540026816, 3.7032811943798816
io.shiftleft.codepropertygraph.generated.nodes.BlockDb, 151865, 8504440, 0.06359905354188915,  56.0, 3.561546998345792
io.shiftleft.codepropertygraph.generated.nodes.MethodParameterIn, 182554, 7302160, 0.07645120087107649,  40.0, 3.05804803484306
=========>



<=========
Running copy and load cpg file in 2965010740 ns  == 2965.0107399999997 ms == 1.2417072847959463 us/node and costing 212889912 bytes == 203.02764129638672 MB == 89.15547961555374 bytes/node.
Top 20 account for 95.25464785762136%:
Class, #instances, total size, instances/node, bytes/instance, bytes/node
long[], 13970, 42436376, 0.005850451242749754,  3037.6790264853257, 17.771793035575936
io.shiftleft.codepropertygraph.generated.nodes.Call, 709958, 28398320, 0.29732102100215674,  40.0, 11.89284084008627
io.shiftleft.codepropertygraph.generated.nodes.Identifier, 548633, 21945320, 0.22976024457147642,  40.0, 9.190409782859057
int[], 393, 21020464, 1.6458320246246625E-4,  53487.185750636134, 8.803092321544487
byte[], 81732, 12451800, 0.034228280670896415,  152.34914109528702, 5.214649161379484
Object[], 3951, 12415568, 0.001654626546893649,  3142.3862313338395, 5.199475678958058
overflowdb.Node[], 1, 11676976, 4.1878677471365454E-7,  1.1676976E7, 4.8901631174487505
io.shiftleft.codepropertygraph.generated.nodes.FieldIdentifier, 200829, 8033160, 0.08410452917896853,  40.0, 3.364181167158741
io.shiftleft.codepropertygraph.generated.nodes.Literal, 179171, 7166840, 0.0750344452122202,  40.0, 3.001377808488808
io.shiftleft.codepropertygraph.generated.nodes.Block, 151865, 6074600, 0.06359905354188915,  40.0, 2.543962141675566
java.util.concurrent.ConcurrentHashMap$Node[], 50716, 4648704, 0.021239190066377703,  91.66148749901411, 1.9468157547584648
java.util.concurrent.ConcurrentHashMap$Node, 139703, 4470496, 0.05850576878782168,  32.0, 1.8721846012102938
io.shiftleft.codepropertygraph.generated.nodes.MethodParameterIn, 91277, 3651080, 0.03822560043553824,  40.0, 1.52902401742153
io.shiftleft.codepropertygraph.generated.nodes.MethodParameterOut, 91277, 3651080, 0.03822560043553824,  40.0, 1.52902401742153
java.util.concurrent.ConcurrentHashMap, 50718, 3245952, 0.02124002763992713,  64.0, 1.3593617689553363
io.shiftleft.codepropertygraph.generated.nodes.ControlStructure, 80994, 3239760, 0.03391921603115774,  40.0, 1.3567686412463094
io.shiftleft.codepropertygraph.generated.nodes.Local, 56877, 2275080, 0.02381933538538853,  40.0, 0.9527734154155412
io.shiftleft.codepropertygraph.generated.nodes.Method, 55929, 2237160, 0.023422325522959985,  40.0, 0.9368930209183994
io.shiftleft.codepropertygraph.generated.nodes.MethodReturn, 55929, 2237160, 0.023422325522959985,  40.0, 0.9368930209183994
io.shiftleft.codepropertygraph.generated.nodes.Binding, 37791, 1511640, 0.015826371003203717,  40.0, 0.6330548401281487
=========>



<=========
Running load/create indexes in 33230 ns  == 0.033229999999999996 ms == 1.3916284523734743E-5 us/node and costing 88392 bytes == 0.08429718017578125 MB == 0.03701740059048935 bytes/node.
Top 20 account for 100.0%:
Class, #instances, total size, instances/node, bytes/instance, bytes/node
byte[], 1035, 64352, 4.334443118286325E-4,  62.17584541062802, 0.026949766526373098
String, 1035, 24840, 4.334443118286325E-4,  24.0, 0.010402663483887178
jdk.internal.ref.CleanerImpl$PhantomCleanableRef, -10, -480, -4.187867747136545E-6,  48.0, -2.0101765186255417E-4
java.lang.invoke.MethodHandleNatives$CallSiteContext, -10, -320, -4.187867747136545E-6,  32.0, -1.3401176790836945E-4
=========>



<=========
Running count edges (force complete loading) in 14156499550 ns  == 14156.499549999999 ms == 5.928554787779802 us/node and costing 969210216 bytes == 924.3108901977539 MB == 405.89242037816445 bytes/node.
Top 20 account for 95.22555754818829%:
Class, #instances, total size, instances/node, bytes/instance, bytes/node
Object[], 2389000, 413926328, 1.0004816047909206,  173.2634273754709, 173.34687187218628
byte[], 2992001, 180426632, 1.253010448730029,  60.302998561832034, 75.56028728772745
overflowdb.AdjacentNodes, 2387850, 57308400, 1.0,  24.0, 24.0
io.shiftleft.codepropertygraph.generated.nodes.CallDb, 709958, 51116976, 0.29732102100215674,  72.0, 21.407113512155288
io.shiftleft.codepropertygraph.generated.nodes.IdentifierDb, 548633, 35112512, 0.22976024457147642,  64.0, 14.704655652574491
Integer, 1798992, 28783872, 0.7533940574156668,  16.0, 12.054304918650669
io.shiftleft.codepropertygraph.generated.nodes.Call, 709958, 28398320, 0.29732102100215674,  40.0, 11.89284084008627
io.shiftleft.codepropertygraph.generated.nodes.Identifier, 548633, 21945320, 0.22976024457147642,  40.0, 9.190409782859057
java.util.concurrent.ConcurrentHashMap$Node, 622977, 19935264, 0.2608945285507884,  32.0, 8.348624913625228
String, 623331, 14959944, 0.261042779069037,  24.0, 6.265026697656888
io.shiftleft.codepropertygraph.generated.nodes.FieldIdentifierDb, 200829, 11246424, 0.08410452917896853,  56.0, 4.709853634022237
io.shiftleft.codepropertygraph.generated.nodes.LiteralDb, 179171, 10033576, 0.0750344452122202,  56.0, 4.201928931884331
io.shiftleft.codepropertygraph.generated.nodes.BlockDb, 151865, 8504440, 0.06359905354188915,  56.0, 3.561546998345792
io.shiftleft.codepropertygraph.generated.nodes.FieldIdentifier, 200829, 8033160, 0.08410452917896853,  40.0, 3.364181167158741
io.shiftleft.codepropertygraph.generated.nodes.Literal, 179171, 7166840, 0.0750344452122202,  40.0, 3.001377808488808
io.shiftleft.codepropertygraph.generated.nodes.Block, 151865, 6074600, 0.06359905354188915,  40.0, 2.543962141675566
io.shiftleft.codepropertygraph.generated.nodes.MethodParameterInDb, 91277, 5841728, 0.03822560043553824,  64.0, 2.4464384278744475
io.shiftleft.codepropertygraph.generated.nodes.MethodParameterOutDb, 91277, 5111512, 0.03822560043553824,  56.0, 2.1406336243901416
io.shiftleft.codepropertygraph.generated.nodes.ControlStructureDb, 80994, 4535664, 0.03391921603115774,  56.0, 1.8994760977448333
io.shiftleft.codepropertygraph.generated.nodes.MethodDb, 55929, 4474320, 0.023422325522959985,  80.0, 1.8737860418367989
=========>



<=========
Running count edges again in 368101711 ns  == 368.10171099999997 ms == 0.15415612831626777 us/node and costing 95904 bytes == 0.091461181640625 MB == 0.040163326842138324 bytes/node.
Top 20 account for 100.0%:
Class, #instances, total size, instances/node, bytes/instance, bytes/node
byte[], 1102, 69520, 4.6150302573444733E-4,  63.08529945553539, 0.029114056578093263
String, 1102, 26448, 4.6150302573444733E-4,  24.0, 0.011076072617626736
jdk.internal.ref.CleanerImpl$PhantomCleanableRef, -1, -48, -4.1878677471365454E-7,  48.0, -2.010176518625542E-5
java.lang.invoke.MethodHandleNatives$CallSiteContext, -1, -32, -4.1878677471365454E-7,  32.0, -1.3401176790836945E-5
Integer, 1, 16, 4.1878677471365454E-7,  16.0, 6.700588395418473E-6
=========>



<=========
Running close graph in 368532341 ns  == 368.532341 ms == 0.15433647046506271 us/node and costing -25223528 bytes == -24.055030822753906 MB == -10.563279938019557 bytes/node.
Top 20 account for 99.76764550938314%:
Class, #instances, total size, instances/node, bytes/instance, bytes/node
java.util.concurrent.ConcurrentHashMap$Node, -622940, -19934080, -0.260879033440124,  32.0, -8.348129070083967
byte[], -41558, -3084872, -0.017403940783550054,  74.23052119928774, -1.2919035952844609
Long, -44511, -1068264, -0.018640618129279477,  24.0, -0.4473748351027075
org.h2.mvstore.cache.CacheLongKeyLIRS$Entry, -8019, -449064, -0.0033582511464287958,  56.0, -0.18806206420001256
Object[], -3613, -395816, -0.0015130766170404339,  109.55327982286188, -0.16576250602005987
java.lang.ref.WeakReference, -6166, -197312, -0.002582239252884394,  32.0, -0.08263165609230061
org.h2.mvstore.Page$Leaf, -1780, -85440, -7.454404589903051E-4,  48.0, -0.035781142031534645
int[], 4, 65600, 1.6751470988546182E-6,  16400.0, 0.027472412421215738
org.h2.mvstore.Page$PageReference, -1875, -60000, -7.852252025881023E-4,  32.0, -0.025127206482819273
java.nio.ByteBuffer[], 11, 45232, 4.6066545218502E-6,  4112.0, 0.018942563393848023
String, 1181, 28344, 4.945871809368261E-4,  24.0, 0.011870092342483825
org.h2.mvstore.Chunk, -232, -25984, -9.715853173356786E-5,  112.0, -0.0108817555541596
org.h2.mvstore.Page$PageReference[], -68, -8680, -2.847750068052851E-5,  127.6470588235294, -0.0036350692045145215
Class, 56, 6352, 2.3452059383964653E-5,  113.42857142857143, 0.0026601335929811337
org.h2.mvstore.Page$NonLeaf, -68, -3808, -2.847750068052851E-5,  56.0, -0.0015947400381095965
Integer, -216, -3456, -9.045794333814938E-5,  16.0, -0.00144732709341039
org.h2.mvstore.MVStore$RemovedPageInfo, 103, 2472, 4.3135037795506416E-5,  24.0, 0.001035240907092154
java.lang.invoke.MethodType$ConcurrentWeakInternSet$WeakEntry, 68, 2176, 2.847750068052851E-5,  32.0, 9.112800217769123E-4
jdk.internal.ref.CleanerImpl$PhantomCleanableRef, 35, 1680, 1.4657537114977908E-5,  48.0, 7.035617815189396E-4
=========>



<=========
Running free memory in 9961 ns  == 0.009961 ms == 4.171535062922713E-6 us/node and costing -1155444832 bytes == -1101.9180603027344 MB == -483.8850145528404 bytes/node.
Top 20 account for 92.39256037453114%:
Class, #instances, total size, instances/node, bytes/instance, bytes/node
Object[], -2388659, -425891760, -1.0003387985007433,  178.29742964567149, -178.35783654752183
byte[], -3022523, -189265712, -1.2657926586678392,  62.618452200363734, -79.26197709236342
overflowdb.AdjacentNodes, -2387850, -57308400, -1.0,  24.0, -24.0
io.shiftleft.codepropertygraph.generated.nodes.Call, -1419916, -56796640, -0.5946420420043135,  40.0, -23.78568168017254
io.shiftleft.codepropertygraph.generated.nodes.CallDb, -709958, -51116976, -0.29732102100215674,  72.0, -21.407113512155288
io.shiftleft.codepropertygraph.generated.nodes.Identifier, -1097266, -43890640, -0.45952048914295285,  40.0, -18.380819565718113
long[], -3, -42029376, -1.2563603241409637E-6,  1.4009792E7, -17.60134681826748
io.shiftleft.codepropertygraph.generated.nodes.IdentifierDb, -548633, -35112512, -0.22976024457147642,  64.0, -14.704655652574491
Integer, -1798992, -28783872, -0.7533940574156668,  16.0, -12.054304918650669
int[], -11, -21072552, -4.6066545218502E-6,  1915686.5454545454, -8.82490608706577
String, -672391, -16137384, -0.28158845823648887,  24.0, -6.758122997675733
io.shiftleft.codepropertygraph.generated.nodes.FieldIdentifier, -401658, -16066320, -0.16820905835793706,  40.0, -6.728362334317482
io.shiftleft.codepropertygraph.generated.nodes.Literal, -358342, -14333680, -0.1500688904244404,  40.0, -6.002755616977616
io.shiftleft.codepropertygraph.generated.nodes.Block, -303730, -12149200, -0.1271981070837783,  40.0, -5.087924283351132
overflowdb.Node[], -1, -11676976, -4.1878677471365454E-7,  1.1676976E7, -4.8901631174487505
io.shiftleft.codepropertygraph.generated.nodes.FieldIdentifierDb, -200829, -11246424, -0.08410452917896853,  56.0, -4.709853634022237
io.shiftleft.codepropertygraph.generated.nodes.LiteralDb, -179171, -10033576, -0.0750344452122202,  56.0, -4.201928931884331
java.util.concurrent.ConcurrentHashMap$Node[], -50682, -8826464, -0.021224951316037438,  174.15382186969734, -3.696406390686182
io.shiftleft.codepropertygraph.generated.nodes.BlockDb, -151865, -8504440, -0.06359905354188915,  56.0, -3.561546998345792
io.shiftleft.codepropertygraph.generated.nodes.MethodParameterIn, -182554, -7302160, -0.07645120087107649,  40.0, -3.05804803484306
=========>



# WARNING: Unable to attach Serviceability Agent. You can try again with escalated privileges. Two options: a) use -Djol.tryWithSudo=true to try with sudo; b) echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
VM details according to JOL: # Running 64-bit HotSpot VM.
# Using compressed oop with 3-bit shift.
# Using compressed klass with 3-bit shift.
# WARNING | Compressed references base/shifts are guessed by the experiment!
# WARNING | Therefore, computed addresses are just guesses, and ARE NOT RELIABLE.
# WARNING | Make sure to attach Serviceability Agent to get the reliable addresses.
# Objects are 8 bytes aligned.
# Field sizes by type: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes]
# Array element sizes: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes]
. Layout of top consumers: 


overflowdb.AdjacentNodes object internals:
OFF  SZ                 TYPE DESCRIPTION                             VALUE
  0   8                      (object header: mark)                   N/A
  8   4                      (object header: class)                  N/A
 12   4   java.lang.Object[] AdjacentNodes.nodesWithEdgeProperties   N/A
 16   4     java.lang.Object AdjacentNodes.offsets                   N/A
 20   4                      (object alignment gap)                  
Instance size: 24 bytes
Space losses: 0 bytes internal + 4 bytes external = 4 bytes total


io.shiftleft.codepropertygraph.generated.nodes.Call object internals:
OFF  SZ                TYPE DESCRIPTION               VALUE
  0   8                     (object header: mark)     N/A
  8   4                     (object header: class)    N/A
 12   4    overflowdb.Graph NodeRef.graph             N/A
 16   8                long NodeRef.id                N/A
 24   4   overflowdb.NodeDb NodeRef.node              N/A
 28   4                     (alignment/padding gap)   
 32   8                long Call.id                   N/A
Instance size: 40 bytes
Space losses: 4 bytes internal + 0 bytes external = 4 bytes total


io.shiftleft.codepropertygraph.generated.nodes.CallDb object internals:
OFF  SZ                                    TYPE DESCRIPTION                       VALUE
  0   8                                         (object header: mark)             N/A
  8   4                                         (object header: class)            N/A
 12   1                                 boolean NodeDb.dirty                      N/A
 13   3                                         (alignment/padding gap)           
 16   4                      overflowdb.NodeRef NodeDb.ref                        N/A
 20   4                overflowdb.AdjacentNodes NodeDb.adjacentNodes              N/A
 24   4                                     int CallDb._argumentIndex             N/A
 28   4                                     int CallDb._order                     N/A
 32   4                        java.lang.String CallDb._argumentName              N/A
 36   4                        java.lang.String CallDb._code                      N/A
 40   4                       java.lang.Integer CallDb._columnNumber              N/A
 44   4                        java.lang.String CallDb._dispatchType              N/A
 48   4   scala.collection.immutable.IndexedSeq CallDb._dynamicTypeHintFullName   N/A
 52   4                       java.lang.Integer CallDb._lineNumber                N/A
 56   4                        java.lang.String CallDb._methodFullName            N/A
 60   4                        java.lang.String CallDb._name                      N/A
 64   4                        java.lang.String CallDb._signature                 N/A
 68   4                        java.lang.String CallDb._typeFullName              N/A
Instance size: 72 bytes
Space losses: 3 bytes internal + 0 bytes external = 3 bytes total


io.shiftleft.codepropertygraph.generated.nodes.Identifier object internals:
OFF  SZ                TYPE DESCRIPTION               VALUE
  0   8                     (object header: mark)     N/A
  8   4                     (object header: class)    N/A
 12   4    overflowdb.Graph NodeRef.graph             N/A
 16   8                long NodeRef.id                N/A
 24   4   overflowdb.NodeDb NodeRef.node              N/A
 28   4                     (alignment/padding gap)   
 32   8                long Identifier.id             N/A
Instance size: 40 bytes
Space losses: 4 bytes internal + 0 bytes external = 4 bytes total


io.shiftleft.codepropertygraph.generated.nodes.IdentifierDb object internals:
OFF  SZ                                    TYPE DESCRIPTION                             VALUE
  0   8                                         (object header: mark)                   N/A
  8   4                                         (object header: class)                  N/A
 12   1                                 boolean NodeDb.dirty                            N/A
 13   3                                         (alignment/padding gap)                 
 16   4                      overflowdb.NodeRef NodeDb.ref                              N/A
 20   4                overflowdb.AdjacentNodes NodeDb.adjacentNodes                    N/A
 24   4                                     int IdentifierDb._argumentIndex             N/A
 28   4                                     int IdentifierDb._order                     N/A
 32   4                        java.lang.String IdentifierDb._argumentName              N/A
 36   4                        java.lang.String IdentifierDb._code                      N/A
 40   4                       java.lang.Integer IdentifierDb._columnNumber              N/A
 44   4   scala.collection.immutable.IndexedSeq IdentifierDb._dynamicTypeHintFullName   N/A
 48   4                       java.lang.Integer IdentifierDb._lineNumber                N/A
 52   4                        java.lang.String IdentifierDb._name                      N/A
 56   4                        java.lang.String IdentifierDb._typeFullName              N/A
 60   4                                         (object alignment gap)                  
Instance size: 64 bytes
Space losses: 3 bytes internal + 4 bytes external = 7 bytes total


io.shiftleft.codepropertygraph.generated.nodes.FieldIdentifier object internals:
OFF  SZ                TYPE DESCRIPTION               VALUE
  0   8                     (object header: mark)     N/A
  8   4                     (object header: class)    N/A
 12   4    overflowdb.Graph NodeRef.graph             N/A
 16   8                long NodeRef.id                N/A
 24   4   overflowdb.NodeDb NodeRef.node              N/A
 28   4                     (alignment/padding gap)   
 32   8                long FieldIdentifier.id        N/A
Instance size: 40 bytes
Space losses: 4 bytes internal + 0 bytes external = 4 bytes total


io.shiftleft.codepropertygraph.generated.nodes.Literal object internals:
OFF  SZ                TYPE DESCRIPTION               VALUE
  0   8                     (object header: mark)     N/A
  8   4                     (object header: class)    N/A
 12   4    overflowdb.Graph NodeRef.graph             N/A
 16   8                long NodeRef.id                N/A
 24   4   overflowdb.NodeDb NodeRef.node              N/A
 28   4                     (alignment/padding gap)   
 32   8                long Literal.id                N/A
Instance size: 40 bytes
Space losses: 4 bytes internal + 0 bytes external = 4 bytes total


io.shiftleft.codepropertygraph.generated.nodes.Block object internals:
OFF  SZ                TYPE DESCRIPTION               VALUE
  0   8                     (object header: mark)     N/A
  8   4                     (object header: class)    N/A
 12   4    overflowdb.Graph NodeRef.graph             N/A
 16   8                long NodeRef.id                N/A
 24   4   overflowdb.NodeDb NodeRef.node              N/A
 28   4                     (alignment/padding gap)   
 32   8                long Block.id                  N/A
Instance size: 40 bytes
Space losses: 4 bytes internal + 0 bytes external = 4 bytes total


io.shiftleft.codepropertygraph.generated.nodes.FieldIdentifierDb object internals:
OFF  SZ                       TYPE DESCRIPTION                        VALUE
  0   8                            (object header: mark)              N/A
  8   4                            (object header: class)             N/A
 12   1                    boolean NodeDb.dirty                       N/A
 13   3                            (alignment/padding gap)            
 16   4         overflowdb.NodeRef NodeDb.ref                         N/A
 20   4   overflowdb.AdjacentNodes NodeDb.adjacentNodes               N/A
 24   4                        int FieldIdentifierDb._argumentIndex   N/A
 28   4                        int FieldIdentifierDb._order           N/A
 32   4           java.lang.String FieldIdentifierDb._argumentName    N/A
 36   4           java.lang.String FieldIdentifierDb._canonicalName   N/A
 40   4           java.lang.String FieldIdentifierDb._code            N/A
 44   4          java.lang.Integer FieldIdentifierDb._columnNumber    N/A
 48   4          java.lang.Integer FieldIdentifierDb._lineNumber      N/A
 52   4                            (object alignment gap)             
Instance size: 56 bytes
Space losses: 3 bytes internal + 4 bytes external = 7 bytes total


io.shiftleft.codepropertygraph.generated.nodes.LiteralDb object internals:
OFF  SZ                                    TYPE DESCRIPTION                          VALUE
  0   8                                         (object header: mark)                N/A
  8   4                                         (object header: class)               N/A
 12   1                                 boolean NodeDb.dirty                         N/A
 13   3                                         (alignment/padding gap)              
 16   4                      overflowdb.NodeRef NodeDb.ref                           N/A
 20   4                overflowdb.AdjacentNodes NodeDb.adjacentNodes                 N/A
 24   4                                     int LiteralDb._argumentIndex             N/A
 28   4                                     int LiteralDb._order                     N/A
 32   4                        java.lang.String LiteralDb._argumentName              N/A
 36   4                        java.lang.String LiteralDb._code                      N/A
 40   4                       java.lang.Integer LiteralDb._columnNumber              N/A
 44   4   scala.collection.immutable.IndexedSeq LiteralDb._dynamicTypeHintFullName   N/A
 48   4                       java.lang.Integer LiteralDb._lineNumber                N/A
 52   4                        java.lang.String LiteralDb._typeFullName              N/A
Instance size: 56 bytes
Space losses: 3 bytes internal + 0 bytes external = 3 bytes total


io.shiftleft.codepropertygraph.generated.nodes.BlockDb object internals:
OFF  SZ                                    TYPE DESCRIPTION                        VALUE
  0   8                                         (object header: mark)              N/A
  8   4                                         (object header: class)             N/A
 12   1                                 boolean NodeDb.dirty                       N/A
 13   3                                         (alignment/padding gap)            
 16   4                      overflowdb.NodeRef NodeDb.ref                         N/A
 20   4                overflowdb.AdjacentNodes NodeDb.adjacentNodes               N/A
 24   4                                     int BlockDb._argumentIndex             N/A
 28   4                                     int BlockDb._order                     N/A
 32   4                        java.lang.String BlockDb._argumentName              N/A
 36   4                        java.lang.String BlockDb._code                      N/A
 40   4                       java.lang.Integer BlockDb._columnNumber              N/A
 44   4   scala.collection.immutable.IndexedSeq BlockDb._dynamicTypeHintFullName   N/A
 48   4                       java.lang.Integer BlockDb._lineNumber                N/A
 52   4                        java.lang.String BlockDb._typeFullName              N/A
Instance size: 56 bytes
Space losses: 3 bytes internal + 0 bytes external = 3 bytes total


io.shiftleft.codepropertygraph.generated.nodes.MethodParameterIn object internals:
OFF  SZ                TYPE DESCRIPTION               VALUE
  0   8                     (object header: mark)     N/A
  8   4                     (object header: class)    N/A
 12   4    overflowdb.Graph NodeRef.graph             N/A
 16   8                long NodeRef.id                N/A
 24   4   overflowdb.NodeDb NodeRef.node              N/A
 28   4                     (alignment/padding gap)   
 32   8                long MethodParameterIn.id      N/A
Instance size: 40 bytes
Space losses: 4 bytes internal + 0 bytes external = 4 bytes total

@bbrehm bbrehm changed the title add some benchmarks add some benchmarks for odbv1 memory consumption Feb 17, 2023
@bbrehm
Copy link
Contributor Author

bbrehm commented Feb 17, 2023 •
edited

A further giant fail:

io.shiftleft.codepropertygraph.generated.nodes.Block object internals:
OFF  SZ                TYPE DESCRIPTION               VALUE
  0   8                     (object header: mark)     N/A
  8   4                     (object header: class)    N/A
 12   4    overflowdb.Graph NodeRef.graph             N/A
 16   8                long NodeRef.id                N/A
 24   4   overflowdb.NodeDb NodeRef.node              N/A
 28   4                     (alignment/padding gap)   
 32   8                long Block.id                  N/A
Instance size: 40 bytes
Space losses: 4 bytes internal + 0 bytes external = 4 bytes total

our generated code adds a second id field instead of inheriting, WTF?

PS. Smells like scala/bug#4762

ml86
ml86 requested changes Feb 20, 2023
View reviewed changes
build.sbt Outdated Show resolved Hide resolved
@bbrehm bbrehm requested a review from ml86 February 20, 2023 11:26
@bbrehm bbrehm mentioned this pull request Feb 20, 2023
Merged
@mpollmeier
fixup slf4j
ee835e5 
```
SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
```
@mpollmeier
gitignore
e1007b0
@bbrehm bbrehm mentioned this pull request Feb 20, 2023
Merged
@bbrehm bbrehm merged commit eefa3fd into master Feb 20, 2023
@delete-merged-branch delete-merged-branch bot deleted the bbrehm/benchJoern branch February 20, 2023 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ml86 ml86 ml86 approved these changes

@mpollmeier mpollmeier Awaiting requested review from mpollmeier

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@bbrehm @ml86 @mpollmeier