Permalink
Switch branches/tags
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
138 lines (111 sloc) 6 KB

In this module we study the effect of Spring and Spring Boot on startup time using the webflux and functional webflux programming models.

Results:

Benchmark           (sample)  Mode  Cnt  Score   Error  Units Webstack
MainBenchmark.main      demo    ss   10  1.124 ± 0.050   s/op mvc
MainBenchmark.main      flux    ss   10  1.023 ± 0.080   s/op webflux
MainBenchmark.main      actr    ss   10  1.548 ± 0.049   s/op mvc
MainBenchmark.main      demo    ss   10  0.993 ± 0.027   s/op flux.fn
MainBenchmark.main      actr    ss   10  1.397 ± 0.068   s/op webflux

Summary: webflux.fn and @webflux are comparable, and both are faster on Netty than MVC on Tomcat (>~10%).

The "actr" sample is with Actuators. The others are a single HTTP endpoint.

Minimal Rest Controller

How fast can Spring serve a simple rest endpoint? With full webflux features:

Benchmark            (sample)  Mode  Cnt  Score   Error  Units Classes
MainBenchmark.main       jlog    ss   10  0.933 ± 0.065   s/op 4367
MainBenchmark.main       demo    ss   10  1.081 ± 0.075   s/op 5779
MiniBenchmark.boot       jlog    ss   10  0.486 ± 0.020   s/op 2974
MiniBenchmark.boot       demo    ss   10  0.579 ± 0.041   s/op 4138
MiniBenchmark.mini       jlog    ss   10  0.420 ± 0.011   s/op 2351
MiniBenchmark.mini       demo    ss   10  0.538 ± 0.009   s/op 3138
MiniBenchmark.micro      jlog    ss   10  0.186 ± 0.006   s/op 1371
MiniBenchmark.micro      demo    ss   10  0.288 ± 0.006   s/op 2112
pubchart?oid=976086548&format=image
Figure 1. Number of Classes vs. Startup Time

Notes:

  • The MainBenchmark.main(demo) is full Boot + Webflux + autoconfiguration.

  • The boot samples use Spring Boot but no autoconfiguration.

  • The jlog samples exclude logback as well as Hibernate Validator and Jackson.

  • The mini samples do not use Spring Boot (just @EnableWebFlux).

  • The micro samples do not use @EnableWebflux, just a manual route registration.

The mini jlog sample ran in about 52MB memory (16 heap, 36 non-heap). The micro jlog sample ran in 44MB (14 heap, 30 non-heap). Non-heap is really what matters for these smaller apps.

Open J9

Here are the results (pretty impressive):

Benchmark            (sample)  Mode  Cnt  Score   Error  Units Classes
MainBenchmark.main       jlog    ss   10  0.709 ± 0.034   s/op 4536
MainBenchmark.main       demo    ss   10  0.939 ± 0.027   s/op 5954
MiniBenchmark.boot       jlog    ss   10  0.406 ± 0.085   s/op 3090
MiniBenchmark.boot       demo    ss   10  0.505 ± 0.035   s/op 4314
MiniBenchmark.mini       jlog    ss   10  0.340 ± 0.018   s/op 2427
MiniBenchmark.mini       demo    ss   10  0.432 ± 0.019   s/op 3256
MiniBenchmark.micro      jlog    ss   10  0.152 ± 0.045   s/op 1436
MiniBenchmark.micro      demo    ss   10  0.204 ± 0.019   s/op 2238

The results are quite variable with Open J9. Those above are the best runs - the median results were the same, but the average was longer when the tail hit.

Lazy Beans

There’s a bean factory post processor in Spring Boot issue 9685 that makes all beans lazy by default. It’s quite interesting to see what happens if we add that to our mini applications:

Benchmark            (sample)  Mode  Cnt  Score   Error  Units Classes
MainBenchmark.main       jlog    ss   10  0.729 ± 0.059   s/op 3949
MainBenchmark.main       demo    ss   10  0.887 ± 0.012   s/op 5311
MiniBenchmark.boot       jlog    ss   10  0.356 ± 0.020   s/op 2238
MiniBenchmark.boot       demo    ss   10  0.401 ± 0.022   s/op 2644
MiniBenchmark.mini       jlog    ss   10  0.278 ± 0.015   s/op 1735
MiniBenchmark.mini       demo    ss   10  0.327 ± 0.014   s/op 2132

It’s cheating a bit, because those beans end up being initialized on the first HTTP request, but probably most of them are not needed in this sample, so they could stay uninitialized and no-one would care. We’d need more logic / conditions in @EnableWebFlux to capitalize on it without cheating

Open J9 and Lazy Beans

Benchmark            (sample)  Mode  Cnt  Score   Error  Units Classes
MainBenchmark.main       jlog    ss   10  0.618 ± 0.020   s/op 4097
MainBenchmark.main       demo    ss   10  0.827 ± 0.027   s/op 5524
MiniBenchmark.boot       jlog    ss   10  0.364 ± 0.007   s/op 2744
MiniBenchmark.boot       demo    ss   10  0.483 ± 0.015   s/op 3576
MiniBenchmark.mini       jlog    ss   10  0.214 ± 0.004   s/op 1826
MiniBenchmark.mini       demo    ss   10  0.255 ± 0.007   s/op 2255

Strangely the "boot" sample doesn’t benefit at all from the combination (it’s not faster with J9 once the lazy beans are on).

Spring Boot 2.1.0

Spring 5.1 has some performance improvements.

Benchmark            (sample)  Mode  Cnt  Score   Error  Units
MainBenchmark.main       empt    ss   10  0.596 ± 0.010   s/op
MainBenchmark.main       jlog    ss   10  0.770 ± 0.022   s/op
MainBenchmark.main       demo    ss   10  0.939 ± 0.037   s/op
MainBenchmark.main       actr    ss   10  1.270 ± 0.025   s/op
MainBenchmark.main       jdbc    ss   10  1.057 ± 0.018   s/op
MainBenchmark.main       actj    ss   10  1.449 ± 0.059   s/op
MiniBenchmark.boot       jlog    ss   10  0.458 ± 0.007   s/op
MiniBenchmark.boot       demo    ss   10  0.572 ± 0.014   s/op
MiniBenchmark.micro      jlog    ss   10  0.199 ± 0.013   s/op
MiniBenchmark.micro      demo    ss   10  0.304 ± 0.003   s/op
MiniBenchmark.mini       jlog    ss   10  0.412 ± 0.022   s/op
MiniBenchmark.mini       demo    ss   10  0.529 ± 0.013   s/op

On J9:

MainBenchmark.main       empt    ss   10  0.455 ± 0.053   s/op
MainBenchmark.main       jlog    ss   10  0.644 ± 0.038   s/op
MainBenchmark.main       demo    ss   10  0.851 ± 0.026   s/op
MainBenchmark.main       actr    ss   10  1.231 ± 0.048   s/op
MainBenchmark.main       jdbc    ss   10  0.973 ± 0.040   s/op
MainBenchmark.main       actj    ss   10  1.389 ± 0.040   s/op