upgrade http4k, and fix database URL to have port, implement docker f… by daviddenton · Pull Request #3206 · TechEmpower/FrameworkBenchmarks

daviddenton · 2018-01-25T22:21:01Z

…or postgres

If we could get this into final R15, that'd be awesome as it fixes the long broken DB tests

…or postgres

NateBrady23 · 2018-01-25T22:33:05Z

@daviddenton we're currently right in the middle of working on a way to integrate docker with the suite, but right now any tests using docker break our continuous benchmarking. See: #3187

daviddenton · 2018-01-25T22:53:36Z

The tests don’t use docker. There’s a just a Dockerfile and a script which would need to be manually run. Is that a problem?

daviddenton · 2018-01-25T22:55:44Z

If it is a problem, feel free to ditch the file when you merge it in, or I can do it. But I’d like it not to block the merge :)

daviddenton · 2018-01-26T00:31:50Z

Closing until R16, to ensure that we get results for R15.

NateBrady23 · 2018-01-26T15:49:46Z

hey @daviddenton sorry, I didn't dig too deep there but I saw the dockerfile and a docker run command so wanted to make sure I let you know where we stood there. We can revisit after r15 results are posted (should be very soon ®)

michaelhixson · 2018-01-26T21:58:16Z

@daviddenton For what it's worth, I tried running this PR on Server Central and I still got 0 RPS for the database tests. I'm a little confused how it passes verification but has exactly 0 requests in the full test.

daviddenton · 2018-01-26T22:00:54Z

I’m presuming that the change is good though for that environment? It does need the port to connect - or is there something unusual about that env?

daviddenton · 2018-01-26T22:02:07Z

And obviously it does work with the dockerised env that I included in the dockerfile. :)

daviddenton · 2018-01-26T22:04:46Z

Could it be the benchmark JSON - is the name of the database correct? Not sure if it’s case sensitive..

EDIT: Following the above suspicion, I've updated the name of the database - if anyone would like to test it again against SC that'd be awesome.

…red upgrade to http4k

michaelhixson · 2018-01-28T00:59:43Z

@daviddenton It's like it stops responding to requests after wrk starts the load test. It's ok at first, and it returns the right content from all the db-related endpoints so we know it's reaching the database, but then it falls over when wrk starts making a ton of requests.

I got it to respond with an error page when did a curl while the load test was running:

curl localhost:9000/queries?queries=1
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 500 Server Error</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /queries. Reason:
<pre>    Server Error</pre></p><h3>Caused by:</h3><pre>java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30000ms.
        at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:666)
        at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:182)
        at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:147)
        at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:85)
        at Database.withConnection(Database.kt:40)
        at WorldRoutes$multipleRoute$1.invoke(WorldRoutes.kt:44)
        at WorldRoutes$multipleRoute$1.invoke(WorldRoutes.kt:23)
        at Http4kBenchmarkServer$headers$1$1.invoke(Http4kBenchmarkServer.kt:14)
        at Http4kBenchmarkServer$headers$1$1.invoke(Http4kBenchmarkServer.kt:9)
        at org.http4k.core.Http4kKt$then$2.invoke(Http4k.kt:17)
        at org.http4k.core.Http4kKt$then$2.invoke(Http4k.kt)
        at org.http4k.routing.TemplateRoutingHttpHandler$match$1.invoke(internal.kt:72)
        at org.http4k.routing.TemplateRoutingHttpHandler$match$1.invoke(internal.kt:67)
        at org.http4k.routing.RoutingKt$routes$2.invoke(routing.kt:26)
        at org.http4k.routing.RoutingKt$routes$2.invoke(routing.kt:25)
        at org.http4k.servlet.HttpHandlerServlet.service(servlet.kt:19)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:791)
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
        at org.eclipse.jetty.server.Server.handle(Server.java:561)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:334)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:104)
        at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:243)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:679)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:597)
        at java.lang.Thread.run(Thread.java:748)
</pre>
<hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.7.v20170914</a><hr/>

</body>
</html>

daviddenton · 2018-01-28T09:36:04Z

@michaelhixson found the problem - it was that we were not closing off resources properly, so it basically eats up all the connections and then chokes. Thanks a miiion for your help in getting this resolved. 👏 ⭐️ 😄

I'll reopen this as soon as R15 is done. :)

…size

daviddenton · 2018-02-15T09:42:42Z

Reopening to get into R16 previews. Keep up the good work people! 😄

NateBrady23 · 2018-02-20T00:17:38Z

Hi @daviddenton can you check out #3308 just make sure the benchmark_config.json structure is correct. Also, could you add the framework to .travis.yml in the appropriate place. Seems like that was missed before.

daviddenton · 2018-02-20T09:29:31Z

@nbrady-techempower yep - that all looks good. Thanks for fixing that JSON thing - I always wondered why we got that warning and the split of the tests JSON array was too subtle for me to spot.

On that subject, I'd actually like to ditch the "default" completely, since we don't really have a "default" - we have several pluggable backends (currently the default is actually the jetty implementation). Is this possible? If it isn't, then please forget I said anything and just merge this :)

NateBrady23 · 2018-02-20T14:32:39Z

@daviddenton The default section is more for the test suite and right now it's required as part of the setup.

Now that travis is picking up the test, it looks like it's not passing there.

michaelhixson · 2018-02-20T18:08:38Z

@daviddenton Could you remove the local testing scripts like run_db.sh and benchmark_server.sh? I understand that scripts like that can be useful to the maintainer, but I don't think this repository is a good home for them.

* Delete benchmark_all.sh * Delete benchmark_server.sh * Delete killbenchmark.sh * Delete run_db.sh * Delete Dockerfile

daviddenton · 2018-02-20T19:07:13Z

@michaelhixson no problem. tidied as requested.

daviddenton added 2 commits January 25, 2018 22:20

upgrade http4k, and fix database URL to have port, implement docker f…

00d1f8c

…or postgres

remove netty again :(

777f41b

daviddenton closed this Jan 26, 2018

fix name of database

b423470

daviddenton reopened this Jan 27, 2018

daviddenton closed this Jan 27, 2018

added apache and sunhttp servers. not plugged in yet because of requi…

4cfa186

…red upgrade to http4k

daviddenton added 2 commits January 28, 2018 09:14

fix benchmark database routes

95de981

further fixes to database

9792213

daviddenton reopened this Jan 28, 2018

daviddenton closed this Jan 28, 2018

daviddenton added 7 commits January 28, 2018 09:41

Tidy extension method

2e77743

adding scripts for local benchmarking, also upped db connection pool …

e56abb8

…size

added results generation to scripts

1f53a43

more results mangling

9195767

more benchmarking script

172dd30

remove quiet

1d44f24

further tweaks to local benchmarker

2734449

daviddenton reopened this Feb 15, 2018

added apache, netty and sunhttp server backends

ccae926

daviddenton added 2 commits February 20, 2018 09:19

adding http4k

cc6c639

made benchmark profiles only a single entry in the tests list.

153fa7f

daviddenton added 3 commits February 20, 2018 15:49

tweak modules for gradle

57edb30

upgrade http4k

1a095d2

fix sunhttp script

8888eeb

Remove testing scripts

25105be

* Delete benchmark_all.sh * Delete benchmark_server.sh * Delete killbenchmark.sh * Delete run_db.sh * Delete Dockerfile

NateBrady23 merged commit d72b487 into TechEmpower:master Feb 21, 2018

Conversation

daviddenton commented Jan 25, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NateBrady23 commented Jan 25, 2018

Uh oh!

daviddenton commented Jan 25, 2018

Uh oh!

daviddenton commented Jan 25, 2018

Uh oh!

daviddenton commented Jan 26, 2018

Uh oh!

NateBrady23 commented Jan 26, 2018

Uh oh!

michaelhixson commented Jan 26, 2018

Uh oh!

daviddenton commented Jan 26, 2018

Uh oh!

daviddenton commented Jan 26, 2018

Uh oh!

daviddenton commented Jan 26, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

michaelhixson commented Jan 28, 2018

Uh oh!

daviddenton commented Jan 28, 2018

Uh oh!

daviddenton commented Feb 15, 2018

Uh oh!

NateBrady23 commented Feb 20, 2018

Uh oh!

daviddenton commented Feb 20, 2018

Uh oh!

NateBrady23 commented Feb 20, 2018

Uh oh!

michaelhixson commented Feb 20, 2018

Uh oh!

daviddenton commented Feb 20, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

daviddenton commented Jan 25, 2018 •

edited

Loading

daviddenton commented Jan 26, 2018 •

edited

Loading