Investigate our performance on https://github.com/the-benchmarker/web-frameworks

[helenmasters]

It has been reported in our Slack that we are not performing well when running this benchmark, in comparison to other Swift frameworks.

As we are not familiar with this benchmark we will take a look.

[pushkarnk]

The Kitura implementation doesn't seem to include the NIO mode. May be we should add that too.

[helenmasters]

David Jones and I investigated yesterday and he immediately noticed that Kitura is running inside an Ubuntu 14.04 Docker container, which is a performance penalty compared to running on 16.04 or 18.04 (related to an old + poorly performing libicu on that old distro)

Unfortunately, our generated projects specify ibmcom/swift-ubuntu:5.0.1 rather than using the official swift:5.0.1 image. The former is based on 14.04, which is something that David has raised that we need to address. Long-term we believe we will be moving Kitura projects to use the official Swift images.

We can change
FROM: ibmcom/swift-ubuntu:5.0.1

to

RUN apt-get update && apt-get install -y openssl-dev libcurl4-openssl-dev

after which we are ~35% faster.

Here's a set of results which David ran locally (bare-metal Ubuntu 16.04 server, he modified the client/docker commands to run on 8 CPUs equivalent to the benchmark author's setup):

Language (Runtime)	Framework (Middleware)	Requests / s	Throughput
swift (4.2)	[perfect] (3.1)	142786.33	134.38 MB
swift (5.0)	[kitura-1804] (2.7)	100213.67	186.64 MB
swift (5.0)	[kitura-1804] (2.6)	99507.67	185.31 MB
swift (5.0)	[vapor] (3.3)	93971.33	157.05 MB
swift (5.0)	[kitura-1604] (2.7)	89525.33	166.74 MB
swift (4.2)	[kitura-1604] (2.7)	80761.00	150.42 MB
swift (5.0)	[kitura-1404] (2.6)	74396.33	138.55 MB
swift (5.0)	[kitura-nio-1804] (2.7)	73765.33	139.60 MB
swift (5.0)	[kitura-1404] (2.7)	73496.33	136.87 MB
swift (5.0)	[kitura-nio] (2.7)	60809.00	115.02 MB

... we also don't see the regression between Kitura 2.6 and Kitura 2.7. They are about equal in performance.

David has submitted a PR to update the Kitura dockerfiles to 18.04:
FYI, the-benchmarker/web-frameworks#1346

At this point we're going to close off this issue and have raised two other separate issues covering two other items which we noticed whilst running this benchmark:

• Investigate poor performance of empty POST with the web-frameworks benchmark #1452
• Poor performance with the web-frameworks benchmark Kitura-NIO#207

[waghanza]

FYI, I am working on running this on production-ready IaaS (instead of docker) on the-benchmarker/web-frameworks#632