New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: http layer - cache couchdb version #56

Closed
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
3 participants
@robertkowalski
Contributor

robertkowalski commented May 29, 2015

I have taken a look at our http stack with erlang:trace and flamegraphs.
It turned out that we spend a lot of our time for every request in the
function couch_server:get_version which simply gets the current CouchDB
version. [1]

http://robert-kowalski.de/assets/data/2015-05-29-erlang-perf/flame--unpatched-second-request.png

So I tried to memoize it, with pretty good looking flamegraph. I got
good suggestions from Alex and the flamegraph also looked nice [2].

http://robert-kowalski.de/assets/data/2015-05-29-erlang-perf/flame--patched-second-request.png

I re-benchmarked the unclustered interface, but (almost?) all our
requests are accessing this function so it should also be beneficial
for chttpd & friends.

Part of the benchmarks was also to run Apache ab and later to run
Siege [3] on it:

All benchmarks (also the flamegraph creation) were run using this test
protocol:

  1. turn off all auto starting apps, especially dropbox & co
  2. run make with path
  3. reboot
  4. wait 60secs
  5. boot cluster and wait until successful connected
  6. wait 60secs
  7. run test

Siege:

10000 requests, concurrency 120, reading one doc, unclustered
interface:

old version, overall run times:
test 1: 4.82
test 2: 4.78

patched version, overall run times:
test 1: 4.39
test 2: 4.37

mean difference:

(4.82 + 4.78) / (4.39 + 4.37) * 100

patched version is 9.5% faster

[1] http://robert-kowalski.de/assets/data/2015-05-29-erlang-perf/flame--unpatched-second-request.png
[2] http://robert-kowalski.de/assets/data/2015-05-29-erlang-perf/flame--patched-second-request.png
[3] http://robert-kowalski.de/assets/data/2015-05-29-erlang-perf/results-siege.txt

@banjiewen

This comment has been minimized.

Contributor

banjiewen commented May 29, 2015

Using the application environment would be faster than the cache implemented here. Plus it wouldn't represent a potential failure due to mailbox overflow.

You would, of course, need to hack the version in to the app env somehow - I don't think it's there already.

@kxepal

This comment has been minimized.

Member

kxepal commented May 29, 2015

Alternative solution:

diff --git a/src/couch_server.erl b/src/couch_server.erl
index 228edc3..48046c6 100644
--- a/src/couch_server.erl
+++ b/src/couch_server.erl
@@ -45,12 +45,9 @@ dev_start() ->
     couch:start().

 get_version() ->
-    Apps = application:loaded_applications(),
-    case lists:keysearch(couch, 1, Apps) of
-    {value, {_, _, Vsn}} ->
-        Vsn;
-    false ->
-        "0.0.0"
+    case application:get_key(couch, vsn) of
+        {ok, Vsn} -> Vsn;
+        undefined -> "0.0.0"
     end.
 get_version(short) ->
   %% strip git hash from version string

No need to fetch all the apps from the ets table and filter them twice (by load status and by name) since we already know our app name and it's already loaded.

@robertkowalski robertkowalski force-pushed the robertkowalski:increase-perf branch from 4f205dc to cf6fc0c May 29, 2015

@robertkowalski

This comment has been minimized.

Contributor

robertkowalski commented May 29, 2015

thanks for all the input and suggestions! updated!

@kxepal

This comment has been minimized.

Member

kxepal commented May 29, 2015

@robertkowalski if you update commit message to make it actual and shorter (no need to explain there about flamegraphs and benchmark, but about what it does and why it makes things better) then +1 from me (:

@robertkowalski robertkowalski force-pushed the robertkowalski:increase-perf branch from cf6fc0c to bd062c7 May 29, 2015

perf: http layer - speedup couchdb version lookup
It turned out that we spend a lot of our time for every request in the
function couch_server:get_version which simply gets the current CouchDB
version.

Patched version is ~9.5% faster for a simple /get on a document.

(Almost?) All our of request-handlers are accessing this function so it
should be beneficial for large areas of the http layer.

We achieve this by not using an expensive ets:filter function
internally by avoiding `loaded_applications` but instead using a
simple ets:lookup internally by changing to `get_key`.

@robertkowalski robertkowalski force-pushed the robertkowalski:increase-perf branch from bd062c7 to 6423fde May 29, 2015

@robertkowalski

This comment has been minimized.

Contributor

robertkowalski commented Jun 2, 2015

Cool, merged with your recommendations I just got on IRC.

@robertkowalski robertkowalski deleted the robertkowalski:increase-perf branch Jun 2, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment