Skip to content

Latest commit

 

History

History
547 lines (454 loc) · 27 KB

performance.org

File metadata and controls

547 lines (454 loc) · 27 KB
Raw

This benchmark report consists of two main sections. First, we conducted a short benchmark using real world data generated by memacs-imap.py. This showed some interesting things.

A second benchmark was conducted afterward with artificial generated benchmark files that can easily be generated and checked at your side too. This benchmark is somewhat more detailed.

Real World Data Test using Data generated by memacs-imap.py

Test Files

All test files do have the same header line:

* description  :tags:

… followed by 19044 entries with the format:

Test file 1: «no_drawer»

** <2011-12-31 Sat 10:49:52> [[mailto:Karl.Voit@example.com][Karl Voit]]: Re: Hoffe die mail kommt an

Test file 2: «selfdefined_drawer»

** <2011-12-31 Sat 10:49:52> [[mailto:Karl.Voit@example.com][Karl Voit]]: Re: Hoffe die mail kommt an
  :MEMACSDRAWER:
  :TO:         Armin Wieser <armin.wieser@example.com>
  :ID:         <2011-12-31T10-49-17@example.com>
  :END:

Test file 3: «properties_drawer»

** <2011-12-31 Sat 10:49:52> [[mailto:Karl.Voit@example.com][Karl Voit]]: Re: Hoffe die mail kommt an
  :PROPERTIES:
  :TO:         Armin Wieser <armin.wieser@example.com>
  :ID:         <2011-12-31T10-49-17@example.com>
  :END:

Testcases

  • open open file in Emacs with org-mode
  • expand1 expand all ~19050 entries
  • collapse collapse all ~19050 entries
  • expand2 second expand (after expand + collapse)

Results

Time is measured three times (by hand) in seconds:

TestfileTestcasetime1time2time3meancomment
no_draweropen1111.0ok
no_drawerexpand11111.0ok
no_drawercollapse1111.0ok
no_drawerexpand21111.0ok
selfdefined_draweropen1111.0very fast although a drawer exists
selfdefined_drawerexpand170908080.0better than «propreties_drawer» but still not fast
selfdefined_drawercollapse1111.0ok
selfdefined_drawerexpand270808076.7nearly same as «propreties_drawer»
properties_draweropen190166205187.0slow
properties_drawerexpand1400430450426.7slow
properties_drawercollapse1111.0ok
properties_drawerexpand270908581.7slow

Conclusions

  • Best performance: only without any drawers at all
  • Acceptable performance when using another drawer than :PROPERTIES:
  • Worst performance when using :PROPERTIES: drawer

Test Machine

  • Lenovo X301
  • Samsung SSD
  • 4GB Ram
  • CPU Intel(R) Core(TM)2 Duo CPU U9400 @1.40GHz
  • GNU/Linux: Ubuntu 11.10 oneiric
  • Org-mode version 7.8.02
  • GNU Emacs 23.3.1 (x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
    • of 2011-08-14 on allspice
    • modified by Debian
  • some programs are running in parallel - will make no huge difference

Benchmarking with Artificial Generated Files

Test Files

Each test file consists of one header line and 20,000 entries in the second layer. Following sections shows the start of each test file up to the second item in the second layer.

The test file «benchmarkfile_properties.org» was generated by the Python script described in the section below «Generating Script for the Benchmark Files». All other test files were derived from that file by mainly search and replace corresponding parts.

  • «benchmarkfile_nodrawer.org»
  • «benchmarkfile_owndrawer.org»
    • search and replace of «PROPERTIES» with «MEMACSDRAWER»
  • «benchmarkfile_properties_noemail.org»
    • search and replace of email like structures with normal text
  • «benchmarkfile_properties_notags.org»
    • removing the tags «:tag1:tag2:» and «:tag3:tag4:»

Test file «benchmarkfile_properties.org»

This is a test file which uses many Org-mode features such as tags, time stamps, PROPERTIES-drawer, links to email addresses, links to web pages, and links to files.

*** multi word description                                      :tag1:tag2:
**** <2011-12-31 Sat 10:49:52> [[mailto:foo.bar@example.com][Foo Bar]]: This is the subject line :tag3:tag4:
:PROPERTIES:
:TO: Foo Bar <foo.bar@example.com>
:ID: <myid_1@example.com>
:END:
**** <2011-12-31 Sat 10:49:52> [[mailto:foo.bar@example.com][Foo Bar]]: This is the subject line :tag3:tag4:
:PROPERTIES:
:TO: Foo Bar <foo.bar@example.com>
:ID: <myid_2@example.com>
:END:

Test file «benchmarkfile_nodrawer.org»

Intent: what’s the performance difference when using no drawer at all?

Generated by grepping only lines starting with an asterisk:

egrep "^\*" benchmarkfile_properties.org > benchmarkfile_nodrawer.org

** multi word description                                      :tag1:tag2:
*** <2011-12-31 Sat 10:49:52> ([[http://example.com/URL1][link]]) [[mailto:foo.bar@example.com][Foo Bar]]: [[file:/home/user/path1/file1][This is the subject line]] :tag3:tag4:
*** <2011-12-31 Sat 10:49:52> ([[http://example.com/URL2][link]]) [[mailto:foo.bar@example.com][Foo Bar]]: [[file:/home/user/path1/file2][This is the subject line]] :tag3:tag4:

Test file «benchmarkfile_owndrawer.org»

Intent: what’s the performance difference when using another drawer than PROPERTIES?

Generated by replacing «PROPERTIES» with «MEMACSDRAWER»:

sed 's/PROPERTIES/MEMACSDRAWER/' < benchmarkfile_properties.org > benchmarkfile_owndrawer.org

*** multi word description                                      :tag1:tag2:
**** <2011-12-31 Sat 10:49:52> ([[http://example.com/URL1][link]]) [[mailto:foo.bar@example.com][Foo Bar]]: [[file:/home/user/path1/file1][This is the subject line]] :tag3:tag4:
:MEMACSDRAWER:
:TO: Foo Bar <foo.bar@example.com>
:ID: <myid_1@example.com>
:END:
**** <2011-12-31 Sat 10:49:52> ([[http://example.com/URL2][link]]) [[mailto:foo.bar@example.com][Foo Bar]]: [[file:/home/user/path1/file2][This is the subject line]] :tag3:tag4:
:MEMACSDRAWER:
:TO: Foo Bar <foo.bar@example.com>
:ID: <myid_2@example.com>
:END:

Test file «benchmarkfile_properties_noemail.org»

Intent: what effect do have links to email addresses?

Generated by removing the mailto-link:

sed 's/\[\[mailto:foo.bar@example.com\]\[Foo Bar\]\]/Foo Bar/' < benchmarkfile_properties.org > benchmarkfile_noemail.org

*** multi word description                                      :tag1:tag2:
**** <2011-12-31 Sat 10:49:52> ([[http://example.com/URL1][link]]) Foo Bar: [[file:/home/user/path1/file1][This is the subject line]] :tag3:tag4:
:PROPERTIES:
:TO: Foo Bar <foo.bar@example.com>
:ID: <myid_1@example.com>
:END:
**** <2011-12-31 Sat 10:49:52> ([[http://example.com/URL2][link]]) Foo Bar: [[file:/home/user/path1/file2][This is the subject line]] :tag3:tag4:
:PROPERTIES:
:TO: Foo Bar <foo.bar@example.com>
:ID: <myid_2@example.com>
:END:

Test file «benchmarkfile_properties_notags.org»

Intent: what effect do have tags?

Generated by removing the tags «:tag1:tag2:» and «:tag3:tag4:»:

sed 's/\(:tag1:tag2:\)\|\(:tag3:tag4:\)//' < benchmarkfile_properties.org > benchmarkfile_notags.org

*** multi word description  
**** <2011-12-31 Sat 10:49:52> ([[http://example.com/URL1][link]]) [[mailto:foo.bar@example.com][Foo Bar]]: [[file:/home/user/path1/file1][This is the subject line]]  
:PROPERTIES:
:TO: Foo Bar <foo.bar@example.com>
:ID: <myid_1@example.com>
:END:
**** <2011-12-31 Sat 10:49:52> ([[http://example.com/URL2][link]]) [[mailto:foo.bar@example.com][Foo Bar]]: [[file:/home/user/path1/file2][This is the subject line]]  
:PROPERTIES:
:TO: Foo Bar <foo.bar@example.com>
:ID: <myid_2@example.com>
:END:

Test file «benchmarkfile_properties_notimestamp.org»

Intent: what effect do have time stamps in headings?

Generated by removing the time stamp string «<2011-12-31 Sat 10:49:52>»:

sed 's/<2011-12-31 Sat 10:49:52> //' < benchmarkfile_properties.org > benchmarkfile_notimestamp.org

*** multi word description                                      :tag1:tag2:
**** ([[http://example.com/URL1][link]]) [[mailto:foo.bar@example.com][Foo Bar]]: [[file:/home/user/path1/file1][This is the subject line]] :tag3:tag4:
:PROPERTIES:
:TO: Foo Bar <foo.bar@example.com>
:ID: <myid_1@example.com>
:END:
**** ([[http://example.com/URL2][link]]) [[mailto:foo.bar@example.com][Foo Bar]]: [[file:/home/user/path1/file2][This is the subject line]] :tag3:tag4:
:PROPERTIES:
:TO: Foo Bar <foo.bar@example.com>
:ID: <myid_2@example.com>
:END:

Test file «benchmarkfile_properties_nofilelink.org»

Intent: what effect do have links to files?

sed 's/\[\[file:\/home\/user\/path1\/file\(.*\)\]\[\(This is the subject line\)\]\]/\1 \2/' < benchmarkfile_properties.org > benchmarkfile_nofilelink.org

*** multi word description                                      :tag1:tag2:
**** <2011-12-31 Sat 10:49:52> ([[http://example.com/URL1][link]]) [[mailto:foo.bar@example.com][Foo Bar]]: 1 This is the subject line :tag3:tag4:
:PROPERTIES:
:TO: Foo Bar <foo.bar@example.com>
:ID: <myid_1@example.com>
:END:
**** <2011-12-31 Sat 10:49:52> ([[http://example.com/URL2][link]]) [[mailto:foo.bar@example.com][Foo Bar]]: 2 This is the subject line :tag3:tag4:
:PROPERTIES:
:TO: Foo Bar <foo.bar@example.com>
:ID: <myid_2@example.com>
:END:

Test file «benchmarkfile_properties_nohreflink.org»

Intent: what effect do have links to web pages?

sed 's/\[\[http:\/\/example.com\/URL\(.*\)\]\[link\]\]/link \1/' < benchmarkfile_properties.org > benchmarkfile_nohreflink.org

*** multi word description                                      :tag1:tag2:
**** <2011-12-31 Sat 10:49:52> (link 1) [[mailto:foo.bar@example.com][Foo Bar]]: [[file:/home/user/path1/file1][This is the subject line]] :tag3:tag4:
:PROPERTIES:
:TO: Foo Bar <foo.bar@example.com>
:ID: <myid_1@example.com>
:END:
**** <2011-12-31 Sat 10:49:52> (link 2) [[mailto:foo.bar@example.com][Foo Bar]]: [[file:/home/user/path1/file2][This is the subject line]] :tag3:tag4:
:PROPERTIES:
:TO: Foo Bar <foo.bar@example.com>
:ID: <myid_2@example.com>
:END:

Test file «benchmarkfile_properties_simpleID.org»

Intent: what effect do have non trivial IDs compared to simple IDs?

sed 's/<myid_\(.*\)@example.com>/myid_\1/' < benchmarkfile_properties.org > benchmarkfile_simpleID.org

*** multi word description                                      :tag1:tag2:
**** <2011-12-31 Sat 10:49:52> ([[http://example.com/URL1][link]]) [[mailto:foo.bar@example.com][Foo Bar]]: [[file:/home/user/path1/file1][This is the subject line]] :tag3:tag4:
:PROPERTIES:
:TO: Foo Bar <foo.bar@example.com>
:ID: myid_1
:END:
**** <2011-12-31 Sat 10:49:52> ([[http://example.com/URL2][link]]) [[mailto:foo.bar@example.com][Foo Bar]]: [[file:/home/user/path1/file2][This is the subject line]] :tag3:tag4:
:PROPERTIES:
:TO: Foo Bar <foo.bar@example.com>
:ID: myid_2
:END:

Testcases

  • open open file in Emacs with org-mode
  • expand1 expand all ~19050 entries
  • collapse collapse all ~19050 entries
  • expand2 second expand (after expand + collapse)

Results

Time is measured three times (by hand) in seconds:

Testfilemachine 1machine 2meancomment
properties174.6174.6
nodrawer4.4.0
owndrawer14.714.7
noemail202.3202.3
notags208.3208.3
notimestamp209.6209.6
nofilelink209.4209.4
nohreflink210.9210.9
simpleID216.3216.3

Conclusions

FIXXME

Detailed Results

Time is measured three times (by hand) in seconds.

Test Machine 1: Lenovo X301

  • Samsung SSD
  • 4GB Ram
  • CPU Intel(R) Core(TM)2 Duo CPU U9400 @1.40GHz
  • GNU/Linux: Ubuntu 11.10 oneiric
  • Org-mode version 7.8.02
  • GNU Emacs 23.3.1 (x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
    • of 2011-08-14 on allspice
    • modified by Debian
  • some programs are running in parallel - will make no huge difference
filecasetime 1time 2time 3mean
propertiesopen35353635.3
propertiesexpand1130140120130.0
propertiescollapse1111.0
propertiesexpand29888.3
sum174.6
filecasetime 1time 2time 3mean
nodraweropen1111.0
nodrawerexpand11111.0
nodrawercollapse1111.0
nodrawerexpand21111.0
sum4.0
filecasetime 1time 2time 3mean
owndraweropen1111.0
owndrawerexpand17887.7
owndrawercollapse1111.0
owndrawerexpand21775.0
sum14.7
filecasetime 1time 2time 3mean
properties_noemailopen35363535.3
properties_noemailexpand1160160155158.3
properties_noemailcollapse1111.0
properties_noemailexpand28877.7
sum202.3
filecasetime 1time 2time 3mean
properties_notagsopen36333434.3
properties_notagsexpand1162167168165.7
properties_notagscollapse1111.0
properties_notagsexpand27877.3
sum208.3
filecasetime 1time 2time 3mean
properties_notimestampopen34343534.3
properties_notimestampexpand1167168167167.3
properties_notimestampcollapse1111.0
properties_notimestampexpand27777.0
sum209.6
filecasetime 1time 2time 3mean
properties_nofilelinkopen32333332.7
properties_nofilelinkexpand1167170167168.0
properties_nofilelinkcollapse1111.0
properties_nofilelinkexpand27887.7
sum209.4
filecasetime 1time 2time 3mean
properties_nohreflinkopen36343334.3
properties_nohreflinkexpand1168169168168.3
properties_nohreflinkcollapse1111.0
properties_nohreflinkexpand27787.3
sum210.9
filecasetime 1time 2time 3mean
properties_simpleIDopen35363435.0
properties_simpleIDexpand1172178170173.3
properties_simpleIDcollapse1111.0
properties_simpleIDexpand27777.0
sum216.3

Test Machine 2: FIXXME

  • FIXXME:Harddisk
  • FIXXME: GB Ram
  • FIXXME: CPU GHz
  • FIXXME: OS
  • FIXXME: Org-mode version
  • FIXXME: GNU Emacs version
filecasetime 1time 2time 3mean
propertiesopen0.0
propertiesexpand10.0
propertiescollapse0.0
propertiesexpand20.0
sum0.
filecasetime 1time 2time 3mean
nodraweropen0.0
nodrawerexpand10.0
nodrawercollapse0.0
nodrawerexpand20.0
sum0.
filecasetime 1time 2time 3mean
owndraweropen0.0
owndrawerexpand10.0
owndrawercollapse0.0
owndrawerexpand20.0
sum0.
filecasetime 1time 2time 3mean
properties_noemailopen0.0
properties_noemailexpand10.0
properties_noemailcollapse0.0
properties_noemailexpand20.0
sum0.
filecasetime 1time 2time 3mean
properties_notagsopen0.0
properties_notagsexpand10.0
properties_notagscollapse0.0
properties_notagsexpand20.0
sum0.
filecasetime 1time 2time 3mean
properties_notimestampopen0.0
properties_notimestampexpand10.0
properties_notimestampcollapse0.0
properties_notimestampexpand20.0
sum0.
filecasetime 1time 2time 3mean
properties_nofilelinkopen0.0
properties_nofilelinkexpand10.0
properties_nofilelinkcollapse0.0
properties_nofilelinkexpand20.0
sum0.
filecasetime 1time 2time 3mean
properties_nohreflinkopen0.0
properties_nohreflinkexpand10.0
properties_nohreflinkcollapse0.0
properties_nohreflinkexpand20.0
sum0.
filecasetime 1time 2time 3mean
properties_simpleIDopen0.0
properties_simpleIDexpand10.0
properties_simpleIDcollapse0.0
properties_simpleIDexpand20.0
sum0.

Generating Script for the Benchmark Files

«benchmarkfile_properties.org» is exactly related to «benchmarkfile.org» which was generated by following script:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Time-stamp: <2012-01-05 16:26:04 vk>

import sys
import os
import time

OUTPUTFILE="benchmarkfile_properties.org"
NUM_OF_ENTRIES=20000

if __name__ == "__main__":
    output = open(OUTPUTFILE, 'w')
    output.write("* multi word description  :tag1:tag2:\n")
    for number in range(1, NUM_OF_ENTRIES):
        output.write("** <2011-12-31 Sat 10:49:52> ([[http://example.com/URL" + \
                         str(number) + "]" + "[link]" + "]) [" + \
                         "[mailto:foo.bar@example.com]" + "[Foo Bar]" + "]:" +
                         " [" + "[file:/home/user/path1/file" + str(number) + \
                         "]" + "[This is the subject line]" + "]  :tag3:tag4:\n" + \
                         ":PROPERTIES:\n" + \
                         ":TO: Foo Bar <foo.bar@example.com>\n" + \
                         ":ID: <myid_" + str(number) + "@example.com>\n" + \
                         ":END:\n")

Some brackets were slightly obfuscated to prevent github from interpreting this as Org-mode syntax.