Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
427 lines (319 sloc) 15.691 kb

## Time-stamp: <2015-01-02 16:00:11 vk> ## This file is best viewed with GNU Emacs Org-mode: http://orgmode.org/

Best Practices

Invocation

Most of the time you probably want to invoke Memacs modules automatically and periodically. Here are some tips for achieving this.

For each Memacs module there is documentation <memacs_module.org> located in the same folder as the file. Additionally you can get help for each module by executing it with the “-h” argument e.g.:

./bin/memacs_module.py -h

Important: Pythonpath environment variable

The Memacs folder has to be on the PYTHONPATH. Just Google for it how to set it.

If you are using crontab you have to set the PYTHONPATH there too. You can put following line at the top of your crontab:

PYTHONPATH=/path/to/Memacs

GNU/Linux (like Ubuntu, SuSE, RedHat, Debian, …)

You are lucky because cron provides a fairly easy to use interface for this!

Wikipedia offers a cool example section on how to generate cronjob entries. Most of the time you only have to call «crontab -e» and you can start entering a line.

Mac OS X

Older versions of Mac OS X used to support cron. But since OS X 10.4 «Tiger» you have to use launchd instead.

Microsoft Windows

Please refer to Windows Task Scheduler.

Error Handling

Because our memacs-modules run automatically, it would be difficult to be notified in case of an error. So memacs-modules handle errors by appending them to a file:

When you specify an outputfile like this:

memacs-module.py -o /path/to/output.org_archive

Then errors are automatically added to

/path/to/error.org
  • Create the file containing following information:
    ## -*- coding: utf-8 mode: org -*-
    ## this file is for error-messages of Memacs
    ## to add this file to your org-agenda open error.org with emacs and: M-x org-agenda-file-to-front
    ##
    ## Main idea of this file:
    ## if a memacs-module has an error following entry will be generated:#
    ## ** <TIMESTAMP-ERROR-occured +1d> memacs-foo had a problem: <error-msg> #
    ## the "+1d" means that this entry will appear in org agenda view until the entry is deleted from this file
    ##
    * Memacs error 					      :Memacs:ERROR:
    # start deleting from here:
        
  • Add the file to Org-Agenda:
    • open the file in emacs and M-x org-agenda-file-to-front

If an error occurs it will be added to the error.org with a repeating timestamp every day. You have to delete the error manually.

Autotagging

Autotagging allows tags to be set for each entry automatically.

For now autotagging is only available for the <headline> of the entry.

*** <1900-00-00> <headline> :tags:
    <additional data>
    :PROPERTIES:
    :END:

Example Autotag file:

[autotag]
haha=foo, fooo bar, foobar
lala=bar, baaar

Autotag file style:

[autotag]
<TAG>=names to match for that tag seperated by a comma
<TAG>=....
....

Example file without Autotag:

** <1970-01-01 Thu 00:00> foo
** <1970-01-01 Thu 00:00> bar	:tag1:tag2

Example file with Autotag:

.../bin/memacs_example.py --autotag /tmp/autotags
** <1970-01-01 Thu 00:00> foo	:haha:
** <1970-01-01 Thu 00:00> bar	:tag1:tag2:lala:

Performance and Scalability

File names of generated Memacs files: archive

Memacs usually generates Org-mode archive files:

  • «emails.org» … only a stub
  • «emails.org_archive» … containing Orgmode heading lines with Memacs data

Thus in day to day use Memacs entries are not processed by Org Agenda commands, so do not slow down your Agenda.

Only when you choose to view the archive files (“v A” in Agenda-view) will you get the Memacs data displayed in your Orgmode Agenda.

So your daily work will not be slowed down, but you have the opportunity to get the verbose information on demand.

How to use archive files

  1. Generate a stub file like «~/org-mode/files.org»
  2. You might want to enter some information there but you can choose to leave the file empty
  3. Let Memacs generate your Org-mode archive file like «~/org-mode/files.org_archive»
  4. Open «~/org-mode/files.org» in your GNU Emacs and invoke «M-x org-agenda-file-to-front»
  5. You have successfully added this Memacs module to your Org-mode Agenda
  6. Whenever you want to see the entries of «files.org_archive» (or any other archive file) in your Agenda, invoke «v A» when you are in your normal Agenda view.

See: “v A” in Agenda-view

Performance of the Agenda

I am using Org-mode with following archive files containing 238287 headings (as of 2015-01-02):

   lines file
--------------------------------
   53652 archive.org_archive
    2204 bank.org_archive
   35513 datebk6.org_archive
   23155 delicious.org_archive
       1 error.org_archive
  372165 files.org_archive
    1523 gcal.org_archive
   14538 git.org_archive
   19077 mbox.org_archive
    6830 news.org_archive
   33918 phonecalls.org_archive
   57136 roylog.org_archive
   69284 sms.org_archive
   51066 tweets.org_archive
  403645 www.org_archive
--------------------------------
 1143707 total

Additionally, my non-memacs Org-mode files do contain 400328 lines in 17869 headings. So in total, I’ve got over 600000 lines and over 250000 headings.

Starting the «normal» Agenda (without archive files activated) is not affected by the *.org_archive files at all.

When I am in agenda view and I switch to the agenda entries as well (using «v A»), I face a delay of less than four seconds.

After that it depends on how many entries you have got for that specific view (day, week, …). I get only a slightly worse performance, but it is perfectly usable for me.

Optimize Emacs/Org-mode configuration for performance

You might as well check this page on Worg for optimizations regarding the agenda generation process.

SSD versus HDD

By using SSDs instead of hard disks, you get a huge performance boost. For example with the following command:

find /home -name '[12][0-9][0-9][0-9]-[01][0-9]-[0123][0-9]*' -type f 2>/dev/null | \
egrep -v '(/s/|temporary|/\.|/restricted/)' > /tmp/files-tmp && \
~/bin/memacs-filenametimestamps.py -f /tmp/files-tmp \
       -w -o ~/org-mode/memacs/files.org_archive

It crawls over 132,000 files of my home folder, filters using a regular expression, removes paths containing temporary or restricted folders, and generates Orgmode Memacs entries for over 24,000 files resulting in a file of four and a half megabytes.

This command takes no longer than two and a half seconds on an SSD which is quite remarkable. Using a normal hard disk, a comparable call takes one minute and fifteen seconds.

File organization

The more Memacs modules you are using, the more files (*.org and *.org_archive) you get in your (single?) Orgmode-folder.

You might want to create a separate Memacs folder containing all generated Memacs Orgmode files:

/home/user/orgmode/work.org
/home/user/orgmode/work.org_archive
/home/user/orgmode/private.org
/home/user/orgmode/private.org_archive
/home/user/orgmode/memacs/module1.org
/home/user/orgmode/memacs/module1.org_archive
/home/user/orgmode/memacs/module2.org
/home/user/orgmode/memacs/module2.org_archive
/home/user/orgmode/memacs/error.org          <- See Section Error Handling

Power Consumption while on battery

When you are using Memacs on a notebook, you might not want to execute certain cron jobs while being on battery power.

Ubuntu GNU/Linux uses the proc file system where you can access many hardware-related information such as battery state:

vk@gary ~ % cat /proc/acpi/battery/BAT0/state
present:                 yes
capacity state:          ok
charging state:          charged
present rate:            0 mW
remaining capacity:      35290 mWh
present voltage:         12526 mV
vk@gary ~ %

:

## now I disconnect the battery from external power supply

:

vk@gary ~ % cat /proc/acpi/battery/BAT0/state
present:                 yes
capacity state:          ok
charging state:          discharging
present rate:            18452 mW
remaining capacity:      35270 mWh
present voltage:         12426 mV
vk@gary ~ %

With a simple shell script named «no-power-supply.sh», you can execute cron job commands only when connected to an external power supply:

#!/bin/sh
grep discharging /proc/acpi/battery/BAT0/state >/dev/null

or if you have more than one battery:

#!/bin/sh
grep discharging /proc/acpi/battery/BAT0/state /proc/acpi/battery/BAT1/state >/dev/null

Then some example cron jobs look like:

5-59/10 1,8-23 * * * /usr/local/bin/no-power-supply.sh || /home/vk/bin/do_some_things.sh
10 * * * * /usr/local/bin/no-power-supply.sh || { find ....  | egrep '...' > tmpfile && do_that.sh }

Whenever your notebook is in state «discharge» those cronjobs are not executed.

Tracking office hours

There are multiple ways to track office hours. Here, I describe one possible method which requires an Android phone and Memacs.

Phone:

Using Tasker, I am writing a log entry in case I recognize the WiFi network of my company (see memacs_simplephonelogs.org for details).

In this case, the lines have to look like this:

2013-10-01 # 09.14 # wifi-office # 90 # 9742
2013-10-01 # 19.00 # wifi-office-end # 66 # 44906

2013-10-02 # 08.57 # wifi-office # 98 # 4313
2013-10-02 # 12.29 # wifi-office-end # 91 # 17066
2013-10-02 # 16.02 # wifi-office # 91 # 29836
2013-10-02 # 17.37 # wifi-office-end # 80 # 35537

2013-10-03 # 08.58 # wifi-office # 97 # 5300
2013-10-03 # 18.41 # wifi-office-end # 69 # 11166

2013-10-04 # 09.02 # wifi-office # 97 # 5591
2013-10-04 # 13.28 # wifi-office-end # 89 # 21512

It is mandatory that you are using wifi-office and wifi-office-end as logging strings. This way, memacs\_simplephonelogs is recognizing office hours and handle them accordingly (and different).

Example scripts, data format, and more is explained in the documentation of the simplephonelogs module.

Computer:

From time to time, I use rsync for Android to transfer this log file to my computer. There, memacs_simplephonelogs.org parses it and generates ~/org/memacs/phonelog.org_archive.

The phone log lines from above result in following headings:

** <2013-10-01 Tue 09:14> wifi-office (not office for 15:14:00)
** <2013-10-01 Tue 19:00> wifi-office-end (office for 9:46:00; today 9:46:00; today total 9:46:00)
** <2013-10-02 Wed 08:57> wifi-office (not office for 13:57:00)
** <2013-10-02 Wed 12:29> wifi-office-end (office for 3:32:00; today 3:32:00; today total 3:32:00)
** <2013-10-02 Wed 16:02> wifi-office (not office for 3:33:00)
** <2013-10-02 Wed 17:37> wifi-office-end (office for 1:35:00; today 5:07:00; today total 8:40:00)
** <2013-10-03 Thu 08:58> wifi-office (not office for 15:21:00)
** <2013-10-03 Thu 18:41> wifi-office-end (office for 9:43:00; today 9:43:00; today total 9:43:00)
** <2013-10-04 Fri 09:02> wifi-office (not office for 14:21:00)
** <2013-10-04 Fri 13:28> wifi-office-end (office for 4:26:00; today 4:26:00; today total 4:26:00)

Using a babel script, I grab all office-end-lines, re-format them to get a decent Org-mode table result, and filter for a month (here: 2013-10):

#+NAME: office_hours
#+BEGIN_SRC sh
echo " day | duration  today-sum  today-total"
grep '\*\* ' ~/org/memacs/phonelog.org_archive | \
grep '(office' | \
sed 's/:00)//' | \
sed 's/:00//g' | \
sed 's/\*\*//' | \
sed 's/wifi-office-end (office for//' | \
sed 's/today//' | \
sed 's/today total//' | \
sed 's/;//g' | \
grep 2013-10
#+END_SRC

This results in something similar to this:

The column duration is the difference between entering and leaving the office on an event basis.

The column today-sum is the sum of all durations (office time) of that day.

The last column today-total is the difference between entering the office for the first time and leaving the office for the last time (on that day; including all absent times as well!).

You can manually remove lines in case you left and re-entered office. In the example above, I would delete the line with <2013-10-02 Wed 12:29> because I got all necessary information in the last line of that day: <2013-10-02 Wed 17:37>.

With the usual spreadsheet features, you can use this data to calculate even more or you can enter them to the time-tracking system of your company.

Have fun :-)

Internals

How to write a memacs module?

see

Memacs/bin/memacs_example.py
Memacs/memacs/example.py
Memacs/tests/example_test.py

Testing

Use nosetests for executing tests install it with:

% easy_install nosetests

or

# aptitude install python-nose

how does the appendmode of memacs work? / How is the :ID: Property generated?

All properties (:PROPERTIES: drawer) are stored in a dict

i.e.: :FOO: <bar>
key = "FOO , value = <bar>

generation:

id-hash = sha1(<all values> + <all keys>)

Before writing an entry to the org-file, the id is generated.

If a Memacs module is in appendmode, it looks for those :ID: properties and stores them in a list. On writing(append) it first checks against that list.

what to do if our :PROPERTIES: do not give unique data?

… so that a hash is not unique?

you can set OrgProperties(data_for_hashing=”more_data”) to solve this problem

FAQs

If you have a question, please contact «memacs at Karl minus Voit dot at» and he is happy to answer it.

Emacs always asks about what to do with changed org files when Memacs re-generates them in the background

The solution is to add this to your emacs config file (.emacs):

(global-auto-revert-mode t)

Why is Memacs implemented in Python and not in Elisp?

We are power-users of GNU/Emacs. Unfortunately, we do not have much Elisp knowledge. Therefore, we decided to implement Memacs with Python, a scripting language we knew and which is far more common than Elisp.

This way, other Emacs users without Elisp knowledge are able to implement Memacs modules as well.

Feel free and re-implement Memacs in Elisp and let us know!

Jump to Line
Something went wrong with that request. Please try again.