Skip to content


Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
tree: ebc98c5348
Fetching contributors…

Cannot retrieve contributors at this time

3642 lines (3091 sloc) 145.689 kb

Org Mode - Organize Your Life In Plain Text!

1 Getting Started

I use org-mode in most of my emacs buffers.

1.1 Org-Mode Setup

The following setup in my .emacs enables org-mode for most buffers. org-mode is the default mode for .org, .org_archive, and .txt files.
;;; Org Mode
(add-to-list 'load-path (expand-file-name "~/git/org-mode/lisp"))
(add-to-list 'auto-mode-alist '("\\.\\(org\\|org_archive\\|txt\\)$" . org-mode))
(require 'org-install)
;; Standard key bindings
(global-set-key "\C-cl" 'org-store-link)
(global-set-key "\C-ca" 'org-agenda)
(global-set-key "\C-cb" 'org-iswitchb)
function! FormatTags(line)
    let tagmatch = matchlist(getline(a:line),'\(:\S*:\)\s*$')
    if !empty(tagmatch)
        let linetags = tagmatch[1]
        " add newtags back in, including new tag
        call setline(a:line,getline(a:line) . '    ' 
                    \ . repeat(' ', winwidth(0) - len(getline(a:line)) - len(linetags) - 15) 
                    \ . linetags)

orgstruct++-mode is enabled in Gnus message buffers to aid in creating structured email messages.

(setq message-mode-hook
      (quote (orgstruct++-mode
              (lambda nil (setq fill-column 72) (flyspell-mode 1))

flyspell-mode is enabled for almost everything to help prevent creating documents with spelling errors. yasnippets are enabled to speed up creation of standard text blocks in most editing modes.

;; Make TAB the yas trigger key in the org-mode-hook and enable flyspell mode and autofill
(add-hook 'org-mode-hook
          (lambda ()
            ;; yasnippet
            (make-variable-buffer-local 'yas/trigger-key)
            (org-set-local 'yas/trigger-key [tab])
            (define-key yas/keymap [tab] 'yas/next-field-group)
            ;; flyspell mode for spell checking everywhere
            (flyspell-mode 1)
            ;; auto-fill mode on
            (auto-fill-mode 1)))

1.2 Organizing Your Life Into Org Files

Tasks are separated into logical groupings or projects. Use separate org files for large task groupings.

Here are sample files that I use.

The following org files collect non-work related tasks:

Filename Description Personal tasks and things to keep track of Google Summer of Code stuff for 2009 Farm related tasks Tasks related to my son Mark Org-mode related tasks Git related tasks BZFlag related tasks

The following org-file collects org capture notes and tasks:

Filename Description Capture task bucket

The following work-related org-files keep my business notes (using fictitious client names)

Filename Description Norang tasks and notes XYZ Corp tasks and notes ABC Ltd tasks ABC Ltd tasks for their client DEF Corp ABC Ltd tasks for their client KKK Inc YYY Inc tasks

Org-mode is great for dealing with multiple clients and client projects. An org file becomes the collection of projects, notes, etc. for a single client or client-project.

Clients (ABC Ltd) has multiple customer systems that I work on. Separating the tasks for each client-customer into separate org files helps keep things logically grouped and since clients come and go this allows entire org files to be added or dropped from my agenda to keep only what is important visible in agenda views.

Other org files are used for publishing only and do not contribute to the agenda. See Publishing for more details.

1.3 Agenda Setup

Here is my current org-agenda-files setup. It is shown above formatted as a setq for clarity but in reality this is saved in my custom.el file.
(setq org-agenda-files (quote ("~/git/org/"
                               ; client org files removed

org-mode manages the org-agenda-files variable. I just visit an org file and add it to the agenda with C-c [. To remove a file I just visit it and hit C-c ] and all of the tasks in that file are instantly removed from my agenda views until I add them back again.

1.4 Org File Structure

Most of my org files are set up with level 1 headings as main categories only. Tasks normally start as level 2.

Here are some examples of my level 1 headings in

  • Appointments
  • Special Dates

    Includes level 2 headings for

    • Birthdays
    • Anniversaries
    • Holidays
  • Finances
  • Health
  • House Maintenance
  • Medical
  • Miscellaneous
  • Lawn and Garden

  • System Maintenance
  • Payroll
  • Accounting
  • Finances
  • Hardware Maintenance
  • Quotes
  • Administration
  • Research

Each of these level 1 tasks normally has a property drawer specifying the archive location and category for any tasks in that tree. Level 1 headings are set up like this:

* Appointments
  :ARCHIVE:  %s_archive:* Appointments
* Miscellaneous
  :CATEGORY: todo
  :ARCHIVE: %s_archive:* Miscellaneous

This ensures that any level 2 task that I archive from this heading (I archive by subtree) gets saved in the archive file under the appropriate level 1 heading so I can find it back again if needed.

This keeps my main org files and my archives with basically the same structure.

1.5 Key bindings

I live in the agenda. To make getting to the agenda faster I mapped F12 to the sequence C-c a since I’m using it hundreds of times a day.

I have the following custom key bindings set up for my emacs (sorted by frequency).

Key For Used
F12 Agenda (1 key less than C-c a) Very Often
C-c b Switch to org file Very Often
C-F11 Clock in a task (show menu with prefix) Very Often
f9 g Gnus - I live in gnus Often
C-M-r Capture a task Often
F11 Goto currently clocked item Often
f5 Show todo items for this subtree Often
S-f5 Widen Often
f9 b Quick access to bbdb data Often
f9 c Calendar access Often
f9 r Boxquote selected region Often
C-S-f12 Save buffers and publish current project Often
C-c l Store a link for retrieval with C-c C-l Often
f8 Go to next org file in org-agenda-files Sometimes
f9 t Insert inactive timestamp Sometimes
f9 v Toggle visible mode (for showing/editing links) Sometimes
C-f9 Previous buffer Sometimes
C-f10 Next buffer Sometimes
C-x n r Narrow to region Sometimes
f9 f Boxquote insert a file Sometimes
f9 i Org-mode Info manual Sometimes
f9 I Punch Clock In (start clocking) Sometimes
f9 O Punch Clock Out (stop clocking) Sometimes
f9 s Switch to scratch buffer Sometimes
M-f9 Remove unmodified buffer and frame Sometimes
f9 h Hide other tasks Rare
f7 Toggle line truncation/wrap Rare
f9 u Untabify region Rare
C-c a Enter Agenda (minimal emacs testing) Rare
M-f11 Resolve open clocks Rare

Here is the keybinding setup in lisp:

;; Custom Key Bindings
(global-set-key (kbd "<f12>") 'org-agenda)
(global-set-key (kbd "<f5>") 'bh/org-todo)
(global-set-key (kbd "<S-f5>") 'bh/widen)
(global-set-key (kbd "<f7>") 'set-truncate-lines)
(global-set-key (kbd "<f8>") 'org-cycle-agenda-files)
(global-set-key (kbd "<f9> b") 'bbdb)
(global-set-key (kbd "<f9> c") 'calendar)
(global-set-key (kbd "<f9> f") 'boxquote-insert-file)
(global-set-key (kbd "<f9> g") 'gnus)
(global-set-key (kbd "<f9> h") 'bh/hide-other)

(defun bh/hide-other ()

(global-set-key (kbd "<f9> i") 'bh/org-info)

(defun bh/org-info ()
  (info "~/git/org-mode/doc/"))

(global-set-key (kbd "<f9> I") 'bh/clock-in)
(global-set-key (kbd "<f9> O") 'bh/clock-out)
(global-set-key (kbd "<f9> r") 'boxquote-region)
(global-set-key (kbd "<f9> s") 'bh/go-to-scratch)

(defun bh/go-to-scratch ()
  (switch-to-buffer "*scratch*")

(global-set-key (kbd "<f9> t") 'bh/insert-inactive-timestamp)
(global-set-key (kbd "<f9> u") 'bh/untabify)

(defun bh/untabify ()
  (untabify (point-min) (point-max)))

(global-set-key (kbd "<f9> v") 'visible-mode)
(global-set-key (kbd "<f9> SPC") 'bh/clock-in-last-task)
(global-set-key (kbd "C-<f9>") 'previous-buffer)
(global-set-key (kbd "C-x n r") 'narrow-to-region)
(global-set-key (kbd "C-<f10>") 'next-buffer)
(global-set-key (kbd "<f11>") 'org-clock-goto)
(global-set-key (kbd "C-<f11>") 'org-clock-in)
(global-set-key (kbd "C-s-<f12>") 'bh/save-then-publish)
(global-set-key (kbd "M-<f11>") 'org-resolve-clocks)
(global-set-key (kbd "C-M-r") 'org-capture)
(global-set-key (kbd "M-<f9>") (lambda ()
                                 (unless (buffer-modified-p)
                                   (kill-buffer (current-buffer)))

The main reason I have special key bindings (like F11, and F12) is so that the keys work in any mode. If I’m in the Gnus summary buffer then C-u C-c C-x C-i doesn’t work, but the C-F11 key combination does and this saves me time since I don’t have to visit an org-mode buffer first just to clock in a recent task.

2 Refiling Tasks

Refiling tasks is easy. After collecting a bunch of new tasks in my file using capture mode I need to move these to the correct org file and topic. All of my active org-files are in my org-agenda-files variable and contribute to the agenda.

I collect capture tasks in for up to a week. I do my weekly review every Monday and one of the tasks for that is to refile all capture tasks. Often I end up refiling tasks the same day I create them because they show up in my daily clock report summary and are obviously in the wrong place.

2.1 Refile Setup

To refile tasks in org you need to tell it where you want to refile things.

In my setup I let any file in org-agenda-files and the current file contribute to the list of valid refile targets. I don’t refile to tasks more then 5 levels deep just to limit the number of displayed targets. I also use ido mode to help find targets quickly.

; Use IDO for target completion
(setq org-completion-use-ido t)

; Targets include this file and any file contributing to the agenda - up to 5 levels deep
(setq org-refile-targets (quote ((org-agenda-files :maxlevel . 5) (nil :maxlevel . 5))))

; Targets start with the file name - allows creating level 1 tasks
(setq org-refile-use-outline-path (quote file))

; Targets complete in steps so we start with filename, TAB shows the next level of targets etc
(setq org-outline-path-complete-in-steps t)

; Allow refile to create parent tasks with confirmation
(setq org-refile-allow-creating-parent-nodes (quote confirm))

; Use IDO only for buffers
; set ido-mode to buffer and ido-everywhere to t via the customize interface
; '(ido-mode (quote both) nil (ido))
; '(ido-everywhere t)

Here is my refile configuration:

; Use IDO for target completion
(setq org-completion-use-ido t)

; Targets include this file and any file contributing to the agenda - up to 5 levels deep
(setq org-refile-targets (quote ((org-agenda-files :maxlevel . 5) (nil :maxlevel . 5))))

; Targets start with the file name - allows creating level 1 tasks
(setq org-refile-use-outline-path (quote file))

; Targets complete in steps so we start with filename, TAB shows the next level of targets etc
(setq org-outline-path-complete-in-steps t)

; Allow refile to create parent tasks with confirmation
(setq org-refile-allow-creating-parent-nodes (quote confirm))

; Use IDO only for buffers
; set ido-mode to buffer and ido-everywhere to t via the customize interface
; '(ido-mode (quote both) nil (ido))
; '(ido-everywhere t)

To refile a task to my file under System Maintenance I just put the cursor on the task and hit C-c C-w and enter nor TAB sys TAB RET and it’s done. I always know what file it’s going into but if I don’t remember the exact task name I can just hit TAB twice and all refile targets that match show up in a list. Just scroll through the list and pick the right refile target. This works great! *** This is a test of org.vim

And text underneath this head. aldkfj adlfkad fjad;flka df ;ajdf; lkadjsf ad;sjfa; dslkfj ads f;jad ;flkja dsf a;dsjfa;dlkfja ;dsfk ad;jfa; dslkfja dsf this si a dkfj adlsf ajdfa ldf adjfad kfj;aldkfj aldkf adsjfa;dkfja dfkjjj ** Refiling Tasks :George:To find tasks to refile I run my agenda view (F12 r = C-c a r) which shows tasks with the REFILE tag. This view shows all tasks (even ones marked in a done state).

My single capture target file has this tag in the FILETAGS header so every task in the file can be found using this view.

I visit each file with REFILE tasks to refile. If there are a few tasks going to the same place (3 or less) I refile the first one, then move to the second one and use C-c C-w up-arrow RET to refile to the same location again. If more than 3 tasks are going to the same place I try to do those last - since refiling everything else away helps to group those together. Then I mark those tasks in m and bulk refile them to the same target with B r in the agenda view.

Refiling all of my tasks tends to take less than a minute so I may do this a couple of times a day. ** Refiling Notes

I keep a * Notes headline in most of my org-mode files. Notes have a NOTE tag which is created by the capture template for notes. This allows finding notes across multiple files easily using the agenda search functions.

Notes created by capture tasks go first to and are later refiled to the appropriate project file. Some notes that are project related get filed to the appropriate project instead of under the catchall * NOTES task. Generally these types of notes are specific to the project and not generally useful – so removing them from the notes list when the project is archived makes sense. ** Refiling Phone Calls :John:home:phone:

Phone calls are handled using a few custom functions and a special key binding. I time my calls using the capture mode template settings to clock in and out the capture task while the phone call is in progress.

Phone call tasks collect in and are later refiled to the appropriate location. Some phone calls are billable and we want these tracked in the appropriate category. * Custom agenda views :phone:

I have 10 custom agenda views defined. Most of my old custom agenda views were rendered obsolete when filtering functionality was added to the agenda in newer versions of org-mode.

Custom agenda views are used for:

  1. Finding tasks waiting on something
  2. Finding tasks to be refiled
  3. Finding notes
  4. Finding NEXT tasks to work on
  5. Reviewing projects
  6. Reviewing other non-project tasks
  7. Findings tasks to be archived
  8. Viewing habits
  9. Finding stuck projects
  10. Setting the default clocking task for punching in

** Setup

(setq org-agenda-custom-commands
      (quote (("w" "Tasks waiting on something" tags "WAITING/!"
               ((org-use-tag-inheritance nil)
                (org-agenda-todo-ignore-scheduled nil)
                (org-agenda-todo-ignore-deadlines nil)
                (org-agenda-todo-ignore-with-date nil)
                (org-agenda-overriding-header "Waiting Tasks")))
              ("r" "Refile New Notes and Tasks" tags "LEVEL=1+REFILE"
               ((org-agenda-todo-ignore-with-date nil)
                (org-agenda-todo-ignore-deadlines nil)
                (org-agenda-todo-ignore-scheduled nil)
                (org-agenda-overriding-header "Tasks to Refile")))
              ("N" "Notes" tags "NOTE"
               ((org-agenda-overriding-header "Notes")))
              ("n" "Next" tags-todo "-WAITING-CANCELLED/!NEXT"
               ((org-agenda-overriding-header "Next Tasks")))
              ("p" "Projects" tags-todo "LEVEL=2-REFILE|LEVEL=1+REFILE/!-DONE-CANCELLED"
               ((org-agenda-skip-function 'bh/skip-non-projects)
                (org-agenda-overriding-header "Projects")))
              ("o" "Other (Non-Project) tasks" tags-todo "LEVEL=2-REFILE|LEVEL=1+REFILE/!-DONE-CANCELLED"
               ((org-agenda-skip-function 'bh/skip-projects)
                (org-agenda-overriding-header "Other Non-Project Tasks")))
              ("A" "Tasks to be Archived" tags "LEVEL=2-REFILE/DONE|CANCELLED"
               ((org-agenda-overriding-header "Tasks to Archive")))
              ("h" "Habits" tags "STYLE=\"habit\""
               ((org-agenda-todo-ignore-with-date nil)
                (org-agenda-todo-ignore-scheduled nil)
                (org-agenda-todo-ignore-deadlines nil)
                (org-agenda-overriding-header "Habits")))
              ("#" "Stuck Projects" tags-todo "LEVEL=2-REFILE|LEVEL=1+REFILE/!-DONE-CANCELLED"
               ((org-agenda-skip-function 'bh/skip-non-stuck-projects)
                (org-agenda-overriding-header "Stuck Projects")))
              ("c" "Select default clocking task" tags "LEVEL=2-REFILE"
                 '(org-agenda-skip-subtree-if 'notregexp "^\\*\\* Organization"))
                (org-agenda-overriding-header "Set default clocking task with C-u C-u I"))))))

My day goes generally like this:

  • Punch in (start the clock)
  • Look at my agenda F12 a
    • make a note of anything important to deal with today this is another line in list
    • this skja sdlfkja fkajdf ajdfa;dfja dfja;dfkajdfa dfjadf;adf asdf
  • Read email and news
    • create notes, and tasks for things that need responses with org-capture
  • Check refile tasks and respond to emails
  • Look at my agenda and knock off tasks scheduled for today
    • Clock it in (I in the agenda or on the beginning of a task headline
      • this changes TODO state tasks to NEXT when there are no unfinished subtasks
    • Work on it until it is DONE or it gets interrupted
  • work on tasks
  • Punch out and go for lunch
  • Punch in and continue work for the afternoon
  • work on tasks
  • Check today’s time log report and refile tasks with clocked time
    • F12 a R - any tasks in should be moved to the appropriate file
    • F12 r to get to refile tasks
    • Tag files to be filed with m collecting all tasks for the same target
    • Bulk refile the tasks to the target location with B r
    • repeat until the agenda timeclock report has all of the time in project files
  • Punch out (stop the clock)

** What do I work on next? :George:computer:phone:reading:

Use the agenda view for NEXT tasks to find stuff in progress and things to clock.

When I look for a new task to work on I generally hit F12 a to get today’s agenda and follow this order:

  • Pick something off today’s agenda
    • deadline for today (do this first - it’s not late yet)
    • deadline in the past (it’s already late)
    • deadline that is coming up soon
    • a scheduled task for today (or in the past)
  • pick a NEXT task
  • If you run out of items to work on look for NEXT task in the current context F12 n / RET

*** Why keep it all on the NEXT list? :computer:home:phone:

I’ve moved to a more GTD way of doing things. I don’t have a STARTED list or todo keyword anymore. Now I just use a NEXT list. If I clock a TODO keyword it changes to NEXT if that is appropriate automagically on clock in. A NEXT task is something that is available to work on now, it was either clocked already or is the next logical step in some project.

I used to have a special keyword ONGOING for things that I do a lot and want to clock but never really start/end. I had a special agenda view for ONGOING tasks that I would pull up to easily find the thing I want to clock.

Since then I’ve moved away from using the ONGOING todo keyword. If a task is clocked-in it automatically moves to the NEXT state from TODO state and shows up on the NEXT task list without having to think about it. Having an agenda view that shows NEXT tasks makes it easy to pick the thing to clock - and I don’t have to remember if I need to look in the ONGOING list or the NEXT list when looking for the task to clock-in. The NEXT list is basically ‘what is current’ - stuff I worked on recently and need to finish and any task that moves a project forward. I want to find the thing to work on as fast as I can and actually do work on it - not spend time hunting through my org files for the task that needs to be clocked-in.

To drop a task off the NEXT list simply move it back to the TODO state. ** Reading email, newsgroups, and conversations on IRC :Sally:computer:home:

When reading email, newsgroups, and conversations on IRC I just let the default task (normally ** Organization) clock the time I spend on these tasks. To read email I go to Gnus and read everything in my inboxes. If there are emails that require a response I use org-capture to create a new task with a heading of ‘Respond to <user>’ for each one. This automatically links to the email in the task and makes it easy to find later. Some emails are quick to respond to and some take research and a significant amount of time to complete. I clock each one in it’s own task just in case I need that clocked time later.

Next, I go to my newly created tasks to be refiled with F12 r and clock in an email task and deal with it. Repeat this until all of the ‘Respond to <user>’ tasks are marked DONE.

I read email and newgroups in Gnus so I don’t separate clocked time for quickly looking at things. If an article has a useful piece of information I want to remember I create a note for it with C-M-r n and enter the topic and file it. This takes practically no time at all and I know the note is safely filed for later retrieval. The time I spend in the capture buffer is clocked with that capture note. ** Filtering

So many tasks, so little time. I have hundreds of tasks at any given time (373 right now). There is so much stuff to look at it can be daunting. This is where agenda filtering saves the day.

It’s 11:53AM and I’m in work mode just before lunch. I don’t want to see tasks that are not work related right now. I also don’t want to work on a big project just before lunch… so I need to find small tasks that I can knock off the list.

How do we do this? Get a list of NEXT tasks with F12 n and then narrow it down with filtering. To find tasks to work on I remove tasks I’m not supposed to be working on now with / RET. Then limit to tasks with estimates of 10 minutes or less with / + 1 and I can pick something that fits the minutes I have left before I take off for lunch.

*** Automatically removing context based tasks with / RET

/ RET in the agenda is really useful. This awesome feature was added to org-mode by John Wiegley. It removes tasks automatically by filtering based on a user-provided function.

I work from home and set up my day as follows:

  • On weekdays 8am-12am, 1pm-5pm I’m working (@office)
  • My son (Mark) is available on weekdays before school 8am-9am and after school to bedtime 4pm-8pm (MARK), and weekends 10am-8pm
  • Home tasks are done outside working hours (@home)

I have the following setup to allow / RET to filter tasks based on what the computer determines my current context to be at the time I run the / RET filter command.

(defun bh/weekday-p ()
  (let ((wday (nth 6 (decode-time))))
    (and (< wday 6) (> wday 0))))

(defun bh/working-p ()
  (let ((hour (nth 2 (decode-time))))
    (and (bh/weekday-p) (or (and (>= hour 8) (<= hour 11))
                           (and (>= hour 13) (<= hour 16))))))

(defun bh/network-p ()
  (= 0 (call-process "/bin/ping" nil nil nil
                     "-c1" "-q" "-t1" "")))

(defun bh/org-auto-exclude-function (tag)
  (and (cond
       ((string= tag "@home")
       ((string= tag "@office")
        (not (bh/working-p)))
       ((or (string= tag "@errand") (string= tag "phone"))
        (let ((hour (nth 2 (decode-time))))
          (or (< hour 8) (> hour 21)))))
       (concat "-" tag)))

(setq org-agenda-auto-exclude-function 'bh/org-auto-exclude-function)

This lets me filter tasks with just / RET on the agenda which removes tasks I’m not supposed to be working on now from the list of returned results.

This helps to keep my agenda clutter-free. * Time Clocking

Okay, I admit it. I’m a clocking fanatic.

I clock everything (well almost everything). Org-mode makes this really easy. I’d rather clock too much stuff than not enough so I find it’s easier to get in the habit of clocking everything.

As an example of what I mean my clock data for April 20, 2009 shows 14 hours 19 minutes of clocked time (which included 3 hours and 17 minutes of painting my basement.) My clocked day started at 6:57AM and ended at 23:11PM. I have only a few holes in my clocked day (where I wasn’t clocking anything):

Missing Clock Data

This makes it possible to look back at the day and see where I’m spending too much time, or not enough time on specific projects.

Without clocking data it’s hard to tell what you did after the fact.

I now use the concept of punching in and punching out at the start and end of my work day. This defines a default task to clock time on whenever the clock would normally stop. I found that with the default org-mode setup I would lose clocked minutes during the day, a minute here, a minute there, and that all adds up. This is especially true if you write notes when moving to a DONE state - in this case the clock normally stops before you have composed the note.

My clocking setup basically works like this:

  • Punch in (start the clock)
    • This identifies a task that is the default task to clock in whenever the clock normally stops
  • Clock in tasks normally, and let moving to a DONE state clock out
    • clocking out automatically clocks time on the default task
  • Continue clocking whatever tasks you work on
  • Punch out (stop the clock)

I’m free to change the default task multiple times during the day. If I’m working on Project X then I can make the top-level Project X task the default and all clocked time goes on that project until I either punch out or change to some other default task.

I now have a default level 2 ** Organization task that I use for every major context I clock time in.

My org files are look like this:

** Organization

** Organization

** Organization

** Organization

If I am working on tasks, then I set the ** Organization task as the default clock task. If I’m working for client SOMECLIENT then I set the ** Organization task in as the default task etc. This allows me to block time on my calendar and work on a single context for some time frame of my day, then totally switch to another context simply by changing my default clocking task. Or course if I get interrupted in the middle of the day clocking in a task puts time on that task regardless of the context I’m supposed to be working in.

This works really well for me.

2.2 Clock Setup

To get started we need to set the default clocking task which we use to clock in whenever the clock would normally stop. I use a special custom agenda view for this and I think of it as selecting the context for what I’m going to work on for the next few hours.

F12 c shows me the tasks I can normally set as the default clocking task. I’m free to set any task as the default but these are the ones I normally use.

Keeping the clock running when moving a subtask to a DONE state means clocking continues to apply to the parent task. I can pick the next task from the parent and clock that in without losing a minute or two while I’m deciding what to work on next.

I keep clock times in a :CLOCK: drawer and state changes in a :LOGBOOK: drawer.

I have the following org-mode settings for clocking:

;; Resume clocking tasks when emacs is restarted
;; Yes it's long... but more is better ;)
(setq org-clock-history-length 28)
;; Resume clocking task on clock-in if the clock is open
(setq org-clock-in-resume t)
;; Change task state to NEXT when clocking in
(setq org-clock-in-switch-to-state (quote bh/clock-in-to-next))
;; Separate drawers for clocking and logs
(setq org-drawers (quote ("PROPERTIES" "LOGBOOK" "CLOCK")))
;; Save clock data in the CLOCK drawer and state changes and notes in the LOGBOOK drawer
(setq org-clock-into-drawer "CLOCK")
;; Sometimes I change tasks I'm clocking quickly - this removes clocked tasks with 0:00 duration
(setq org-clock-out-remove-zero-time-clocks t)
;; Clock out when moving task to a done state
(setq org-clock-out-when-done t)
;; Save the running clock and all clock history when exiting Emacs, load it on startup
(setq org-clock-persist (quote history))
;; Enable auto clock resolution for finding open clocks
(setq org-clock-auto-clock-resolution (quote when-no-clock-is-running))
;; Include current clocking task in clock reports
(setq org-clock-report-include-clocking-task t)

(setq bh/keep-clock-running nil)

(defun bh/clock-in ()
  (setq bh/keep-clock-running t)
  (if (marker-buffer org-clock-default-task)
      (unless (org-clock-is-active)
    (unless (marker-buffer org-clock-default-task)
      (org-agenda nil "c"))))

(defun bh/clock-out ()
  (setq bh/keep-clock-running nil)
  (when (org-clock-is-active)

(defun bh/clock-in-default-task ()
    (org-with-point-at org-clock-default-task

(defun bh/clock-out-maybe ()
  (when (and bh/keep-clock-running (not org-clock-clocking-in) (marker-buffer org-clock-default-task))

(add-hook 'org-clock-out-hook 'bh/clock-out-maybe 'append)

I used to clock in tasks by ID using the following function but with the new punch-in and punch-out I don’t need these anymore. f9-SPC calls bh/clock-in-last-task which switches the clock back to the previously clocked task.

(require 'org-id)  
(defun bh/clock-in-task-by-id (id)
  "Clock in a task by id"
    (org-with-point-at (org-id-find id 'marker)
      (org-clock-in nil))))

(defun bh/clock-in-last-task ()
  "Clock in the interrupted task if there is one"
  (let ((clock-in-to-task (if (org-clock-is-active)
                              (setq clock-in-to-task (cadr org-clock-history))
                            (setq clock-in-to-task (car org-clock-history)))))
    (org-with-point-at clock-in-to-task
      (org-clock-in nil))))

2.3 Clocking in

When I start or continue working on a task I clock it in with C-c C-x C-i (or just I in the agenda or speed key setting). This changes the task state from TODO to NEXT and starts the clock for this task.

2.3.1 Setting a default clock task

I have a default ** Organization task in my file that I tend to put miscellaneous clock time on. While reorganizing my org-files and doing other planning work that isn’t for a specific project I’ll clock in this task while I do things. By clocking this task in with a double prefix C-u C-u C-c C-x C-i it starts the clock and makes this the default clock task. The first punch-in of the day (f9 I) shows the context agenda view if no default task is selected, otherwise it just clocks in the default task.

You can quickly clock in the default task with C-u C-c C-x C-i d

I now set the default clocking task when I punch in and clocking out of any task will clock in this default task until I punch out using the clocking hooks I have set up.

The only thing I need to remember is to set a new default clock task when I switch contexts (stop working for client A and start working for client B).

2.3.2 Using the clock history to clock in old tasks

You can use the clock history to restart clocks on old tasks you’ve clocked or to jump directly to a task you have clocked previously. I use this mainly to clock in whatever got interrupted by something.

Consider the following scenario:

  • You are working on and clocking Task A (Organization)
  • You get interrupted and switch to Task B (Document my use of org-mode)
  • You complete Task B (Document my use of org-mode)
  • Now you want to go back to Task A (Organization) again to continue

This is easy to deal with.

  1. Clock in Task A, work on it
  2. Go to Task B (or create a new task) and clock it in
  3. When you are finished with Task B hit C-u C-c C-x C-i i

This displays a clock history selection window like the following and selects the interrupted [i] entry.

Clock history selection buffer for C-u C-c C-x C-i

Default Task
[d] norang          Organization                          <-- Task B
The task interrupted by starting the last one
[i] norang          Organization                          <-- Task B
Current Clocking Task
[c] org             NEXT Document my use of org-mode      <-- Task A
Recent Tasks
[1] org             NEXT Document my use of org-mode      <-- Task A
[2] norang          Organization                          <-- Task B
[Z] org             DONE Fix default section links        <-- 35 clock task entries ago

2.4 Clock Everything - Create New Tasks

In order to clock everything you need a task for everything. That’s fine for planned projects but interruptions inevitably occur and you need some place to record whatever time you spend on that interruption.

To deal with this we create a new capture task to record the thing we are about to do. The workflow goes something like this:

  • You are clocking some task and an interruption occurs
  • Create a quick capture task C-M-r
  • Type the heading
  • clock it in C-c C-x C-i
  • file it C-c C-c
  • switch the clock back to it F9 SPC
  • Go do it
  • mark it DONE which stops the clock (or switches to the context default clocking task you punched in earlier)
  • clock something else in
  • refile the newly created and clocked task later

This means you can ignore the details like where this task really belongs in your org file layout and just get on with completing the thing. Refiling a bunch of tasks later in a group when it is convenient to refile the tasks saves time in the long run.

2.5 Editing clock entries

Sometimes it is necessary to edit clock entries so they reflect reality. I find I do this for maybe 2-3 entries in a week.

Occassionally I cannot clock in a task on time because I’m away from my computer. In this case the previous clocked task is still running and counts time for both tasks which is wrong.

I make a note of the time and then when I get back to my computer I clock in the right task and edit the start and end times to correct the clock history.

To visit the clock line for an entry quickly use the agenda log mode. F12 a l shows all clock lines for today. I use this to navigate to the appropriate clock lines quickly. F11 goes to the current clocked task but the agenda log mode is better for finding and visiting older clock entries.

Use F12 a l to open the agenda in log mode and show only logged clock times. Move the cursor down to the clock line you need to edit and hit TAB and you’re there.

To edit a clock entry just put the cursor on the part of the date you want to edit (use the keyboard not the mouse - since the clicking on the timestamp with the mouse goes back to the agenda for that day) and hit the S-<up arrow> or S-<down arrow> keys to change the time.

The following setting makes time editing round to 15 minute increments:

(setq org-time-stamp-rounding-minutes (quote (1 15)))

Editing the time with the shift arrow combination also updates the total for the clock line which is a nice convenience.

I always check that I haven’t created task overlaps when fixing time clock entries by viewing them with log mode on in the agenda.

I want my clock entries to be as accurate as possible but editing to the exact minute (instead of rounding to 15 minutes) takes more time and isn’t worth the hassle. Rounding to 15 minutes gets me close to the time I want quickly and if extra refining is needed I can edit the timestamp directly and update the total with C-c C-y.

3 Time reporting and tracking

:<2010-11-09 Tue 13:00>

3.1 Billing clients based on clocked time

At the beginning of the month I invoice my clients for work done last month. This is where I review my clocking data for correctness before billing for the clocked time.

Billing for clocked time basically boils down to the following steps:

  1. Verify that the clock data is complete and correct
  2. Use clock reports to summarize time spent
  3. Create an invoice based on the clock data I currently create invoices in an external software package based on the org-mode clock data.
  4. Archive complete tasks so they are out of the way.

    See *Archiving for more details.

3.1.1 Verify that the clock data is complete and correct

Since I change tasks often (sometimes more than once in a minute) I use the following setting to remove clock entries with a zero duration.
;; Sometimes I change tasks I'm clocking quickly - this removes clocked tasks with 0:00 duration
(setq org-clock-out-remove-zero-time-clocks t)

This setting just keeps my clocked log entries clean - only keeping clock entries that contribute to the clock report.

Before invoicing for clocked time it is important to make sure your clocked time data is correct. If you have a clocked time with an entry that is not closed (ie. it has no end time) then that is a hole in your clocked day and it gets counted as zero (0) for time spent on the task when generating clock reports. Counting it as zero is almost certainly wrong.

To check for unclosed clock times I use the agenda-view log-mode (l in the agenda) with the following setup which shows clocked times only by default. (To see all task state changes you can issue a prefix to this command (C-u l in the agenda)).

To check the last month’s clock data I use F12 a v m b l which shows a full month in the agenda, moves to the previous month, and shows the clocked times only.

The clocked-time only display in the agenda makes it easy to quickly scan down the list to see if an entry is missing an end time. If an entry is not closed you can manually fix the clock entry based on other clock info around that time.

Use the following setup to get log mode in the agenda to only show clocked times:

;; Agenda log mode items to display (clock time only by default)
(setq org-agenda-log-mode-items (quote (clock)))

3.1.2 Using clock reports to summarize time spent

Billable time for clients are kept in separate org files.

To get a report of time spent on tasks for you simply visit the file and run an agenda clock report for the last month with F12 < a v m b R. This limits the agenda to this one file, shows the agenda for a full month, moves to last month, and generates a clock report. Just scroll down to the end of the agenda to see the report.

I export the agenda to a text file with C-x C-w XYZ.txt so I can cut and paste the report and save it as supporting information with the invoice.

My agenda org clock report settings show 2 levels of detail and do not show links so that they are easier to cut and paste into other applications.

;; Agenda clock report parameters (no links, 2 levels deep)
(setq org-agenda-clockreport-parameter-plist (quote (:link nil :maxlevel 2)))

I used to have a monthly clock report dynamic block in each project org file and manually updated them at the end of my billing cycle. I used this as the basis for billing my clients for time spent on their projects. I found updating the dynamic blocks fairly tedious when you have more than a couple of files for the month.

I have since moved to using agenda clock reports shortly after that feature was added. I find this much more convenient. The data isn’t normally for consumption by anyone else so the format of the agenda clock report format is great for my use-case.

3.2 Task Estimates and column view

Estimating how long tasks take to complete is a difficult skill to master. Org-mode makes it easy to practice creating estimates for tasks and then clock the actual time it takes to complete.

By repeatedly estimating tasks and reviewing how your estimate relates to the actual time clocked you can tune your estimating skills.

3.2.1 Creating a task estimate with column mode

I use properties and column view to do project estimates.

I set up column view globally with the following headlines

; Set default column view headings: Task Effort Clock_Summary
(setq org-columns-default-format "%80ITEM(Task) %10Effort(Effort){:} %10CLOCKSUM")

This makes column view show estimated task effort and clocked times side-by-side which is great for reviewing your project estimates.

A property called Effort records the estimated amount of time a given task will take to complete. The estimate times I use are one of:

  • 10 minutes
  • 30 minutes
  • 1 hour
  • 2 hours
  • 3 hours
  • 4 hours
  • 5 hours
  • 6 hours
  • 7 hours
  • 8 hours

These are stored for easy use in column mode in the global property Effort_ALL.

; global Effort estimate values
(setq org-global-properties (quote (("Effort_ALL" . "0:10 0:30 1:00 2:00 3:00 4:00 5:00 6:00 7:00 8:00"))))

To create an estimate for a task or subtree start column mode with C-c C-x C-c and collapse the tree with c. This shows a table overlayed on top of the headlines with the task name, effort estimate, and clocked time in columns.

With the cursor in the Effort column for a task you can easily set the estimated effort value with the quick keys 1 through 9.

After setting the effort values exit column mode with q.

3.2.2 Saving your estimate

For fixed price jobs where you provide your estimate to a client, then work to complete the project it is useful to save the original estimate that is provided to the client.

Save your original estimate by creating a dynamic clock report table at the top of your estimated project subtree. Entering C-c C-x i RET inserts a clock table report with your estimated values and any clocked time to date.

Original Estimate
#+BEGIN: columnview :hlines 1 :id local
| Task                        | Estimated Effort | CLOCKSUM |
| ** TODO Project to estimate |             5:40 |          |
| *** TODO Step 1             |             0:10 |          |
| *** TODO Step 2             |             0:10 |          |
| *** TODO Step 3             |             5:10 |          |
| **** TODO Step 3.1          |             2:00 |          |
| **** TODO Step 3.2          |             3:00 |          |
| **** TODO Step 3.3          |             0:10 |          |
| *** TODO Step 4             |             0:10 |          |

I normally delete the #+BEGIN: and #+END: lines from the original table after providing the estimate to the client to ensure I don’t accidentally update the table by hitting C-c C-c on the #+BEGIN: line.

Saving the original estimate data makes it possible to refine the project tasks into subtasks as you work on the project without losing the original estimate data.

3.2.3 Reviewing your estimate

Column view is great for reviewing your estimate. This shows your estimated time value and the total clock time for the project side-by-side.

Creating a dynamic clock table with C-c C-x i RET is a great way to save this project review if you need to make it available to other applications.

C-c C-x C-d also provides a quick summary of clocked time for the current org file.

4 Tags

Tasks can have any number of arbitrary tags. Tags are used for:
  • filtering todo lists and agenda views
  • providing context for tasks
  • tagging notes
  • tagging phone calls
  • tagging tasks to be refiled
  • tagging tasks in a WAITING state because a parent task is WAITING
  • tagging cancelled tasks because a parent task is CANCELLED
  • preventing export of some subtrees when publishing

I use tags mostly for filtering in the agenda. This means you can find tasks with a specific tag easily across your large number of org-mode files.

Some tags are mutually exclusive. These are defined in a group so that only one of the tags can be applied to a task at a time (disregarding tag inheritance). I use these types for tags for applying context to a task. (Work tasks have an @office tag, and are done at the office, Farm tasks have an @farm tag and are done at the farm – I can’t change the oil on the tractor if I’m not at the farm… so I hide these and other tasks by filtering my agenda view to only @office tasks when I’m at the office.)

Tasks are grouped together in org-files and a #+FILETAGS: entry applies a tag to all tasks in the file. I use this to apply a tag to all tasks in the file. My file creates a NORANG file tag so I can filter tasks in the agenda in the file easily.

4.1 Tags

Here are my tag definitions with associated keys for filtering in the agenda views.

The startgroup - endgroup (@XXX) tags are mutually exclusive - selecting one removes a similar tag already on the task. These are the context tags - you can’t be in two places at once so if a task is marked with @farm and you add @office then the @farm tag is removed automagically.

The other tags QUOTE .. CANCELLED are not mutually exclusive and multiple tags can appear on a single task. Some of those tags are created by todo state change triggers. The shortcut key is used to add or remove the tag using C-c C-q or to apply the task for filtering on the agenda.

I have both FARM and @farm tags. FARM is set by a FILETAGS entry and just gives me a way to filter anything farm related. The @farm tag signifies that the task as to be done at the farm. If I have to call someone about something that would have a FARM tag but I can do that at home on my lunch break. I don’t physically have to be at the farm to make the call.

; Tags with fast selection keys
(setq org-tag-alist (quote ((:startgroup)
                            ("@errand" . ?e)
                            ("@office" . ?o)
                            ("@home" . ?h)
                            ("@farm" . ?f)
                            ("PHONE" . ?P)
                            ("QUOTE" . ?q)
                            ("WAITING" . ?w)
                            ("FARM" . ?F)
                            ("HOME" . ?H)
                            ("ORG" . ?O)
                            ("NORANG" . ?N)
                            ("crypt" . ?c)
                            ("MARK" . ?M)
                            ("NOTE" . ?n)
                            ("CANCELLED" . ?C))))

; Allow setting single tags without the menu
(setq org-fast-tag-selection-single-key (quote expert))

; For tag searches ignore tasks with scheduled and deadline dates
(setq org-agenda-tags-todo-honor-ignore-options t)

4.2 Filetags

Filetags are a convenient way to apply one or more tags to all of the headings in a file.

Filetags look like this:


I have the following #+FILETAGS: entries in my org-mode files:

4.2.1 Non-work related org-mode files


4.2.2 Work related org-mode files

File Tags NORANG @office ABC @office XYZ @office ABC DEF @office ABC KKK @office YYY @office

4.2.3 Refile tasks

File Tags REFILE

4.3 Trigger Tags

The following tags are automatically added or removed by todo state triggers described previously in *ToDo state triggers
  • NEXT

5 Handling Notes

Notes are little gems of knowledge that you come across during your day. They are just like tasks except there is nothing to do (except learn and memorize the gem of knowledge). Unfortunately there are way too many gems to remember and my head explodes just thinking about it.

org-mode to the rescue!

Often I’ll find some cool feature or thing I want to remember while reading the org-mode and git mailing lists in Gnus. To create a note I use my note capture template C-M-r n, type a heading for the note and C-c C-c to save it. The only other thing to do is to refile it (later) to the appropriate project file.

I have an agenda view just to find notes. Notes are refiled to an appropriate project file and task. If there is no specific task it belongs to it goes to the catchall * Notes task. I generally have a catchall notes task in every project file. Notes are created with a NOTE tag already applied by the capture template so I’m free to refile the note anywhere. As long as the note is in a project file that contributes to my agenda (ie. in org-agenda-files) then I can find the note back easily with my notes agenda view by hitting the key combination F12 N. I’m free to limit the agenda view of notes using standard agenda tag filtering.

Short notes with a meaningful headline are a great way to remember technical details without the need to actually remember anything - other than how to find them back when you need them using F12 N.

Notes that are project related and not generally useful can be archived with the project and removed from the agenda when the project is removed.

So my org notes go in and my git notes go in both under the * Notes task. I’ll forever be able to find those. A note about some work project detail I want to remember with the project is filed to the project task under the appropriate work org-mode file and eventually gets removed from the agenda when the project is complete and archived. * Handling Phone Calls

Phone calls are interruptions and I use capture mode to deal with these. Most of the heavy lifting for phone calls is done by capture mode. I use a special capture template for phone calls but activate it with a custom key binding f9-p. I’ve removed my phone capture mode template from my regular templates since I always use the bh/phone-call function mapped to f9-p to invoke the capture template. The definition of this template is now local to the function.

f9 p prompts for who is calling and looks up the entered name in my bbdb database with completion. The capture template is then filled in with the appropriate contact data and capture mode starts the clock using the :clock-in t setting in the template.

Here is my set up for phone calls. I would like to thank Gregory J. Grubbs for the bbdb lookup functions.

; Set f9-p to prompt for who is calling and preload the capture template
(global-set-key (kbd "<f9> p") 'bh/phone-call)

;; Phone capture template handling with BBDB lookup
;; modified from the original code by Gregory J. Grubbs
(defvar gjg/capture-phone-record nil
  "Either BBDB record vector, or person's name as a string, or nil")

(defun bh/phone-call ()
  (let* ((myname (completing-read "Who is calling? " (bbdb-hashtable) 'bbdb-completion-predicate 'confirm))
         (my-bbdb-name (if (> (length myname) 0) myname nil)))
    (setq gjg/capture-phone-record
          (if my-bbdb-name
              (first (or (bbdb-search (bbdb-records) my-bbdb-name nil nil)
                         (bbdb-search (bbdb-records) nil my-bbdb-name nil)))
    (other-window 1)
    (let ((org-capture-templates '(("P" "Phone" entry (file "~/git/org/") "* TODO Phone %(gjg/bbdb-name) - %(gjg/bbdb-company)               :PHONE:\n  %U\n  %?" :clock-in t :clock-resume t))))

(defun gjg/bbdb-name ()
  "Return full name of saved bbdb record, or empty string - for use in Capture templates"
  (if (and gjg/capture-phone-record (vectorp gjg/capture-phone-record))
      (concat "[[bbdb:"
              (bbdb-record-name gjg/capture-phone-record) "]["
              (bbdb-record-name gjg/capture-phone-record) "]]")

(defun gjg/bbdb-company ()
  "Return company of saved bbdb record, or empty string - for use in Capture templates"
  (if (and gjg/capture-phone-record (vectorp gjg/capture-phone-record))
      (or (bbdb-record-company gjg/capture-phone-record) "")

6 GTD stuff

Most of my day is deadline/schedule driven. I work off of the agenda first and then pick items from the todo lists as outlined in *What do I work on next

6.1 Weekly Review Process

The first day of the week (usually Monday) I do my weekly review. I keep a list like this one to remind me what needs to be done.

To keep the agenda fast I set

(setq org-agenda-ndays 1)

so only today’s date is shown by default. I only need the weekly view during my weekly review and this keeps my agenda generation fast.

I have a recurring task which keeps my weekly review checklist handy. This pops up as a reminder on Monday’s. This week I’m doing my weekly review on Tuesday since Monday was a holiday.

** NEXT Weekly Review [0/5]
   SCHEDULED: <2009-05-18 Mon ++1w> 

   What to review:

    - [ ] Check follow-up folder
    - [ ] Review new tasks                                  F12-r
      - if it takes less than 5 minutes just do it
      - otherwise assign an estimated time and file it somewhere
      - Refile billable work to appropriate location
    - [ ] Check for stuck projects and add next tasks       F12-#         
    - [ ] Review tasks                                      F12 t
      - [ ] Waiting tasks                                         / W
      - [ ] Next Tasks                                      F12 n
        - Move NEXT tag to subtasks or remove as required
    - [ ] Make plan for the week (out of NEXT tasks)
      - schedule important items onto the agenda
      - [ ] Review weekly plan                              F12 a v w


    - start work
      - daily agenda first - knock off items
        - complete them or adjust deadline warning days appropriately
      - when agenda is empty - work on next tasks

The first item [ ] Check follow-up folder makes me pull out the paper file I dump stuff into all week long - things I need to take care of but are in no particular hurry to deal with. Stuff I get in the mail etc that I don’t want to deal with now. I just toss it in my Follow-Up folder in the filing cabinet and forget about it until the weekly review.

I go through the folder and weed out anything that needs to be dealt with. After that everything else is in org-mode. I tend to schedule tasks onto the agenda for the coming week so that I don’t spend lots of time trying to find what needs to be worked on next.

This works for me. You’re mileage may vary ;)

6.2 Project definition and finding stuck projects

I’m using a new lazy project definition to mark tasks as projects. This requires zero effort from me. Any task with a subtask using a todo keyword is a project. Period.

Projects are ‘stuck’ if they have no subtask with a NEXT todo keyword task defined.

Org-mode stuck projects lists projects that have no NEXT task defined. I normally review these in my weekly review and assign a NEXT task to all projects to clear the stuck project list. This helps to keep projects moving forward.

I use a custom agenda view that overrides the default org-stuck-projects definition to find stuck projects.

The stuck project view is available with F12 #.

I have the following helper functions defined for projects. These are used by agenda views.

(defun bh/is-project-p ()
  "Any task with a todo keyword subtask"
  (let ((has-subtask)
        (subtree-end (save-excursion (org-end-of-subtree t))))
      (forward-line 1)
      (while (and (not has-subtask)
                  (< (point) subtree-end)
                  (re-search-forward "^\*+ " subtree-end t))
        (when (member (org-get-todo-state) org-todo-keywords-1)
          (setq has-subtask t))))

(defun bh/skip-non-stuck-projects ()
  "Skip trees that are not stuck projects"
  (let* ((subtree-end (save-excursion (org-end-of-subtree t)))
         (has-next (save-excursion
                     (forward-line 1)
                     (and (< (point) subtree-end)
                          (re-search-forward "^\\*+ NEXT " subtree-end t)))))
    (if (and (bh/is-project-p) (not has-next))
        nil ; a stuck project, has subtasks but no next task

(defun bh/skip-non-projects ()
  "Skip trees that are not projects"
  (let* ((subtree-end (save-excursion (org-end-of-subtree t))))
    (if (bh/is-project-p)

(defun bh/skip-projects ()
  "Skip trees that are projects"
  (let* ((subtree-end (save-excursion (org-end-of-subtree t))))
    (if (bh/is-project-p)

6.2.1 Anothe rhead with text

and here’s text for this head and somemore text and more

7 Archiving

just a line

7.1 Archiving Subtrees

My normal archiving procedure is to move entire subtrees to a separate archive file for the project. Task subtrees in get archived to FILE.org_archive using the a y command in the agenda.

I archive entire projects and subtrees into a single forever-growing file. My archive files are huge but so far I haven’t found a need to split them by year (or decade) :)

All of my tasks to archive start at level 2. I use an agenda custom command F12 A to list candidate tasks for archiving. My normal sequence is F12 A followed by repeated n SPC in the agenda to display the task candidate for archiving. If I closed it over a month ago I archive it. If it was closed this month or last month I skip it with n SPC. I do this repeatedly for all tasks in the list and then I’m done archiving until next month.

Archiving is easy. When I find a candidate I can archive I just do a y SPC to archive the current task and display the next candidate task. Then I’m back to n SPC if I’m skipping this next candidate task or a y SPC to archive it. Rinse and repeat.

I actually use a keyboard macro to do the archiving to save a few keystrokes. In the agenda view when I find the first task to archive I do C-x ( a y SPC C-x ). For the next task to archive I just do C-x e to repeat the macro and if there are multiple adjacent tasks to archive I just hit e again for each sequential task to archive to repeat the archive macro. I still use n SPC to skip over tasks that should not be archived.

I used to archive by visiting one file at a time and doing a tags match for LEVEL=2 – using the agenda does all of my files in org-agenda-files much more efficiently.

7.2 Archive Setup

Each of my level 1 tasks has a property which specifies where level 2 tasks under that heading should be archived. This is done with the ARCHIVE property as specified in the Org File Structure.

The following setting ensures that task states are untouched when they are archived. This makes it possible to archive tasks that are not marked DONE.

(setq org-archive-mark-done nil)

7.3 Archive Tag - Hiding Information

The only time I set the ARCHIVE tag on a task is to prevent it from opening by default because it has tons of information I don’t really need to look at on a regular basis. I can open the task with C-TAB if I need to see the gory details (like a huge table of data related to the task) but normally I don’t need that information displayed.

7.4 When to Archive

Archiving monthly works well for me. I keep completed tasks around for at least two months before archiving them. This keeps current clocking information for the last 30 to 60 days out of the archives. This keeps my files that contribute to the agenda fairly current (this month, and last month, and anything that is unfinished). I only rarely visit tasks in the archive when I need to pull up ancient history for something.

Archiving keeps my main working files clutter-free. If I ever need the detail for the archived tasks they are available in the appropriate archive file.

8 Publishing

I don’t do a lot of publishing for other people but I do keep a set of private client system documentation online. Most of this documentation is a collection of notes exported to HTML.

Almost everything at is generated by publishing org-files. The notable exception to that is the index page which is currently automatically generated from a Python script based on the HTML files that exist in the document directory.

It is supposed to be possible to generate index files from org-mode but I’ve never spent the time to figure that out since I already have a working index page in place.

Org-mode can export to a variety of publishing formats including (but not limited to)

  • ASCII (plain text - but not the original org-mode file)
  • HTML
  • LaTeX
  • Docbook which enables getting to lots of other formats like ODF, XML, etc
  • PDF via LaTeX or Docbook
  • iCal

I haven’t begun the scratch the surface of what org-mode is capable of doing. My main use case for org-mode publishing is just to create HTML documents for viewing online conveniently. Someday I’ll get time to try out the other formats when I need them for something.

8.1 org-babel Setup

Now I’ve discovered org-babel and how easy it is to generate decent graphics using ditta and graphviz.

The setup is really easy. ditaa is provided with the org-mode source. You’ll have to install the graphviz package for your system.

 (setq org-ditaa-jar-path "~/java/ditaa0_6b.jar")

 (add-hook 'org-babel-after-execute-hook 'org-display-inline-images)

 (setq org-babel-load-languages (quote ((emacs-lisp . t)
                                        (dot . t)
                                        (ditaa . t)
                                        (R . t)
                                        (python . t)
                                        (ruby . t)
                                        (gnuplot . t)
                                        (clojure . t)
                                        (sh . t))))

; Do not prompt to confirm evaluation
; This may be dangerous - make sure you understand the consequences
; of setting this -- see the docstring for details
(setq org-confirm-babel-evaluate nil)

Now you just create a begin-src block for the appropriate tool, edit the text, and build the pictures with C-c C-c. After evaluating the block results are displayed. You can toggle display of inline images with C-c C-x C-v

8.2 Playing with ditaa

ditaa is a great tool for quickly generating graphics to convey ideas and ditaa is included with org-mode! All of the graphics in this document are automatically generated by org-mode using plain text source.

Artist mode makes it easy to create boxes and lines for ditaa graphics.

The following graphic is one example of what you can do easily with ditaa:


#+begin_src ditaa :file communication.png :cmdline -r -s 0.8
        +-----------+        +---------+  
        |    PLC    |        |         |                
        |  Network  +<------>+   PLC   +<---=---------+ 
        |    cRED   |        |  c707   |              | 
        +-----------+        +----+----+              | 
                                  ^                   | 
                                  |                   | 
                                  |  +----------------|-----------------+
                                  |  |                |                 |
                                  v  v                v                 v
          +----------+       +----+--+--+      +-------+---+      +-----+-----+       Windows clients
          |          |       |          |      |           |      |           |      +----+      +----+
          | Database +<----->+  Shared  +<---->+ Executive +<-=-->+ Operator  +<---->|cYEL| . . .|cYEL|
          |   c707   |       |  Memory  |      |   c707    |      | Server    |      |    |      |    |
          +--+----+--+       |{d} cGRE  |      +------+----+      |   c707    |      +----+      +----+
             ^    ^          +----------+             ^           +-------+---+
             |    |                                   |                        
             |    +--------=--------------------------+                    
    |                 |                                                         
    | Millwide System |            -------- Data ---------                      
    | cBLU            |            --=----- Signals ---=--                      

becomes this!

    +-----------+        +---------+  
    |    PLC    |        |         |                
    |  Network  +<------>+   PLC   +<---=---------+ 
    |    cRED   |        |  c707   |              | 
    +-----------+        +----+----+              | 
                              ^                   | 
                              |                   | 
                              |  +----------------|-----------------+
                              |  |                |                 |
                              v  v                v                 v
      +----------+       +----+--+--+      +-------+---+      +-----+-----+       Windows clients
      |          |       |          |      |           |      |           |      +----+      +----+
      | Database +<----->+  Shared  +<---->+ Executive +<-=-->+ Operator  +<---->|cYEL| . . .|cYEL|
      |   c707   |       |  Memory  |      |   c707    |      | Server    |      |    |      |    |
      +--+----+--+       |{d} cGRE  |      +------+----+      |   c707    |      +----+      +----+
         ^    ^          +----------+             ^           +-------+---+
         |    |                                   |                        
         |    +--------=--------------------------+                    
|                 |                                                         
| Millwide System |            -------- Data ---------                      
| cBLU            |            --=----- Signals ---=--                      

8.3 Playing with graphviz

Graphviz is another great tool for creating graphics in your documents.


#+begin_src dot :file gv01.png :cmdline -Kdot -Tpng
digraph G {
  edge [dir=both]
  plcnet [shape=box, label="PLC Network"]
  subgraph cluster_wrapline {
    label="Wrapline Control System"
    subgraph {
    sharedmem [style=filled, fillcolor=lightgrey, shape=box]
    edge[style=dotted, dir=none]
    exec -> opserver
    exec -> db
    plc -> exec
    edge [style=line, dir=both]
    exec -> sharedmem
    sharedmem -> db
    plc -> sharedmem
    sharedmem -> opserver
  plcnet -> plc [constraint=false]
  millwide [shape=box, label="Millwide System"]
  db -> millwide

  subgraph cluster_opclients {
    label="Operator Clients"
    client1 -> client2 -> client3 [constraint=false]
    opserver -> client1
    opserver -> client2
    opserver -> client3

becomes this!

digraph G {
  edge [dir=both]
  plcnet [shape=box, label="PLC Network"]
  subgraph cluster_wrapline {
    label="Wrapline Control System"
    subgraph {
    sharedmem [style=filled, fillcolor=lightgrey, shape=box]
    edge[style=dotted, dir=none]
    exec -> opserver
    exec -> db
    plc -> exec
    edge [style=line, dir=both]
    exec -> sharedmem
    sharedmem -> db
    plc -> sharedmem
    sharedmem -> opserver
  plcnet -> plc [constraint=false]
  millwide [shape=box, label="Millwide System"]
  db -> millwide

  subgraph cluster_opclients {
    label="Operator Clients"
    client1 -> client2 -> client3 [constraint=false]
    opserver -> client1
    opserver -> client2
    opserver -> client3

The -Kdot is optional (defaults to dot) but you can substitute other graphviz types instead here (ie. twopi, neato, circo, etc).

8.4 Publishing Single Files

Org-mode exports the current file to one of the standard formats by invoking an export function. The standard key binding for this is C-c C-e followed by the key for the type of export you want.

This works great for single files or parts of files – if you narrow the buffer to only part of the org-mode file then you only get the narrowed detail in the export.

8.5 Publishing Projects

I mainly use publishing for publishing multiple files or projects. I don’t want to remember where the created export file needs to move to and org-mode projects are a great solution to this.

The website (and a bunch of other files that are not publicly available) are all created by editing org-mode files and publishing the project the file is contained in. This is great for people like me who want to figure out the details once and forget about it. I love stuff that Just Works(tm).

I have 3 main projects I use org-mode publishing for currently:

  • norang (website)
  • (website)
  • org files (which are selectively included by other websites)

Here’s my publishing setup:

; experimenting with docbook exports - not finished
(setq org-export-docbook-xsl-fo-proc-command "fop %s %s")
(setq org-export-docbook-xslt-proc-command "xsltproc --output %s /usr/share/xml/docbook/stylesheet/nwalsh/fo/docbook.xsl %s")
; Inline images in HTML instead of producting links to the image
(setq org-export-html-inline-images t)
; Do not use sub or superscripts - I currently don't need this functionality in my documents
(setq org-export-with-sub-superscripts nil)
; Use org.css from the norang website for export document stylesheets
(setq org-export-html-style-extra "<link rel=\"stylesheet\" href=\"\" type=\"text/css\" />")
(setq org-export-html-style-include-default nil)
; Do not generate internal css formatting for HTML exports
(setq org-export-htmlize-output-type (quote css))
; Export with LaTeX fragments
(setq org-export-with-LaTeX-fragments t)

; List of projects
; norang -
; doc    -
; org    - miscellaneous todo lists for publishing
(setq org-publish-project-alist
;  (norang website)
; norang-org are the org-files that generate the content
; norang-extra are images and css files that need to be included
; norang is the top-level project that gets published
      (quote (("norang-org"
               :base-directory "~/git/"
               :publishing-directory "/ssh:www-data@www:~/"
               :recursive t
               :section_numbers nil
               :table-of-contents nil
               :base-extension "org"
               :publishing-function org-publish-org-to-html
               :style-include-default nil
               :section-numbers nil
               :table-of-contents nil
               :style-include-default nil
               :style "<link rel=\"stylesheet\" href=\"norang.css\" type=\"text/css\">"
               :author-info nil
               :creator-info nil)
               :base-directory "~/git/"
               :publishing-directory "/ssh:www-data@www:~/"
               :base-extension "css\\|pdf\\|png\\|jpg\\|gif"
               :publishing-function org-publish-attachment
               :recursive t
               :author nil)
               :components ("norang-org" "norang-extra"))
;  (norang website)
; doc-org are the org-files that generate the content
; doc-extra are images and css files that need to be included
; doc is the top-level project that gets published
               :base-directory "~/git/"
               :publishing-directory "/ssh:www-data@www:~/"
               :recursive t
               :section_numbers nil
               :table-of-contents nil
               :base-extension "org"
               :publishing-function (org-publish-org-to-html org-publish-org-to-org)
               :plain-source t
               :htmlized-source t
               :style-include-default nil
               :style "<link rel=\"stylesheet\" href=\"/org.css\" type=\"text/css\">"
               :author-info nil
               :creator-info nil)
               :base-directory "~/git/"
               :publishing-directory "/ssh:www-data@www:~/"
               :base-extension "css\\|pdf\\|png\\|jpg\\|gif"
               :publishing-function org-publish-attachment
               :recursive t
               :author nil)
               :components ("doc-org" "doc-extra"))
; Miscellaneous pages for other websites
; org are the org-files that generate the content
               :base-directory "~/git/org/"
               :publishing-directory "/ssh:www-data@www:~/org"
               :recursive t
               :section_numbers nil
               :table-of-contents nil
               :base-extension "org"
               :publishing-function org-publish-org-to-html
               :style-include-default nil
               :style "<link rel=\"stylesheet\" href=\"/org.css\" type=\"text/css\">"
               :author-info nil
               :creator-info nil))))

; I'm lazy and don't want to remember the name of the project to publish when I modify
; a file that is part of a project.  So this function saves the file, and publishes
; the project that includes this file
; It's bound to C-S-F12 so I just edit and hit C-S-F12 when I'm done and move on to the next thing
(defun bh/save-then-publish ()

(global-set-key (kbd "C-s-<f12>") 'bh/save-then-publish)

The norang and doc projects publish directly into the webserver directory that serves that site. Publishing one of these projects exports all modified pages, generates images with ditaa, copies the resulting files to the webserver so that they are immediately available for viewing.

The site contains subdirectories with client documentation that are restricted access using Apache Basic authentication and I don’t create links to these sites from the publicly viewable pages. would show the index for any org files under ~/git/ if that is set up as a viewable website. I use most of the information myself but give access to clients if they are interested in the information/notes that I keep about their systems.

This works great for me - I know where my notes are and I can access them from anywhere on the internet. I’m also free to share notes with other people by simply giving them the link to the appropriate site.

All I need to remember to do is edit the appropriate org file and publish it with C-S-F12 – not exactly hard :)

9 Reminders

I use appt for reminders. It’s simple and unobtrusive – putting pending appointments in the status bar and beeping as 12, 9, 6, 3, and 0 minutes before the appointment is due.

Everytime the agenda is displayed (and that’s lots for me) the appointment list is erased and rebuilt from the current agenda details for today. This means everytime I reschedule something, add or remove tasks that are time related the appointment list is automatically updated the next time I look at the agenda.

9.1 Reminder Setup

; Erase all reminders and rebuilt reminders for today from the agenda
(defun bh/org-agenda-to-appt ()
  (setq appt-time-msg-list nil)

; Rebuild the reminders everytime the agenda is displayed
(add-hook 'org-finalize-agenda-hook 'bh/org-agenda-to-appt)

; This is at the end of my .emacs - so appointments are set up when Emacs starts

; Activate appointments so we get notifications
(appt-activate t)

; If we leave Emacs running overnight - reset the appointments one minute after midnight
(run-at-time "24:01" nil 'bh/org-agenda-to-appt)

10 Productivity Tools

This section is a miscellaneous collection of Emacs customizations that I use with org-mode so that it Works-For-Me(tm).

10.1 Yasnippets

Yasnippets is cool! You type the snippet name and TAB and yasnippet expands the name with the contents of the snippet text - substituting snippet variables as appropriate.

Yasnippet comes with lots of snippets for programming languages. So far I only use 1 snippet (block) for org-mode.

I downloaded and installed the unbundled version of yasnippet so that I can edit the predefined snippets. I unpacked the yasnippet software in my ~/.emacs.d/plugins directory, renamed yasnippet0.5.10 to yasnippet and added the following setup in my .emacs:

(add-to-list 'load-path (expand-file-name "~/.emacs.d/plugins"))

(require 'yasnippet)
(yas/load-directory "~/.emacs.d/plugins/yasnippet/snippets")

;; Make TAB the yas trigger key in the org-mode-hook and enable flyspell mode and autofill
(add-hook 'org-mode-hook
          (lambda ()
            ;; yasnippet
            (make-variable-buffer-local 'yas/trigger-key)
            (org-set-local 'yas/trigger-key [tab])
            (define-key yas/keymap [tab] 'yas/next-field-group)
            ;; flyspell mode for spell checking everywhere
            (flyspell-mode 1)
            ;; auto-fill mode on
            (auto-fill-mode 1)))

Here is the definition for the block snippet:

org-mode Yasnippet: ~/.emacs.d/plugins/yasnippet/snippets/text-mode/org-mode/block

#name : #+begin_...#+end_
  # --

I use this to create #+begin_* blocks like

  • #+begin_example
  • #+begin_ditaa
  • #+begin_dot
  • #+begin_src
  • etc.

Simply type block then TAB and it replaces the block text with the snippet contents. Then type src TAB emacs-lisp TAB and your snippet block is done.

Hit C-c SingeQuote(') and insert whatever emacs-lisp code you need. While in this block you’re in a mode that knows how to format and colourize emacs lisp code as you enter it which is really nice. C-c SingleQuote(') exits back to org-mode. This recognizes any emacs editing mode so all you have to do is enter the appropriate mode name for the block.

This is a great time saver.

10.2 Limit your view to what you are working on

There is more than one way to do this. Use what works for you.

10.2.1 Narrowing to a subtree with bh/org-todo

f5 and S-f5 are bound the functions for narrowing and widening the emacs buffer as follows:
(global-set-key (kbd "<f5>") 'bh/org-todo)

(defun bh/org-todo ()
  (org-show-todo-tree nil))

(global-set-key (kbd "<S-f5>") 'bh/widen)

(defun bh/widen ()

This makes it easy to hide all of the other details in your org-file temporarily by limiting your view to this task subtree. Tasks are folded and hilighted so that only tasks which are incomplete are shown.

I hit f5 a lot. This basically does a org-narrow-to-subtree and C-c C-v combination leaving the buffer in a narrowed state. I use S-f5 to widen back to the normal view.

10.2.2 Limiting the agenda to a subtree

C-c C-x < turns on the agenda restriction lock for the current subtree. This keeps your agenda focused on only this subtree. Alarms and notifications are still active outside the agenda restriction. C-c C-x > turns off the agenda restriction lock returning your agenda view back to normal.

I don’t normally use the agenda restriction lock. I normally want to see all work tasks which are in multiple files so agenda view filtering works better for me.

10.2.3 Limiting the agenda to a file

You can limit the agenda view to a single file in multiple ways.

You can use the agenda restriction lock C-c C-x < on the any line before the first heading to set the agenda restriction lock to this file only. This lock stays in effect until you remove it with C-c C-x >.

Another way is to invoke the agenda with F12 < a while visiting an org-mode file. This limits the agenda view to just this file. I occassionally use this to view a file not in my org-agenda-files in the agenda.

10.3 Tuning the Agenda Views

Various customizations affect how the agenda views show task details. This section shows each of the customizations I use in my workflow.

10.3.1 Highlight the current agenda line

The following code in my .emacs file keeps the current agenda line highlighted. This makes it obvious what task will be affected by commands issued in the agenda. No more acting on the wrong task by mistake!
;; Always hilight the current agenda line
(add-hook 'org-agenda-mode-hook '(lambda () (hl-line-mode 1)))

10.3.2 Remove tasks with dates from the global todo lists

Tasks with dates (SCHEDULED:, DEADLINE:, or active dates) show up in the agenda when appropriate. I use the following settings to remove these tasks from the global todo lists when they are too far in the future to be interesting now. The idea here is the agenda has date-related items and the global todo lists have everything else. Keeping tasks on one list only prevents having to review tasks more than once when browsing the lists.

Tasks with dates are scheduled into the future sometime and you don’t need to deal with them until the date approaches.

;; Keep tasks with dates off the global todo lists
(setq org-agenda-todo-ignore-with-date nil)

;; Allow deadlines which are due soon to appear on the global todo lists
(setq org-agenda-todo-ignore-deadlines (quote far))

;; Keep tasks scheduled in the future off the global todo lists
(setq org-agenda-todo-ignore-scheduled (quote future))

;; Remove completed deadline tasks from the agenda view
(setq org-agenda-skip-deadline-if-done t)

;; Remove completed scheduled tasks from the agenda view
(setq org-agenda-skip-scheduled-if-done t)

;; Remove completed items from search results
(setq org-agenda-skip-timestamp-if-done t)

10.3.3 Use the Diary for Holidays only

I don’t use the emacs Diary for anything but I like seeing the holidays on my agenda. This helps with planning for those days when you’re not supposed to be working.
(setq org-agenda-include-diary nil)
(setq org-agenda-diary-file "~/git/org/")

I don’t use a ~/diary file anymore. That is just there as a zero-length file to keep Emacs happy. I use org-mode’s diary functions instead. Inserting entries with i in the emacs agenda creates date entries in the ~/git/org/ file.

I include holidays from the calendar in my file as follows:

* Appointments
  :ARCHIVE:  %s_archive:* Appointments
** Holidays
   :Category: Holiday
** Some other Appointment

10.3.4 Searches include archive files

I keep a single archive file for each of my org-mode project files. This allows me to search the current file and the archive when I need to dig up old information from the archives.

I don’t need this often but it sure is handy on the occasions that I do need it.

;; Include agenda archive files when searching for things
(setq org-agenda-text-search-extra-files (quote (agenda-archives)))

10.3.5 Agenda view tweaks

The following agenda customizations control
  • display of repeating tasks
  • display of empty dates on the agenda
  • task sort order
  • start the agenda weekly view with today
  • display of the grid
  • habits at the bottom

I use a custom sorting function so that my daily agenda lists tasks in order of importance. Tasks on the daily agenda are listed in the following order:

  1. tasks with times at the top so they are hard to miss
  2. tasks for today (not scheduled or deadline tasks)
  3. late deadline tasks
  4. deadlines due today
  5. late scheduled items
  6. scheduled items for today
  7. pending deadlines (due soon)
  8. habits

The lisp for this isn’t particularly pretty but it works.

Here are the .emacs settings:

;; Show all future entries for repeating tasks
(setq org-agenda-repeating-timestamp-show-all t)

;; Show all agenda dates - even if they are empty
(setq org-agenda-show-all-dates t)

;; Sorting order for tasks on the agenda
(setq org-agenda-sorting-strategy
      (quote ((agenda habit-down time-up user-defined-up priority-down effort-up category-keep)
              (todo priority-down)
              (tags priority-down))))

;; Start the weekly agenda today
(setq org-agenda-start-on-weekday nil)

;; Disable display of the time grid
(setq org-agenda-time-grid
      (quote (nil "----------------"
                  (800 1000 1200 1400 1600 1800 2000))))

;; Display tags farther right
(setq org-agenda-tags-column -102)

;; Agenda sorting functions
(setq org-agenda-cmp-user-defined 'bh/agenda-sort)

(defun bh/agenda-sort (a b)
  "Sorting strategy for agenda items.
Late deadlines first, then scheduled, then non-late deadlines"
  (let (result num-a num-b)
     ; time specific items are already sorted first by org-agenda-sorting-strategy

     ; non-deadline and non-scheduled items next
     ((bh/agenda-sort-test 'bh/is-not-scheduled-or-deadline a b))

     ; late deadlines next
     ((bh/agenda-sort-test-num 'bh/is-late-deadline '< a b))

     ; deadlines for today next
     ((bh/agenda-sort-test 'bh/is-due-deadline a b))

     ; late scheduled items next
     ((bh/agenda-sort-test-num 'bh/is-scheduled-late '> a b))

     ; scheduled items for today next
     ((bh/agenda-sort-test 'bh/is-scheduled-today a b))

     ; pending deadlines last
     ((bh/agenda-sort-test-num 'bh/is-pending-deadline '< a b))

     ; finally default to unsorted
     (t (setq result nil)))

(defmacro bh/agenda-sort-test (fn a b)
  "Test for agenda sort"
    ; if both match leave them unsorted
    ((and (apply ,fn (list ,a))
          (apply ,fn (list ,b)))
     (setq result nil))
    ; if a matches put a first
    ((apply ,fn (list ,a))
     ; if b also matches leave unsorted
     (if (apply ,fn (list ,b))
         (setq result nil)
       (setq result -1)))
    ; otherwise if b matches put b first
    ((apply ,fn (list ,b))
     (setq result 1))
    ; if none match leave them unsorted
    (t nil)))

(defmacro bh/agenda-sort-test-num (fn compfn a b)
    ((apply ,fn (list ,a))
     (setq num-a (string-to-number (match-string 1 ,a)))
     (if (apply ,fn (list ,b))
           (setq num-b (string-to-number (match-string 1 ,b)))
           (setq result (if (apply ,compfn (list num-a num-b))
       (setq result -1)))
    ((apply ,fn (list ,b))
     (setq result 1))
    (t nil)))

(defun bh/is-not-scheduled-or-deadline (date-str)
  (and (not (bh/is-deadline date-str))
       (not (bh/is-scheduled date-str))))

(defun bh/is-due-deadline (date-str)
  (string-match "Deadline:" date-str))

(defun bh/is-late-deadline (date-str)
  (string-match "In *\\(-.*\\)d\.:" date-str))

(defun bh/is-pending-deadline (date-str)
  (string-match "In \\([^-]*\\)d\.:" date-str))

(defun bh/is-deadline (date-str)
  (or (bh/is-due-deadline date-str)
      (bh/is-late-deadline date-str)
      (bh/is-pending-deadline date-str)))

(defun bh/is-scheduled (date-str)
  (or (bh/is-scheduled-today date-str)
      (bh/is-scheduled-late date-str)))

(defun bh/is-scheduled-today (date-str)
  (string-match "Scheduled:" date-str))

(defun bh/is-scheduled-late (date-str)
  (string-match "Sched\.\\(.*\\)x:" date-str))

10.4 Handling blocked tasks

Blocked tasks are tasks that have subtasks which are not in a done todo state. Blocked tasks show up in a grayed font by default in the agenda.

To enable task blocking set the following variable:

(setq org-enforce-todo-dependencies t)

This setting prevents tasks from changing to DONE if any subtasks are still open. This works pretty well except for repeating tasks. I find I’m regularly adding TODO tasks under repeating tasks and not all of the subtasks need to be complete before the next repeat cycle.

You can override the setting temporarily by changing the task with C-u C-u C-u C-c C-t but I never remember that. I set a permanent property on the repeated tasks as follows:

* TODO New Repeating Task
  SCHEDULED: <2009-06-16 Tue +1w>
** TODO Subtask

This prevents the New Repeating Task from being blocked if some of the items under it are not complete.

Occassionally I need to complete tasks in a given order. Org-mode has a property ORDERED that enforces this for subtasks.

* TODO Some Task
** TODO Step 1
** TODO Step 2
** TODO Step 3

In this case you need to complete Step 1 before you can complete Step 2, etc. and org-mode prevents the state change to a done task until the preceding tasks are complete.

10.5 Org Task structure and presentation

This section describes various org-mode settings I use to control how tasks are displayed while I work on my org mode files.

10.5.1 Controlling display of leading stars on headlines

Org-mode has the ability to show or hide the leading starts on task headlines. It’s also possible to have headlines at odd levels only so that the stars and heading task names line up in sublevels.

I don’t hide leading stars - I want to see the heading levels explicitly. When I tried the hide leading stars setting I found myself typing ’ *’ when adding a new heading and then the font lock shows I messed up and created a list instead.

To make org show leading stars use

(setq org-hide-leading-stars nil)

10.5.2 Show headings at odd levels only or odd-even levels

I’ve converted my files between odd-levels-only and odd-even using the functions org-convert-to-odd-levels and org-convert-to-oddeven-levels functions a number of times. I ended up going back to odd-even levels to reduce the amount of leading whitespace on tasks. I didn’t find that lining up the headlines and tasks in odd-levels-only to be all that helpful.
(setq org-odd-levels-only nil)

10.5.3 Handling blank lines

Blank lines are evil :). They keep getting inserted in between headlines and I never want to see them in collapsed (contents) views. When I use TAB to fold (cycle) tasks I don’t want to see any blank lines.

The following setting hides all blank lines inside folded contents of a tasks:

(setq org-cycle-separator-lines 0)

I find extra blank lines in lists and headings a bit of a nuisance. To get a body after a list you need to include a blank line between the list entry and the body – and indent the body appropriately. Most of my lists have no body detail so I like the look of collapsed lists with no blank lines better.

The following setting prevents creating blank lines before list items and headings:

(setq org-blank-before-new-entry (quote ((heading)

10.5.4 Adding new tasks quickly without disturbing the current task content

To create new headings in a project file it is really convenient to use C-S-RET. This inserts a new headline. With the following setting
(setq org-insert-heading-respect-content t)

Org adds the new heading after the content of the current item. This lets you hit C-S-RET in the middle of an entry and the new heading is added after the body of the current entry.

10.5.5 Notes at the top

I enter notes for tasks with C-c C-z (or just z in the agenda). Changing tasks states also sometimes prompt for a note (e.g. moving to WAITING prompts for a note and I enter a reason for why it is waiting). These notes are saved at the top of the task so unfolding the task shows the note first.
(setq org-reverse-note-order nil)

10.5.6 Searching and showing results

Org-mode’s searching capabilities are really effective at finding data in your org files. C-c / / does a regular expression search on the current file and shows matching results in a collapsed view of the org-file.

I have org-mode show the hierarchy of tasks above the matched entries and also the immediately following sibling task (but not all siblings) with the following settings:

(setq org-show-following-heading t)
(setq org-show-hierarchy-above t)
(setq org-show-siblings nil)

This keeps the results of the search relatively compact and mitigates accidental errors by cutting too much data from your org file with C-k. Cutting folded data (including the …) can be really dangerous since it cuts text (including following subtrees) which you can’t see. For this reason I always show the following headline when displaying search results.

10.5.7 Editing and Special key handling

Org-mode allows special handling of the C-a, C-e, and C-k keys while editing headlines. I also use the setting that pastes (yanks) subtrees and adjusts the levels to match the task I am pasting to. See the docstring (C-h v org-yank-adjust-subtrees) for more details on each variable and what it does.
(setq org-special-ctrl-a/e t)
(setq org-special-ctrl-k t)
(setq org-yank-adjusted-subtrees t)

10.6 Attachments

Attachments are great for getting large amounts of data related to your project out of your org-mode files. Before attachments came along I was including huge blocks of SQL code in my org files to keep track of changes I made to project databases. This bloated my org file sizes badly.

Now I can create the data in a separate file and attach it to my project task so it’s easily located again in the future.

I set up org-mode to generate unique attachment IDs with org-id-method as follows:

(setq org-id-method (quote uuidgen))

Say you want to attach a file x.sql to your current task. Create the file data in /tmp/x.sql and save it.

Attach the file with C-c C-a a and enter the filename: x.sql. This generates a unique ID for the task and adds the file in the attachment directory.

   :Attachments: x.sql
   :ID:       f1d38e9a-ff70-4cc4-ab50-e8b58b2aaa7b

The attached file is saved in data/f1/d38e9a-ff70-4cc4-ab50-e8b58b2aaa7b/. Where it goes exactly isn’t important for me – as long as it is saved and retrievable easily. Org-mode copies the original file /tmp/x.sql into the appropriate attachment directory.

Tasks with attachments automatically get an ATTACH tag so you can easily find tasks with attachments with a tag search.

To open the attachment for a task use C-c C-a o. This prompts for the attachment to open and TAB completion works here.

The ID changes for every task header when a new ID is generated.

It’s possible to use named directories for attachments but I haven’t needed this functionality yet – it’s there if you need it.

I store my org-mode attachments with my org files in a subdirectory data. These are automatically added to my git repository along with any other org-mode changes I’ve made.

10.7 Deadlines and Agenda Visibility

Deadlines and due dates are a fact or life. By default I want to see deadlines in the agenda 30 days before the due date.

The following setting accomplishes this:

(setq org-deadline-warning-days 30)

This gives me plenty of time to deal with the task so that it is completed on or before the due date.

I also use deadlines for repeating tasks. If the task repeats more often than once per month it would be always bugging me on the agenda view. For these types of tasks I set an explicit deadline warning date as follows:

** TODO Pay Wages
   DEADLINE: <2009-07-01 Wed +1m -0d> 

This example repeats monthly and shows up in the agenda on the day it is due (with no prior warning). You can set any number of lead days you want on DEADLINES using -Nd where N is the number of days in advance the task should show up in the agenda. If no value is specified the default org-deadline-warning-days is used.

10.8 Exporting Tables to CSV

I generate org-mode tables with details of task specifications and record structures for some of my projects. My clients like to use spreadsheets for this type of detail.

It’s easy to share the details of the org-mode table by exporting in HTML but that isn’t easy for anyone else to work with if they need to edit data.

To solve this problem I export my table as comma delimited values (CSV) and then send that to the client (or read it into a spreadsheet and email the resulting spreadsheet file).

Org-mode can export tables as TAB or comma delimited formats. I set the default format to CSV with:

(setq org-table-export-default-format "orgtbl-to-csv")

Exporting to CSV format is the only one I use and this provides the default so I can just hit RETURN when prompted for the format.

To export the following table I put the cursor inside the table and hit M-x org-table-export which prompts for a filename and the format which defaults to orgtbl-to-csv from the setting above.

One Two Three
1 1 2
3 6 5
fred kpe mary
234.5 432.12 324.3

This creates the file with the following data


10.9 Visiting links

Links to emails, web pages, and other files are sprinkled all over my org files. The following setting control how org-mode handles opening the link.

(setq org-link-frame-setup ((vm . vm-visit-folder)
                            (gnus . org-gnus-no-new-news)
                            (file . find-file-other-window)))

I like to keep links in the same window so that I don’t end up with a ton of frames in my window manager. I normally work in a full-screen window and having links open in the same window just works better for me.

10.10 Logging stuff

Most of my logging is controlled by the global org-todo-keywords

My logging settings are set as follows:

(setq org-log-done (quote time))
(setq org-log-into-drawer t)

With org-todo-keywords set as

(setq org-todo-keywords
      (quote ((sequence "TODO(t)" "NEXT(n)" "WAITING(w@/!)" "SOMEDAY(s!)" "|" "DONE(d!/!)" "CANCELLED(c@/!)")
              (sequence "QUOTE(Q!)" "QUOTED(D!)" "|" "APPROVED(A@)" "EXPIRED(E@)" "REJECTED(R@)")
              (sequence "OPEN(O)" "|" "CLOSED(C)"))))

This adds a log entry whenever a task moves to any of the following states:

  • to or out of DONE status
  • to WAITING status (with a note) or out of WAITING status
  • to SOMEDAY status
  • to CANCELLED status (with a note) or out of CANCELLED status
  • to QUOTE status
  • to QUOTED status
  • to APPROVED status (with a note)
  • to EXPIRED status (with a note)
  • to REJECTED status (with a note)

I keep clock times and states in the LOGBOOK drawer to keep my tasks uncluttered. If a task is WAITING then the reason for why it is waiting is near the top of the LOGBOOK and unfolding the LOGBOOK drawer provides that information.

10.11 Limiting time spent on tasks

Org-mode has this great new feature for signalling alarms when the estimated time for a task is reached. I use this to limit the amount of time I spend on a task during the day.

As an example, I’ve been working on this document for over two months now. I want to get it finished but I can’t just work on it solely until it’s done because then nothing else gets done. I want to do a little bit every day but limit the total amount of time I spend documenting org-mode to an hour a day.

To this end I have a task

** NEXT Document my use of org-mode
   :Effort:   1:00

The task has an estimated effort of 1 hour and when I clock in the task it gives me a total in the mode-line like this

--:**   91% (2348,73) Git:master  (Org Fly yas Font)-----[0:35/1:00 (Document my use of org-mode)]-------

I’ve spent 35 minutes of my 1 hour so far today on this document and other help on IRC.

I set up an alarm so the Star Trek red alert klaxon goes off when the total estimated time is hit. (Yes I’m a Trekkie :) )

(setq org-clock-sound "/usr/local/lib/alert1.wav")

When the one hour time limit is hit the alarm sound goes off and a message states that I should be done working on this task. If I switch tasks and try to clock in this task again I get the sound each and every time I clock in the task. This nags me to go work on something else :)

You can use similar setups for repeated tasks. By default the last repeat time is recorded as a property when a repeating task is marked done. For repeating tasks the mode-line clock total counts since the last repeat time by default. This lets you accumulate time over multiple days and counts towards your estimated effort limit.

10.12 Habit Tracking

John Wiegley recently added support for Habit tracking to org-mode.

I have lots of habits (some bad) but I’d still like to improve and build new good habits. This is what habit tracking is for. It shows a graph on the agenda of how well you have been doing on developing your habits.

I have habits like:

  • Hand wash the dishes
  • 30 minute brisk walk
  • Clean the house

etc. and most of these need a push to get done regularly. Logging of the done state needs to be enabled for habit tracking to work.

A habit is just like a regular task except it has a special PROPERTY value setting and a special SCHEDULED date entry like this:

** TODO Update Org Mode Doc
   SCHEDULED: <2009-11-21 Sat .+7d/30d>
   [2009-11-14 Sat 11:45]
   :STYLE: habit

This marks the task as a habit and separates it from the regular task display on the agenda. When you mark a habit done it shows up on your daily agenda the next time based on the first interval in the SCHEDULED entry (.+1d)

The special SCHEDULED entry states that I want to do this every day but at least every 2 days. If I go 3 days without marking it DONE it shows up RED on the agenda indicating that I have been neglecting this habit.

The world isn’t going to end if you neglect your habits. You can hide and display habits quickly using the K key on the agenda.

These are my settings for habit tracking.

; Enable habit tracking (and a bunch of other modules)
(setq org-modules (quote (org-bbdb org-bibtex org-crypt org-gnus org-id org-info org-jsinfo org-habit org-inlinetask org-irc org-mew org-mhe org-protocol org-rmail org-vm org-wl org-w3m)))
; global STYLE property values for completion
(setq org-global-properties (quote (("STYLE_ALL" . "habit"))))
; position the habit graph on the agenda to the right of the default
(setq org-habit-graph-column 50)

During the day I’ll turn off the habit display in the agenda with K. This is a persistent setting and since I leave my Emacs running for days at a time my habit display doesn’t come back. To make sure I look at the habits daily I have the following settings to redisplay the habits in the agenda each day. This turns the habit display on again at 6AM each morning.

(run-at-time "06:00" 86400 '(lambda () (setq org-habit-show-habits t)))

10.13 Habits only log DONE state changes

I tend to keep habits under a level 1 task * Habits with a special logging property that only logs changes to the DONE state. This allows me to cancel a habit and not record a timestamp for it since that messes up the habit graph. Cancelling a habit just to get it off my agenda because it’s undoable (like get up before 6AM) should not mark the habit as done today. I only cancel habits that repeat every day.

My habit tasks look as follows - and I tend to have one in every org file that can have habits defined

* Habits
  :ARCHIVE:  %s_archive:* Habits

10.14 Auto revert mode

I use git to synchronize my org-mode files between my laptop and my workstation. This normally requires saving all the current changes, pushing to a bare repo, and fetching on the other system. After that I need to revert all of my org-mode files to get the updated information.

I used to use org-revert-all-org-buffers but have since discovered global-auto-revert-mode. With this setting any files that change on disk where there are no changes in the buffer automatically revert to the on-disk version.

This is perfect for synchronizing my org-mode files between systems.

(setq global-auto-revert-mode t)

10.15 Handling Encryption

I used to keep my encrypted data like account passwords in a separate GPG encrypted file. Now I keep them in my org-mode files with a special tag instead. Encrypted data is kept in the org-mode file that it is associated with.

org-crypt allows you to tag headings with a special tag crypt and org-mode can keep data in these headings encrypted when saved to disk. You decrypt the heading temporarily when you need access to the data and org-mode re-encrypts the heading as soon as you save the file.

I use the following setup for encryption:

(require 'org-crypt)
; Encrypt all entries before saving
(setq org-tags-exclude-from-inheritance (quote ("crypt")))
; GPG key to use for encryption
(setq org-crypt-key "F0B66B40")

M-x org-decrypt-entry will prompt for the passphrase associated with your encryption key and replace the encrypted data where the point is with the plaintext details for your encrypted entry. As soon as you save the file the data is re-encrypted for your key. Encrypting does not require prompting for the passphrase - that’s only for looking at the plain text version of the data.

I tend to have a single encrypted entry per file (like * Passwords). I prevent the crypt tag from using inheritance so that I don’t have encrypted data inside encrypted data. I found M-x org-decrypt-entries prompting for the passphrase to decrypt data over and over again (once per entry to decrypt) too inconvenient.

I leave my entries encrypted unless I have to look up data - I decrypt on demand and then save the file again to re-encrypt the data. This keeps the data in plain text as short as possible.

10.16 Speed Commands

There’s a new and exciting feature called org-speed-commands in the latest development version of org-mode.

Speed commands allow access to frequently used commands when on the beginning of a headline - similar to one-key agenda commands. Speed commands are user configurable and org-mode provides a good set of default commands.

I have the following speed commands set up in addition to the defaults. I don’t use priorities so I override the default settings for the 1, 2, and 3 keys.

(setq org-use-speed-commands t)
(setq org-speed-commands-user (quote (("0" . delete-window)
                                      ("1" . delete-other-windows)
                                      ("2" . split-window-vertically)
                                      ("3" . split-window-horizontally)
                                      ("h" . hide-other)
                                      ("k" . org-kill-note-or-show-branches)
                                      ("r" . org-reveal)
                                      ("s" . org-save-all-org-buffers)
                                      ("z" . org-add-note))))

The variable org-speed-commands-default sets a lot of useful defaults for speed command keys. The default keys I use the most are I and O for clocking in and out and t to change todo state.

10.17 Org Protocol

Org protocol is a great way to create capture notes in org-mode from other applications. I use this to create tasks to review interesting web pages I visit in Firefox.

I have a special capture template set up for org-protocol to use (set up with the w key).

My org-mode setup for org-protocol is really simple. It enables org-protocol and creates a single org-protocol capture template as described in Capture Templates.

(require 'org-protocol)

The bulk of the setup is in the Firefox application so that C-M-r on a page in Firefox will trigger the org-protocol capture template with details of the page I’m currently viewing in firefox.

I set up org-protocol in firefox as described in Keybindings for Firefox.

10.18 Add final newline when saving files

(setq require-final-newline t)

10.19 Insert inactive timestamps

I insert inactive timestamps when working on org-mode files. Normally I use this to track when a task was created.

For capture tasks the timestamp is in the capture template but for regular structure editing I normally want a clean outline without timestamps. I find this easier to work with when brainstorming and generating the outline structure for a project. For this reason I’ve turned off the hook I used to use to automatically insert a timestamp when creating headlines.

I have the following function bound to the key sequence f9 t to insert an inactive timestamp in the text on demand.

(defun bh/insert-inactive-timestamp ()
  (org-insert-time-stamp nil t t nil nil nil))

(global-set-key (kbd "<f9> t") 'bh/insert-inactive-timestamp)

10.20 Return follows links

The following setting make RET open links instead of inserting a new line. This setting is a love-hate relationship for me. When it first came out I immediately turned it off because I wanted to insert new lines in front of my links and RET would open the link instead which at the time I found extremely annoying. Now I’ve trained my fingers to do C-o instead for opening the line above the link. I find I’m hitting RET to visit links a lot more than opening lines before the link - so retraining my fingers was the right move for me.
(setq org-return-follows-link t)

10.21 Highlight clock when running overtime

The current clocking task is displayed on the modeline. If this has an estimated time and we run over the limit I make this stand out on the modeline by changing the background to red as follows
  ;; custom-set-faces was added by Custom.
  ;; If you edit it by hand, you could mess it up, so be careful.
  ;; Your init file should contain only one such instance.
  ;; If there is more than one, they won't work right.
 '(org-mode-line-clock ((t (:background "grey75" :foreground "red" :box (:line-width -1 :style released-button)))) t))

10.22 Meeting Notes

I take meeting notes with org-mode. I record meeting conversations in point-form using org-mode lists. If action items are decided on in the meeting I’ll denote them with a bullet and a TODO: or DONE: flag.

A meeting is a task and it is complete when the meeting is over. The body of the task records all of the interesting meeting details. If TODO items are created in the meeting I make separate TODO tasks from those.

I use the function bh/prepare-meeting-notes to prepare the meeting notes for emailing to the participants (in a fixed-width font like “Courier New”). As soon as the meeting is over the notes are basically ready for distribution – there’s not need to waste lots of time rewriting the minutes before they go out. I haven’t bothered with fancy HTML output – the content is more important than the style.

** TODO Sample Meeting
   - Attendees
     - [ ] Joe
     - [X] Larry
     - [X] Mary
     - [X] Fred
   - Joe is on vacation this week
   - Status Updates
     + Larry
       - did this
       - and that
       - TODO: Needs to follow up on this
     + Mary
       - got a promotion for her recent efforts
     + Fred
       - completed all his tasks 2 days early
       - needs more work
       - DONE: everything
** TODO Sample Meeting
   - Attendees
     - [ ] Joe
     - [X] Larry
     - [X] Mary
     - [X] Fred
   - Joe is on vacation this week
   - Status Updates
     + Larry
       - did this
       - and that
>>>>>>>> TODO: Needs to follow up on this
     + Mary
       - got a promotion for her recent efforts
     + Fred
       - completed all his tasks 2 days early
       - needs more work
>>>>>>>> DONE: everything

Here is the formatting function. Just highlight the region for the notes and it turns tabs into spaces, and highlights todo items. The resulting notes are in the kill buffer ready to paste to another application.

(defun bh/prepare-meeting-notes ()
  "Prepare meeting notes for email
   Take selected region and convert tabs to spaces, mark TODOs with leading >>>, and copy to kill ring for pasting"
  (let (prefix)
        (narrow-to-region (region-beginning) (region-end))
        (untabify (point-min) (point-max))
        (goto-char (point-min))
        (while (re-search-forward "^\\( *-\\\) \\(TODO\\|DONE\\): " (point-max) t)
          (replace-match (concat (make-string (length (match-string 1)) ?>) " " (match-string 2) ": ")))
        (goto-char (point-min))
        (kill-ring-save (point-min) (point-max))))))

10.23 Highlights persist after changes

I’m finding I use org-occur C-c / / a lot when trying to find details in my org-files. The following setting keeps the highlighted results of the search even after modifying the text. This allows me to edit the file without having to reissue the org-occur command to find the other matches in my file.
(setq org-remove-highlights-with-change nil)

10.24 Getting up to date org-mode info documentation

I use the org-mode info documentation from the git repository so I set 1300 up emacs to find the info files from git before the regular (out of date) system versions.
(add-to-list 'Info-default-directory-list "~/git/org-mode/doc")

10.25 Turn off prefer future dates

By default org-mode prefers dates in the future. This means that if today’s date is May 2 and you enter a date for April 30th (2 days ago) org-mode will jump to April 30th of next year. I find this annoying when I want to look at what happened last Friday since I have to specify the year.

To make org-mode prefer the current year when entering dates I set the following variable:

(setq org-read-date-prefer-future nil)

10.26 Automatically change list bullets

I take point-form notes during meetings. Having the same list bullet for every list level makes it hard to read the details when lists are indented more than 3 levels.

Org-mode has a way to automatically change the list bullets when you change list levels.

Current List Bullet Next indented list bullet
+ -
* -
1. -
1) -
(setq org-list-demote-modify-bullet (quote (("+" . "-")
                                            ("*" . "-")
                                            ("1." . "-")
                                            ("1)" . "-"))))

11 Using Git for Automatic History, Backups, and Synchronization

Editing folded regions of your org-mode file can be hazardous to your data. My method for dealing with this is to put my org files in a Git source repository. Yahoo My setup saves 2010-sep-20-desktop-stuff/random-pics/Santorini.jpg all of my org-files every hour and creates a commit with my changes automatically. This lets me go back in time and view the state of my org files for any given hour over the lifetime of the document. I’ve used this once or twice to recover data I accidentally removed while editing folded regions.

11.1 Automatic Hourly Commits

My Emacs setup saves all org buffers at 1 minute before the hour using the following code in my .emacs

(run-at-time "00:59" 3600 'org-save-all-org-buffers)

A cron job runs at the top of the hour to commit any changes just saved by the call to org-save-all-org-buffers above. I use a script to create the commits so that I can run it on demand to easily commit all modified work when moving from one machine to another.

crontab details:

0 * * * * ~/bin/ >/dev/null

11.1.1 ~/bin/

Here is the shell script I use to create a git commit for each of my org-repositories. This loops through multiple repositories and commits any modified files. I have the following org-mode repositories:

This script does not create empty commits - git only creates a commit if something was modified.

# Add org file changes to the repository

for REPO in $REPOS
    echo "Repository: $REPO"
    cd ~/git/$REPO
    # Remove deleted files
    git ls-files --deleted -z | xargs -0 git rm >/dev/null 2>&1
    # Add new files
    git add . >/dev/null 2>&1
    git commit -m "$(date)"

I use the following .gitignore file in my org-mode git repositories to keep export generated files out of my git repositories. If I include a graphic from some other source than ditaa or graphviz then I’ll add it to the repository manually. By default all PNG graphic files are ignored (since I assume they are produced by ditaa during export)


11.2 Git - Edit files with confidence

I use git in all of my directories where editing a file should be tracked.

This means I can edit files with confidence. I’m free to change stuff and break things because it won’t matter. It’s easy to go back to a previous working version or to see exactly what changed since the last commit. This is great when editing configuration files (such as apache webserver, bind9 DNS configurations, etc.)

I find this extremely useful where your edits might break things and having git tracking the changes means if you break it you can just go back to the previous working version easily. This is also true for package upgrades for software where the upgrade modifies the configuration files.

I have every version of my edits in a local git repository.

11.3 Git Repository synchronization

I acquired a Eee PC 1000 HE which now serves as my main road-warrior laptop replacing my 6 year old Toshiba Tecra S1.

I have a server on my LAN that hosts bare git repositories for all of my projects. The problem I was facing is I have to leave in 5 minutes and want to make sure I have up-to-date copies of everything I work on when I take it on the road (without Internet access).

To solve this I use a server with bare git repositories on it. This includes my org-mode repositories as well as any other git repositories I’m interested in.

Just before I leave I run the git-sync script on my workstation to update the bare git repositories and then I run it again on my Eee PC to update all my local repositories on the laptop. For any repositories that give errors due to non-fast-forward merges I manually merge as required and rerun git-sync until it reports no errors. This normally takes a minute to two to do. Then I grab my Eee PC and leave. When I’m on the road I have full up-to-date history of all my git repositories.

The git-sync script replaces my previous scripts with an all-in-one tool that basically does this:

  • for each repository on the current system
    • fetch objects from the remote
    • for each branch that tracks a remote branch and this is a first thing to do and a second one to do.
      • Check if the ref can be moved
        • fast-forwards if behind the remote repository and is fast-forwardable
        • Does nothing if ref is up to date
        • Pushes ref to remote repository if ref is ahead of remote repository and fast-forwardable
        • Fails if ref and remote have diverged

This automatically advances changes on my 35+ git repositories with minimal manual intervention. The only time I need to manually do something in a repository is when I make changes on my Eee PC and my workstation at the same time - so that a merge is required.

Here is the git-sync script


# Local bare repository name

# Display repository name only once
log_repo() {
  [ "x$lastrepo" == "x$repo" ] || {
    printf "\nREPO: ${repo}\n"

# Log a message for a repository
log_msg() {
  printf "  $1\n"

# fast-forward reference $1 to $syncrepo/$1
fast_forward_ref() {
  log_msg "fast-forwarding ref $1"
  current_ref=$(cat .git/HEAD)
  if [ "x$current_ref" = "xref: refs/heads/$1" ]
    # Check for dirty index
    files=$(git diff-index --name-only HEAD --)
    git merge refs/remotes/$syncrepo/$1
    git branch -f $1 refs/remotes/$syncrepo/$1

# Push reference $1 to $syncrepo
push_ref() {
  log_msg "Pushing ref $1"
  if ! git push --tags $syncrepo $1
    exit 1

# Check if a ref can be moved
#   - fast-forwards if behind the sync repo and is fast-forwardable
#   - Does nothing if ref is up to date
#   - Pushes ref to $syncrepo if ref is ahead of syncrepo and fastforwardable
#   - Fails if ref and $syncrop/ref have diverged
check_ref() {
  revlist1=$(git rev-list refs/remotes/$syncrepo/$1..$1)
  revlist2=$(git rev-list $1..refs/remotes/$syncrepo/$1)
  if [ "x$revlist1" = "x" -a "x$revlist2" = "x" ]
    # Ref $1 is up to date.
  elif [ "x$revlist1" = "x" ]
    # Ref $1 is behind $syncrepo/$1 and can be fast-forwarded.
    fast_forward_ref $1 || exit 1
  elif [ "x$revlist2" = "x" ]
    # Ref $1 is ahead of $syncrepo/$1 and can be pushed.
    push_ref $1 || exit 1
    log_msg "Ref $1 and $syncrepo/$1 have diverged."
    exit 1

# Check all local refs with matching refs in the $syncrepo
check_refs () {
  git for-each-ref refs/heads/* | while read sha1 commit ref
    git for-each-ref refs/remotes/$syncrepo/$ref | while read sha2 commit ref2
      if [ "x$sha2" != "x" -a "x$sha2" != "x" ]
        check_ref $ref || exit 1

# For all repositories under $reporoot
#   Check all refs matching $syncrepo and fast-forward, or push as necessary
#   to synchronize the ref with $syncrepo
#   Bail out if ref is not fastforwardable so user can fix and rerun
time {
  if find $reporoot -type d -name '*.git' | { 
      while read repo
        cd ${repo}
        upd=$(git remote update $syncrepo 2>&1 || retval=1)
        [ "x$upd" = "xFetching $syncrepo" ] || {
          printf "$upd\n"
        check_refs || retval=1
      exit $retval
    printf "\nAll done.\n"
    printf "\nFix and redo.\n"

exit $retval

12 Change History - What’s new

This document is created using the publishing features of

Org-mode version 7.01trans (release_7.01h.356.gcbc81)


GNU Emacs 23.2.1 (i486-pc-linux-gnu, GTK+ Version 2.20.0)
 of 2010-08-14 on raven, modified by Debian

The source for this document can be found as colorized HTML and plain text org file.

I try to update this document about once a month.

The change history for this document can be found at git://

Jump to Line
Something went wrong with that request. Please try again.