Assigning dependencies between tasks using org-edna

[novoid]

Hi,

Feature request

I'd like to wish for an extension of org-super-links to be a valuable tool for assigning dependencies between tasks using org-edna. This would improve project management to a great extend since it now requires manual effort and is improving an error prone situation.

I do think that this is also a valid approach for org-super-links since it relates to its very basic goal: linking headings. With this feature, it just adds different kind of links using the same query mechanism and also extending data within drawers.

• How the process should work from my perspective:
  1. manual step: Cursor is placed within a "source heading".
  2. manual step: Trigger function keyboard binding for this new function such as C-c s t (for "trigger")
     • OPTIONAL: with a given universal argument, this function is only operating/querying the current buffer for performance optimization and a high probable locality of reference.
  3. manual step: Function is querying for a "target heading" and user confirms (using helm, …)
     • Very similar to the current functionality of C-c s s (sl-link).
  4. automatically: Target heading gets a property set: :BLOCKER: ids($ID_OF_SOURCE_HEADING)
     • if existing :BLOCKER: property, the ids($ID_OF_SOURCE_HEADING) gets appended since there can be multiple ids()
     • OPTIONAL: create normal org-super-links-backlink that links back to the source heading as long as this feature request of org-edna is not implemented.
  5. automatically: Source heading gets a property set: :TRIGGER: ids($ID_OF_TARGET_HEADING) todo!(TODO) scheduled!(".")
     • if existing :TRIGGER: property, the new values get appended since there can be multiple sets of values, initiated by ids() or similar.
     • OPTIONAL: ask for the content of scheduled!("") which can be arbitrary date expressions such as "+2d", "mon", …
     • OPTIONAL: ask for the todo keyword which can be TODO, WAITING, STARTED and so forth using tab completion.
     • OPTIONAL: define default todo keyword in a defvar variable (I personally prefer "NEXT" in this context).

(FYI: I'd use all features marked as optional in my daily workflows.)

This way, the user is able to define a dependency between two tasks as easy and as quickly as possible. Depending tasks do feature a backlink via :BLOCKER: properties. Those backlinks do have a different format than existing org-super-links backlinks. This is due to the format definition of org-edna. Maybe an org-super-links backlink can be automatically added as well via preferences. I'd use both links (org-edna-dependency-link + org-super-links-backlinks) as long as this feature request of org-edna is not implemented.

Example for org-edna syntax

A minimal example for plain org-edna syntax is shown to understand its syntax, not demonstrating all the properties mentioned in the algorithm above.

** NEXT my first task
SCHEDULED: <2020-07-11 Sat>
:PROPERTIES:
:ID:       2020-07-09-my-first-task
:TRIGGER: ids(2020-07-09-my-next-task) todo!(NEXT) scheduled!(".")   ids(2020-07-09-another-task) todo!(NEXT) scheduled!("++2d")
:END:

** my next task
:PROPERTIES:
:ID:       2020-07-09-my-next-task
:BLOCKER: ids(2020-07-09-my-first-task) ids(a-totally-different-task-not-shown-here)
:END:

** the task after the next task
:PROPERTIES:
:ID:       2020-07-09-another-task
:BLOCKER: ids(2020-07-09-my-first-task)
:END:

In that example, when the user is setting the first task to DONE, the "next task" gets the todo keyword "NEXT" and a SCHEDULED for the very same day. The task after the next task gets the todo keyword "NEXT" and is scheduled for two days after today.

The :BLOCKER: properties are not necessary for org-edna in this example. They are redundant. However, I'd prefer to have them because this way, any task has clearly defined back-references that work with this org-edna syntax.

Please note that in contrast to org-depend, tasks get their todo keyword only when they are "active". My guess is that this is also speeding up performance of large agenda setups.

You can find another org-edna example project at this issue.

Benefits

Using the proposed feature, all :TRIGGER: and :BLOCKER: properties can be set in this example. It only requires two invocations, both when being at the first heading. It is clearly visible that this helps a lot for optimizing manual effort and providing a full set of meta-data (back-references) while eliminating potential errors.

Using org-edna for defining dependencies like that has further benefits. The user does not need to hide blocked tasks via org-agenda-dim-blocked-tasks which I presume offers better performance. Blocked tasks simply do not necessarily feature a todo keyword at all, thus they aren't taken into account for agenda generation which is a plus.

Having the optional possibility to define arbitrary relative scheduled dates offer maximum flexibility when managing projects.

From my personal point of view, this is the way to do project management with dependencies.

[yantar92]

Partial implementation from my config. Feel free to reuse anywhere.

(use-package org-edna
  :bind (:map org-mode-map
	      ("C-c C-x M-p" . yant/org-set-preceding-task))
  :config
  (defun yant/org-set-preceding-task ()
    "Make task at point follow other task.

The current task will be marked WAITING and cannot be marked DONE
until the other task is completed.
Completing the other task will automatically set the current task to
NEXT and schedule it the same day."
    (interactive)
    (let ((uuid (org-id-prompt-id))
	  (cur-uuid (org-id-get-create)))
      (unless uuid (user-error "Did not get a uuid"))
      (org-todo "WAITING")
      (org-set-property "BLOCKER" (format "ids(%s)" uuid))
      (org-with-point-at (org-id-find uuid 'marker)
	(org-set-property "TRIGGER" (format "ids(%s) todo!(NEXT) scheduled!(\".\")" cur-uuid))))))

 (defun org-id-prompt-id ()
  "Prompt for the id during completion of att-id: link."
  (org-id-get-with-outline-path-completion '((org-agenda-files :maxlevel . 100))))

[novoid]

Thanks @yantar92 for your code!
From my perspective, this is a proper proof of concept. However, I do have some concerns with this approach:

• Defining the dependency from the WAITING task to the previous task, the user has to define dependencies from "backward", starting at the last one, going up. I personally would not choose it that way.
• The code is not using the fast helm search functionality. Instead, I had to wait for the index to be updated since I usually apply the function on brand new headings. Updating my target cache takes approximately 30 seconds due to my large Org files. Afterwards, I had to select the path to the target heading manually (with TAB-completion) which is tedious.
• The code ignores the mechanism of org-super-links with sl-pre-link-hook. Instead, it generates a random ID which is not human readable.

Please do not get me wrong: this might be "the solution" for many use-cases. Unfortunately, for the reasons above, the concept does seem to have some issues in my setup.

[yantar92]

Defining the dependency from the WAITING task to the previous task, the user has to define dependencies from "backward", starting at the last one, going up. I personally would not choose it that way.

Depends on your workflow. I wrote this code for very specific case when I capture a bookmark to a video from a video series. If I did not watch previous videos I can quickly make the newly captured task hidden from my agenda until I finish watching previous video in series.

Rewriting the code to create dependency of the task at point is not hard though.

...
(let ((uuid (org-id-prompt-id))
	  (cur-uuid (org-id-get-create)))
      (unless uuid (user-error "Did not get a uuid"))
     (org-with-point-at (org-id-find uuid 'marker)
      (org-todo "WAITING")
      (org-set-property "BLOCKER" (format "ids(%s)" cur-uuid)))
      
	(org-set-property "TRIGGER" (format "ids(%s) todo!(NEXT) scheduled!(\".\")" uuid)))))

The code is not using the fast helm search functionality. Instead, I had to wait for the index to be updated since I usually apply the function on brand new headings. Updating my target cache takes approximately 30 seconds due to my large Org files.

Unfortunatery helm-org-rifle and helm-org-ql never worked on my files for some reason. They just stop updating the results at some point. To use helm-org-*, you can simply replace org-id-prompt-id with helm super links version. All you need is a function that returns headline id.

The code ignores the mechanism of org-super-links with sl-pre-link-hook. Instead, it generates a random ID which is not human readable.

Then do not use (org-id-get-create). Replace it by whatever function you have that creates human-readable ID.

[toshism]

First of all thanks for the thorough write up on this. I think this is interesting and I like the idea, however I think it goes too far away from the purpose of org-super-links. I really would like to keep this small and as focused as possible.

I will look into making it as easy as possible to add functionality like this as an external package though, and maybe even look into creating one myself as I would like to try out org-edna.

[novoid]

I understand and I absolutely do respect your decision. I was very desperate since org-super-links is so close to this, that I saw a chance here.

maybe even look into creating one myself as I would like to try out org-edna.

As you already know: I'm better with concepts and workflows than with elisp: if you plan to implement said feature, I'm happy to contribute with testing, ideas, documentation and whatnot. I really do believe that this would add much value to project management with Org mode in general. EmacsConf is coming most probably this November and this would be a very very nice piece to demo there ... 😏

[toshism]

I've started to work on this a bit. Basically I am extracting out the searching functionality that matches headings into a separate package. This will provide a nice base and make it much easier for building other linkages between two headings, such as org-edna blockers or any other type of "link" really. Probably also make the org-ref links easier as going on in the discussion in #34. I hadn't really considered this when building org-super-links so it's a bit awkward currently.

I'll try to put a rough take on this concept up soon, and maybe a refactored branch in org-super-links to demonstrate the usage.

[toshism]

If anyone is interested in following along I've started toying with the idea for this.

• the extracted heading selection interface stuff org-linker
• the org-edna interface that uses it org-linker-edna
• and there is a branch here in org-super-links named org-linker-refactor that uses org-linker instead of the current built in logic

it's all kind of a mess right now, just started working out the idea, but it's there, and sort of works, if you are interested.