Feature request: Support for recurring events

[tmalsburg]

I'm using org-ql indirectly through org-upcoming-modeline, a very useful package that shows the next event in the agenda in the mode line as a reminder.

One limitation is that org-upcoming-modeline doesn't show recurring events. Example:

** Weekly meeting
  <2018-10-17 Wed 12:00-14:00 +1w>

For this event, I get a reminder for the first instance but none for the repetitions.

Is there a way to include the repetitions in the org-ql query results as well? I think this would be a tremendously useful extension.

[unhammer]

https://github.com/unhammer/org-upcoming-modeline/blob/27fb74f9e7480c40b95af2170f5d427fd21e0953/org-upcoming-modeline.el#L286-L308
along with
https://github.com/unhammer/org-upcoming-modeline/blob/27fb74f9e7480c40b95af2170f5d427fd21e0953/org-upcoming-modeline.el#L248-L258
is a workaround that just picks the first of recurring events, but it seems like something that would be for other org-ql users (e.g. https://github.com/legalnonsense/org-timed-alerts ).

Alternatively, an API for defining ones own selectors would work (my current workaround is quite hacky as you can see, since I can't really change org-ql--pre-process-query.) seems this already exists in org-ql master :)

Also, is there a ts-to-time function? Didn't find one. (And would the actual parsing of recurring strings belongs in ts, not org-ql? But then ts would have to have some representation of recurring events.)

[legalnonsense]

Here's my function (using TS) to update a repeated date until it is current. Maybe seeing my shoddy work will motivate someone to do it better.

https://github.com/legalnonsense/org-timed-alerts/blob/d3ce176f8fa68e21a673275e745a41c37390da1f/org-timed-alerts.el#L246-L265

[alphapapa]

@tmalsburg

Is there a way to include the repetitions in the org-ql query results as well? I think this would be a tremendously useful extension.

It would be useful, yes. In order to do it, you'd need a function to take an Org timestamp that repeats and return the next timestamp repetition after a certain date. Then you could call that function in the org-ql-select ACTION to return that timestamp.

Of course, that wouldn't be appropriate for the default org-ql-select action, but it could be a useful utility function to include in org-ql. I'd be happy to accept a PR for something like that.

@unhammer It looks like you've already worked that out! :)

Alternatively, an API for defining ones own selectors would work (my current workaround is quite hacky as you can see, since I can't really change org-ql--pre-process-query.) seems this already exists in org-ql master :)

Yes, I'm glad to see that it's being useful already. :)

Also, is there a ts-to-time function? Didn't find one.

I don't know what you mean by "time". Maybe you mean, e.g. (ts-unix (ts-now)) for a Unix timestamp?

Edit: Oh, I think you mean ts-internal, which does (apply #'encode-time (decode-time (ts-unix struct))). Apparently I omitted that from the documentation. I'll see about adding it. Thanks.

(And would the actual parsing of recurring strings belongs in ts, not org-ql? But then ts would have to have some representation of recurring events.)

If there were a universal way to represent them, yes, but since this is specific to Org, I think it belongs in org-ql.

@legalnonsense I like that. The suggestion I'd make is to calculate the offset and adjust it once rather than using a loop to adjust it repeatedly.

[unhammer]

> Also, is there a ts-to-time function? Didn't find one. I don't know what you mean by "time". Maybe you mean, e.g. `(ts-unix (ts-now))` for a Unix timestamp?

Oh, I meant that strange format used by `(current-time)`. I'm currently doing https://github.com/unhammer/org-upcoming-modeline/blob/main/org-upcoming-modeline.el#L237-L247 which isn't too bad, but perhaps I'm not the only ts user who has a need of converting back to `(HIGH LOW)`

[unhammer]

Of course, that wouldn't be appropriate for the default
org-ql-select action, but it could be a useful utility function to
include in org-ql. I'd be happy to accept a PR for something like
that.

something like
https://github.com/alphapapa/org-ql/compare/master...unhammer:parse-repeats?expand=1
?

[alphapapa]

Oh, I meant that strange format used by (current-time). I'm currently doing https://github.com/unhammer/org-upcoming-modeline/blob/main/org-upcoming-modeline.el#L237-L247 which isn't too bad, but perhaps I'm not the only ts user who has a need of converting back to (HIGH LOW)

I wish GitHub would send emails when a comment is modified. :) I added this to my message:

"Edit: Oh, I think you mean ts-internal, which does (apply #'encode-time (decode-time (ts-unix struct))). Apparently I omitted that from the documentation. I'll see about adding it. Thanks."

[theottm]

Hi! I'd also be interested in such a feature. I'd be happy to help putting some code together. The function proposed by @legalnonsense seems to go in the right direction. Where can we go from there ?

It would be useful, yes. In order to do it, you'd need a function to take an Org timestamp that repeats and return the next timestamp repetition after a certain date. Then you could call that function in the org-ql-select ACTION to return that timestamp.

Of course, that wouldn't be appropriate for the default org-ql-select action, but it could be a useful utility function to include in org-ql. I'd be happy to accept a PR for something like that.

Is there a way to make this work in org-ql-select and org-ql-query directly ? One problem I see if we do this as an action on the query results is that it would be limited to the time range of the query. For example if i only query headlines for this week but there is a repeating headline the week before I wouldn't get it, right?

I seems the interpretation of repetition should appear way ahead. Any hint on where to start looking in the source?

[alphapapa]

Implementing this may be non-trivial. By default, org-ql-select returns a list of heading elements that match the query. To return additional headings representing timestamps that have been adjusted for repetitions that have not yet occurred (i.e. the timestamp is not yet set to that date in the source file) would mean a change in the type of data returned. It would also mean returning multiple elements for a single matching entry.

Of course, something like this would be necessary to reimplement all of Org Agenda's features, so it's something I would like to do. But I'm not sure of the best way to do so. It may be necessary to implement this as a higher-level API on top of org-ql-select that would take a list of matching heading elements and post-process them to add repetitions in a specified time interval (e.g. when building an agenda view between two dates).

For example, when building an agenda view for a week, it could make 7 "passes" through the results returned by org-ql-select, each time collecting relevant entries for the day corresponding to the pass, and when doing so, it could calculate whether a repeating timestamp should be shown for that day. I'd guess that would be the best way to proceed, rather than to modify org-ql-select or any lower-level functions.

[alphapapa]

Having said that, it may still be necessary to make some changes to the timestamp-based predicates to account for repeating intervals. But they should be carefully considered. For example, if a timestamp has a repeating interval, would it be useful or desired to consider it to be open-ended, i.e. to repeat infinitely? That could cause it to match more queries than expected. As well, calculating such repetitions could adversely affect performance, so benchmarking would be needed.