Build Agenda from multiple files

[plantarum]

Hello,

Thank you for creating organice - it looks great!

I think the agenda feature creates an agenda only for the current file, not all the files together, and I don't see anyway to collect TODOs from multiple files in a single view. I wonder if this is possible, or if it is something you are planning to add?

Thanks again for sharing your work!

Tyler

[munen]

Hi @plantarum

Thank you for your kind words 🙏

You are absolutely right - at this point, organice always only operates on one file at the same time. However, Org mode itself has multiple features which require opening multiple files at once - Agenda is one of them, Archiving would be another and so on.

Having the option to declare org-agenda-files like in Emacs would be great. Here's the doc where those kinds of options would go: https://github.com/200ok-ch/organice#other-customizations

Are you interested in building this feature? I'd happily support with code reviews and hints.

[plantarum]

Hi @munen ,

Thanks for your response. While I would be very interested in contributing to this project, my knowledge of javascript is rudimentary at best. I don't have enough time to learn enough to be useful at the moment.

Best,

Tyler

[danilevy1212]

I would love this feature too! My TODOs are spread all over multiple files and it's kinda cumbersome to go from file to file to check.

[munen]

@danilevy1212 Great to hear! Are you interested in building the feature?

[danilevy1212]

Actually, I would like to! But, to be honest, I am noob at react (but I have plenty of experience with Vue and JS so it would be a great excuse to learn React in a practical way) and at contributing to and open source project, so I don't really know where to begin.

[munen]

Well, then you are pretty much in luck! The required refactor will primarily be JS and Redux related. On the React side it's probably just a new setting where the user can specify the list of agenda files.

I have no Vue experience, but from a quick glance, Vuex is pretty similar to Redux. So if you've worked with that, you're good. In any case, here's a tutorial: https://redux.js.org/tutorials/essentials/part-1-overview-concepts

The first challenge is this: organice is build around the concept that one Org file at a time is open. It's stored in the Redux state and is wrapped with https://github.com/omnidan/redux-undo so that any action on the Org file can be undone and redone. Here's a screenshot that I've taken with the Redux devtools which you can get here: https://github.com/reduxjs/redux-devtools

Coming back to the first challenge. You'll have to change the architecture so that multiple files at the same time can be opened. Maybe you can even keep the current structure as is (which then will solely reflect the 'currently active Org file') and add another data structure to hold the information for the other files. The functionality in the Agenda should then source information not from the currently open Org file, but from all files specified.

[danilevy1212]

Thank you very much for the detailed explanation, I will clone the repo and see what I can get done!

[munen]

Thank you very much for the detailed explanation, I will clone the repo and see what I can get done!

You're very welcome. If you've got any questions, feel free to ask!

In any case, thank you for having a crack at this issue 🙏

[tarnung]

hi @danilevy1212
I looked into multi file support myself and just now saw that there is some recent activity on this old issue. What's your status on this?

I've done some preliminary work on it and I think I have a pretty good handle on the data plumbing side of things.
There are quite a few surrounding issues that I have not yet looked into though. Like how to handle synchronization when multiple files are open. Or how to visually integrate the data of different files in search and task-list (probably breadcrumbs).

If you are still planing to go ahead with your work on this issue, I won't dive any deeper into it to minimize duplicated work.

[tarnung]

Silver lining is that my schedule is already clearing up and I was planning on continuing work on it.

Good to hear!

Anyway, I don't have much advanced at all, maybe we can coordinate and I can help you out wherever, if there is enough work that can be split between us.

I'm sure there is. The question is how we best coordinate.

@munen: Is it possible to have a branch for multi-file support in this repo that we can make PRs against until things are ready to be merged into main as a package?

I'll clean up what I have done so far as a starting point and I'll report back here to discuss the things that are left to do. (Quite a few things ;))

[danilevy1212]

Hi @tarnung.

Unfortunately, I have been really busy with life stuff this past weeks. Other than forking the project, running tests and just looking around it I haven't been able to do much. Silver lining is that my schedule is already clearing up and I was planning on continuing work on it.

Anyway, I don't have much advanced at all, maybe we can coordinate and I can help you out wherever, if there is enough work that can be split between us.

[munen]

excited 😄

[munen]

Github is having some issue. The comments are out of order. Before, I saw the last two comments (inclusive mine) with a timestamp "in one hour" and now they are just out of order.

In any case: @tarnung

@munen: Is it possible to have a branch for multi-file support in this repo that we can make PRs against until things are ready to be merged into main as a package?

Yes, of course! It's good thinking, too. To get multi-file support right might not be a small effort, so it makes sense to have a longer lasting branch.

ILet's also add auto-deployment on merging to this branch to https://staging.organice.200ok.ch/ in a similar manner that we have auto-deployment to https://organice.200ok.ch/ on merging to master. If so, I propose to name this long living branch develop (naming taken from the popular nvie git branching model).

If it helps, I have created a 'GH project' so that you can track and organise issues related to this milestone in a consistent manner: https://github.com/200ok-ch/organice/projects/1

Let me know if I can help with anything else in the meantime.

[munen]

We now have automatic deployment to production and stage instances of organice. Here's the PR including docs: https://github.com/200ok-ch/organice/pull/549/files

[tarnung]

Let's also add auto-deployment

That would be nice.

If so, I propose to name this long living branch develop

That implies that all PRs need to go through develop before landing on master. I'm not sure if this is necessary. Depending on how long of a project this turns out to be, it might be reasonable to allow smaller features (not just bugfixes) to be merged into master if they can be integrated into develop without too much extra effort. But we can look into that once the need arises.

[munen]

Good timing(;

[munen]

That implies that all PRs need to go through develop before landing on master.

If we followed the nvie branching model for every PR, yes. But I agree that we don't have to do that for features that fit into one PR. It would only be overhead and is not that common for web app development, anyway, because those tend to go forward. I have documented it as such here: https://github.com/200ok-ch/organice/pull/549/files

Please review the above PR and let me know if the wording/workflow works for you.

[tarnung]

I dumped what I have so far into #550 and I wrote up some obvious points thad need to be addressed for this to be useable. I'm sure I missed some things.
We can continue the discussion here and then create individual issues once we agree on the direction to go.

[munen]

This is implemeted in #550. Closing.

[aspiers]

Awesome, congrats!

[munen]

@aspiers Thanks you 🙏 @tarnung and I worked a lot on this issue.

In the settings, there's now a "File settings" section where you can configure different attributes for a file (synced on startup, include in agenda, include in search, include in refile, include in task-list):

As a side-effect, organice also became a lot faster and better in offline scenarios. Every opened file is now synchronized to localStorage and will be re-opened instantaneously before the newest version from the storage back-end is pulled.

[aspiers]

Looks and sounds amazing! 🚀 🎆

[munen]

Happy to hear you like it(;