Search for files by title

[AlessandroW]

Feature

Currently opening a new file is only possible through the system's file picker. This issue proposes a file search based on the document +TITLE: property.

Motivation

Thanks to the new update that allows accessing files in a permitted directory a file-based search should be possible to implement. This file-based search allows searching for org-files based on their +TITLE property and is a small step towards features such as Org-Roam compatibility #17

Pitch

Find and open files based on their document title.

Example

Document

#+TITLE: "Testing a REST API using Emacs and Org Mode"
[...]

Path

├── ...
├── literate_programming_against_rest_apis.org
├── ...

Details

This has been already implemented in Org Roam as org-roam-find-file using a SQLite database for caching.

[amake]

Thanks for expanding on your idea.

My main questions are:

1. What does the overall workflow look like when starting from scratch?

   When Orgro starts from a clean install, it has no permissions for anything. To get access to a directory tree, the user must select the directory from a modal file picker. How will you guide the user to do so?

   The user may pick some deeply nested tree at first, but wish to open a file from a different tree later. How will that work?

   Note also that most third-party apps don't support granting directory permissions at all, so we can't require the user to pick a directory in order to use the app. Picking a single file from the system file picker must remain a supported use case.

2. How will this development fit in with the current UI? The current UI is extremely simple:

• The "start" page has a "Choose File" button, from which you pick a single file
• Once you have opened at least one file, the "recent files" page is shown instead. Here you can reopen a single recently opened file, or you can click the FAB to pick a new file.

3. It will not be feasible to actually parse every file with org_parser (large files take tens of seconds even on fast hardware), so the content will need to be looked at with something much simpler. Just extracting the first #+TITLE: should be easy (if it fits on a single line, etc.), but it seems arbitrary to limit the feature to extracting and displaying just the title when there is an obvious extension to doing multi-file full-text search.

   Reconcieving the feature as full-text search rather than strictly a new way to pick files might give hints as to how to reconcile it with the existing file-opening UI.

[amake]

The native-side machinery for examining the files' content will probably have a lot in common with #47.

[AlessandroW]

Thanks for your feedback!

What does the overall workflow look like when starting from scratch?

I'm not sure what the visually best approach is. However, I was thinking about specifying a directory similar to org-directory/org-roam-directory and then parsing only this directory. That is also the Orgzly approach.

How will this development fit in with the current UI? The current UI is extremely simple:

• Following the previous point this would suggest an additional element in the settings to specify the directory?
• Regarding the actual search I was thinking about expanding the floating action button capability if the org-directory is set.

It will not be feasible to actually parse every file with org_parse

I agree, hence the restriction to the +TITLE: property.

but it seems arbitrary to limit the feature to extracting and displaying just the title when there is an obvious extension to doing multi-file full-text search.

Obvious, maybe. Presumably then we would have to follow the route of Orgzly, where full text search is possible, and store the data in a database. Then we would need a syncing mechanism instead of working on org-files. See orgzly/orgzly-android#668 (comment)
Personally, I would rather have a (manual) sync-free approach.

[amake]

However, I was thinking about specifying a directory similar to org-directory/org-roam-directory and then parsing only this directory.

Seems a bit limiting, but OK.

Following the previous point this would suggest an additional element in the settings to specify the directory?

Sure. There isn't really a settings screen right now, but I think this would require one.

Presumably then we would have to follow the route of Orgzly, where full text search is possible, and store the data in a database.

I'm not sure why a database would be needed for the full-text search use case, but not for the TITLE use case.

If you are planning on persisting the parsed titles then that is also part of the design that should be discussed.

Personally, I would rather have a (manual) sync-free approach.

100% agreed. I would like to keep the data stored in or generated by Orgro to the absolute minimum.

[AlessandroW]

I'm not sure why a database would be needed for the full-text search use case, but not for the TITLE use case.

Sorry, I didn't explain myself well. In Orgzly all org files are parsed and saved in the DB and all further modifications are done in the DB. Therefore, a sync-mechanism is needed to update the org files. This is quite tedious and error prone. However, on further thought, the manual syncing is only necessary due to write operations. Because we would only have to read a file refreshing the database can be automated and shouldn't be a problem (ignoring #6).

I would like to keep the data stored in or generated by Orgro to the absolute minimum.

Maybe it makes sense to test first how long it takes to search for a file by title before trying to design a database schema?
I could test it on my private directory containing ~1200 notes.