Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolve id: links to other files #47

Open
amake opened this issue May 25, 2021 · 1 comment
Open

Resolve id: links to other files #47

amake opened this issue May 25, 2021 · 1 comment
Labels
enhancement New feature or request

Comments

@amake
Copy link
Owner

amake commented May 25, 2021

Partially for #17; as discussed in #44.

id: link IDs are UUIDs and do not specify filenames, so in order to resolve such links to sections in other files, we will need to somehow identify which file contains the ID.

We can get away with not doing a full org_parser parse of every potential target file by doing something much simpler:

stream file content linewise and just extract the IDs e.g. with /:ID:\s+[0-9A-Z-]+/. On iOS in particular this could mean forcing downloads of files that aren't available locally, but if the majority use case for Org Roam is lots of small files then it could be OK.

This would probably happen in the native layer, with the resulting file identifier being returned to the Dart layer.

As a first attempt I may do this on-demand (no indexing mechanism) and see how it goes. If it's unbearably slow to resolve a link (quite possible for e.g. large Org Roam databases like https://github.com/jethrokuan/braindump/tree/master/org) then an indexing mechanism will be needed.

The index complicates things significantly:

  • How do we manage the task in the UI?
    • It will presumably be long-running, so the user needs to know the progress
    • We probably don't want to allow multiple indexing tasks to run concurrently, so some restrictions are needed
  • How do we ensure the index is up to date?
    • Can we run it on every file open? How expensive is it to run?
  • What should be in it? For each file:
    • Persistable file ID
    • File modification date (for detecting when an update is needed)
    • List of IDs found
@amake amake added the enhancement New feature or request label May 25, 2021
@amake
Copy link
Owner Author

amake commented Jun 16, 2021

Progress is slow on this so far due to other projects and life stuff.

At the moment I have implemented on iOS searching through files to find a particular ID. Some hurdles:

  • Reading a file line-by-line (as opposed to reading the whole thing into memory) in Swift: solved for the moment via https://stackoverflow.com/a/24648951/448068 and https://github.com/RMJay/LineReader
  • Dealing with not-yet-downloaded iCloud files: when iterating the files on the filesystem, items that exist in the cloud but not yet on the device show up as stub files like .foo.ext.icloud. Once they're downloaded, they will be foo.ext. It seems like you're intended to discover and access files via NSMetadataQuery, not by dealing directly with .icloud files, but I'm not sure how well that works with what I'm doing: NSMetadataQuery results come in asynchronously; I'm not sure how to search just a specific directory tree, etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant