Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(feat): org-roam-doctor: fix broken links #570

Merged
merged 1 commit into from
May 10, 2020
Merged

(feat): org-roam-doctor: fix broken links #570

merged 1 commit into from
May 10, 2020

Conversation

jethrokuan
Copy link
Member

@jethrokuan jethrokuan commented May 5, 2020

Motivation for this change

This library provides org-roam-doctor, a utility for diagnosing and fixing
Org-roam files. Running org-roam-doctor launches a list of checks defined
by org-roam-doctor--checkers. Every checker is an instance of
org-roam-doctor-checker.

Each checker is given the Org parse tree (AST), and is expected to return a
list of errors. The checker can also provide "actions" for auto-fixing errors
(see `org-roam-doctor--remove-link' for an example).

The UX experience is inspired by both org-lint and checkdoc, and their code
is heavily referenced.

doctor

Closes #222

@zaeph
Copy link
Member

zaeph commented May 5, 2020

I like the idea, and I like making this into its own library.

An earlier idea I had for implementing this would have been to run org-lint on every org-roam--list-all-files, since I believe it is checking for broken links.

@jethrokuan
Copy link
Member Author

oh... that saved me a lot of trouble

@jethrokuan jethrokuan marked this pull request as ready for review May 7, 2020 06:28
@jethrokuan jethrokuan requested review from zaeph and progfolio May 7, 2020 06:28
Copy link
Member

@zaeph zaeph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works on my end, but I don't have to check everything. I've made a few comments for now, but I'll check again tomorrow.

org-roam-doctor.el Show resolved Hide resolved
(unless (org-in-regexp org-link-bracket-re 1)
(user-error "No link at point"))
(save-excursion
(delete-region (match-beginning 0) (match-end 0))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be done in a temp-buffer. Otherwise, if you C-g in a middle of the org-roam-insert meant to fix it, the link is removed from the current buffer.

A better option might be to catch errors/interrupts with a condition-case which would revert the buffer to its pre-doctor state.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

org-roam-insert doesn't really work in a temp-buffer though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then the second option I've mentioned might be best.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done... seems a bit inefficient though.

org-roam-doctor.el Show resolved Hide resolved
(match-string-no-properties 2)
(org-link-unescape (match-string-no-properties 1)))))
(delete-region (match-beginning 0) (match-end 0))
(org-roam-insert nil nil label))))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to keep the label, chances are we might be interested in pre-selecting any candidate that starts with that label. For instance, if I have a broken link with the label Metaphor, fixing it should pre-select Metaphor in my list. This could be done by adding a PRE-SELECT argument to org-roam-insert which, when set, would attempt to place the matched candidate at the top of the completions. Sorting also has the benefit of being completion-engine-agnostic.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this be done with regular completing-read? Not familiar with how to do this.

(unless (memq buf existing-buffers)
(save-buffer buf)
(kill-buffer buf)))))
(message "Linting completed."))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could print stats here by incrementing variables for fixed and skipped links.

(p (point)))
(condition-case nil
(save-excursion
(delete-region (match-beginning 0) (match-end 0))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
(delete-region (match-beginning 0) (match-end 0))
(replace-match ""))

Comment on lines 123 to 131
(defun org-roam-doctor--replace-link ()
"Replace the current link with a new link."
(unless (org-in-regexp org-link-bracket-re 1)
(user-error "No link at point"))
(let ((orig (buffer-string))
(p (point)))
(condition-case nil
(save-excursion
(delete-region (match-beginning 0) (match-end 0))
Copy link
Member

@progfolio progfolio May 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to save-match-data here?

Comment on lines 68 to 82
(defun org-roam-doctor-broken-links (ast)
"Checker for detecting broken links.
AST is the org-element parse tree."
(org-element-map ast 'link
(lambda (l)
(when (equal "file" (org-element-property :type l))
(let ((file (org-element-property :path l)))
(and (not (file-remote-p file))
(not (file-exists-p file))
(list (org-element-property :begin l)
(format (if (org-element-lineage l '(link))
"Link to non-existent image file \"%s\"\
in link description"
"Link to non-existent local file \"%s\"")
file))))))))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
(defun org-roam-doctor-broken-links (ast)
"Checker for detecting broken links.
AST is the org-element parse tree."
(org-element-map ast 'link
(lambda (l)
(when (equal "file" (org-element-property :type l))
(let ((file (org-element-property :path l)))
(and (not (file-remote-p file))
(not (file-exists-p file))
(list (org-element-property :begin l)
(format (if (org-element-lineage l '(link))
"Link to non-existent image file \"%s\"\
in link description"
"Link to non-existent local file \"%s\"")
file))))))))
(defun org-roam-doctor-broken-links (ast)
"Checker for detecting broken links.
AST is the org-element parse tree."
(org-element-map ast 'link
(lambda (l)
(when (equal "file" (org-element-property :type l))
(let ((file (org-element-property :path l)))
(and (not (or (file-exists-p file)
(file-remote-p file)))
`(,(org-element-property :begin l)
,(format (if (org-element-lineage l '(link))
"Link to non-existent image file \"%s\"\
in link description"
"Link to non-existent local file \"%s\"")
file))))))))

I grouped the file predicates in an or and reversed the order. This should be more efficient, as the or will short-circuit on the first nil case. I believe that will be file-exists-p more often than file-remote-p. It's probably a micro-optimization, but it should be logically equivalent.

Comment on lines 197 to 221
(defun org-roam-doctor (&optional this-buffer)
"Perform a check on Org-roam files to ensure cleanliness.
If THIS-BUFFER, run the check only for the current buffer."
(interactive "P")
(let ((existing-buffers (org-roam--get-roam-buffers))
files)
(if (not this-buffer)
(setq files (org-roam--list-all-files))
(unless (org-roam--org-roam-file-p)
(user-error "Not in an org-roam file"))
(setq files (list (buffer-file-name))))
(save-window-excursion
(dolist (f files)
(let ((buf (find-file-noselect f)))
(with-current-buffer buf
(org-roam-doctor--check buf org-roam-doctor--checkers))
(unless (memq buf existing-buffers)
(save-buffer buf)
(kill-buffer buf))))))
(message "Linting completed."))
Copy link
Member

@progfolio progfolio May 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we could introduce a customizable variable, org-roam-doctor-files which defaults to (org-roam-list-all-files). org-roam-doctor would run its checks against the files in this list. Users/commands could then customize that list to check a subset.
e.g.

(let ((org-roam-doctor-files '("example.org" "etc..."))) 
  (org-roam-doctor))

This way it's not an all or single file proposition.
similar to org-agenda-files.

org-roam.el Outdated
Comment on lines 466 to 475
(defun org-roam-insert (&optional prefix filter-fn description)
"Find an Org-roam file, and insert a relative org link to it at point.
If PREFIX, downcase the title before insertion.
FILTER-FN is the name of a function to apply on the candidates
which takes as its argument an alist of path-completions. See
which takes as its argument an alist of path-completions.
If DESCRIPTION is provided, use this as the link label. See
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If prefix is meant to downcase the title before insertion, can we give it a descriptive name? e.g. downcase or lowercase?

Comment on lines 198 to 209
;;;###autoload
(defun org-roam-doctor (&optional this-buffer)
"Perform a check on Org-roam files to ensure cleanliness.
If THIS-BUFFER, run the check only for the current buffer."
(interactive "P")
(let (files)
(if (not this-buffer)
(setq files (org-roam--list-all-files))
(unless (org-roam--org-roam-file-p)
(user-error "Not in an org-roam file"))
(setq files (list (buffer-file-name))))
(org-roam-doctor files org-roam-doctor--checkers)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
;;;###autoload
(defun org-roam-doctor (&optional this-buffer)
"Perform a check on Org-roam files to ensure cleanliness.
If THIS-BUFFER, run the check only for the current buffer."
(interactive "P")
(let (files)
(if (not this-buffer)
(setq files (org-roam--list-all-files))
(unless (org-roam--org-roam-file-p)
(user-error "Not in an org-roam file"))
(setq files (list (buffer-file-name))))
(org-roam-doctor files org-roam-doctor--checkers)))
;;;###autoload
(defun org-roam-doctor (&optional all)
"Perform a check on the current Org-roam file to ensure cleanliness.
If ALL is non-nil, run the check only for all org-roam files."
(interactive "P")
(let (files (if all
(org-roam--list-all-files)
(unless (org-roam--org-roam-file-p)
(user-error "Not in an org-roam file"))
`(,(buffer-file-name))))
(org-roam-doctor-start files org-roam-doctor--checkers)))

I think the last line is intended to call org-roam-doctor-start, correct?

Do you think reversing the semantics of org-roam-doctor's argument would be better UX?
The larger one's set of files gets, the longer this command will block Emacs.
Do we want to present that as the typical use-case, or as an optional use-case?
Ideally we could provide the best of both worlds and implement the whole thing asynchronously, but that comes with challenges as well (e.g. what happens if you're editing the files as they're being checked...).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think reversing the semantics of org-roam-doctor's argument would be better UX?

Hmm perhaps so.

The larger one's set of files gets, the longer this command will block Emacs.

I want to add a linted column to the files table, so org-roam-doctor can mark a file as linted once it does a pass-through. Modifying a file would then reset the value of that column to nil. That should make linting fast enough to run, to org-roam-db-build-cache.

`org-roam-doctor` provides a diagnostic tool for checking an Org-roam
file for things that are broken. Currently implemented is a check for
broken links, and methods to fix them. The checker is designed to be
extensible.
@jethrokuan jethrokuan merged commit d68d1f8 into master May 10, 2020
@jethrokuan jethrokuan deleted the feat/doctor branch May 10, 2020 05:57
@jcguu95
Copy link

jcguu95 commented Jul 14, 2020

Is it possible to org-roam-doctor-this-file? So I can fix the file name of the current file, and update all links to this file all at once?

@zaeph
Copy link
Member

zaeph commented Jul 14, 2020

Renaming the file in Emacs will already update all the links. Try to do it in Dired.

@jcguu95
Copy link

jcguu95 commented Jul 14, 2020

@zaeph that's definitely new to me! Would you mind giving a bit more pointers?

@zaeph
Copy link
Member

zaeph commented Jul 14, 2020

Well, there isn't any. When org-roam-mode is active, renaming a file in your org-roam-directory will run a number of checks to make sure that all the files that were pointing to the file being renamed will be updated.

@jcguu95
Copy link

jcguu95 commented Jul 14, 2020

That's surprising.. say I renamed 1.org to 1_.org, how would org-roam know that 1_.org used to be 1.org?

@zaeph
Copy link
Member

zaeph commented Jul 14, 2020

Because the hook we use, rename-file, is aware of the previous name and the new name.

(advice-add 'rename-file :after #'org-roam--rename-file-advice)

org-roam/org-roam.el

Lines 1226 to 1274 in 7a4b15f

(defun org-roam--rename-file-advice (old-file new-file-or-dir &rest _args)
"Rename backlinks of OLD-FILE to refer to NEW-FILE-OR-DIR."
;; When rename-file is passed a directory as an argument, compute the new name
(let ((new-file (if (directory-name-p new-file-or-dir)
(expand-file-name (file-name-nondirectory old-file) new-file-or-dir)
new-file-or-dir)))
(when (and (not (auto-save-file-name-p old-file))
(not (auto-save-file-name-p new-file))
(not (backup-file-name-p old-file))
(not (backup-file-name-p new-file))
(org-roam--org-roam-file-p old-file))
(org-roam-db--ensure-built)
(let* ((old-path (file-truename old-file))
(new-path (file-truename new-file))
(old-slug (org-roam--get-title-or-slug old-file))
(old-desc (org-roam--format-link-title old-slug))
(new-slug (or (org-roam-db--get-titles old-path)
(org-roam--path-to-slug new-path)))
(new-desc (org-roam--format-link-title new-slug))
(new-buffer (or (find-buffer-visiting new-path)
(find-file-noselect new-path)))
(files-to-rename (org-roam-db-query [:select :distinct [from]
:from links
:where (= to $s1)
:and (= type $s2)]
old-path
"file")))
;; Remove database entries for old-file.org
(org-roam-db--clear-file old-file)
;; Insert new headlines locations in new-file.org after removing the previous IDs
(with-current-buffer new-buffer
(org-roam-db--update-headlines))
;; Replace links from old-file.org -> new-file.org in all Org-roam files with these links
(mapc (lambda (file)
(setq file (if (string-equal (file-truename (car file)) old-path)
new-path
(car file)))
(org-roam--replace-link file old-path new-path old-desc new-desc)
(org-roam-db--update-file file))
files-to-rename)
;; If the new path is in a different directory, relative links
;; will break. Fix all file-relative links:
(unless (string= (file-name-directory old-path)
(file-name-directory new-path))
(with-current-buffer new-buffer
(org-roam--fix-relative-links old-path)
(save-buffer)))
(when (org-roam--org-roam-file-p new-file)
(org-roam-db--update-file new-path))))))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Discovering / repairing broken references after moving files
4 participants