look into `org-noter` support

[bdarcus]

The question

What, if anything, do we need to better integrate citar and org-noter, regardless of whether one is also using citar-org-roam?

Details

With announcement of new maintainers for org-noter, it seems its future is more secure.

We should look into ORB-like support here, either in the code itself (say a template variable), or on the wiki.

I originally added this to the citar-org-roam tracker, but realized the below would work, either for inclusion in citar-org, or on the wiki.

(defvar citar-org-noter-title-template
  " * ${title}
- tags ::
- keywords :: ${keywords}
:properties:
:custom_id: ${=key=}
:url: ${url}
:author: ${author editor}
:noter_document: ${file}  ; <== special file keyword: if more than one filename
:noter_page:               ;     is available, the user will be prompted to choose
:end:")

Also, see if citar and citar-org-roam template approaches are, or can be, aligned, so one can mix and match across contexts?

See also #750.

[bdarcus]

@deen1 - with the caveat I'm not sure when I'll have time to implement this (depending on how complicated it is), it might be good to document here what's actually required. I don't ATM use org-noter, so am unsure.

I will add that it looks like I created this issue around the same time I added capture template support to citar-org-roam.

So the question:

What, if anything, do we need to better integrate citar and org-noter, regardless of whether one is also using citar-org-roam?

BTW, the other project I've been working on is a rust project, in which I'm also learning the language. So my coding brain cycles have been wrapped up there for the last six months. Why it's hard to switch back to elisp, and this code :-)

[deen1]

I wouldn't say I'm well-versed in org-noter, much less Emacs package
creation in general, so take the following with a grain of salt.

As well as my reference library (i.e. journal papers and the like), I
also have a separate collection of pdf files for news articles etc. I
use org-noter to take notes on these and store them in large files,
with many headings corresponding to individual articles.

As far as I can tell, the important thing is that under each org
heading, the PROPERTIES drawer contains the NOTER_DOCUMENT property,
which is the path to the file. Then setting
org-noter-notes-search-path and org-noter-default-notes-file-names
will allow org-noter to associate the pdf file and the notes files. So
I can either open the notes file, go under the appropriate org-heading
(and PROPERTIES drawer), do M-x org-noter and get an org-noter frame, OR
I could go to a pdf with an pre-existing notes file (or heading), and do M-x org-noter,
and get the same result.

This at least works for the news article pdfs. For the journal articles
and use with citar, there seems to be a slight wrinkle to do with
making sure the filename of the notes file matches the filename of the
pdf file. For instance, I used to use Zotero to manage my bibliography,
and the names for pdfs took the form %AUTHORS %YEAR %TITLE.pdf.
Creating a notes file for them via citar means that the notes file has
the form ${citekey}.org, which causes an issue when org-noter tries to find the notes
for a given pdf. (But it works vice versa, going to the ${citekey}.org file and doing M-x org-noter
still works properly.)

Finally, and I'm not 100% sure about this, but I think

(setq citar-open-note-function 'orb-citar-edit-note)

might be necessary to get this to all work together, but I'm unsure how.

(P.S. I can appreciate the difficulty toggling between different projects. I should be doing something other than yak-shaving my Emacs config, but I'm convinced it will improve my productivity :-) )

[bdarcus]

I wouldn't say I'm well-versed in org-noter, much less Emacs package creation in general, so take the following with a grain of salt.

No, this is actually useful.

As far as I can tell, the important thing is that under each org heading, the PROPERTIES drawer contains the NOTER_DOCUMENT property, which is the path to the file.

Right.

IIRC (need to check), citar-org-roam defers to org-roam to create the property drawers, so that would be something to figure out.

One additional issue alongside that is to think about workflow with all these moving parts.

With the standard citar note function, I see no problem.

For the journal articles and use with citar, there seems to be a slight wrinkle to do with making sure the filename of the notes file matches the filename of the pdf file. For instance, I used to use Zotero to manage my bibliography, and the names for pdfs took the form %AUTHORS %YEAR %TITLE.pdf. Creating a notes file for them via citar means that the notes file has the form ${citekey}.org, which causes an issue when org-noter tries to find the notes for a given pdf. (But it works vice versa, going to the ${citekey}.org file and doing M-x org-noter still works properly.)

Right, this is another thing to check.

It may be that with noter one would need to write the path in the bib entry?

Finally, and I'm not 100% sure about this, but I think

(setq citar-open-note-function 'orb-citar-edit-note)

might be necessary to get this to all work together, but I'm unsure how.

Does all of this work with orb? If so, that indeed would be another thing to check; how they do it.

[deen1]

It may be that with noter one would need to write the path in the bib entry?

I'm not sure. The bibfile that contains the Zotero-originating citations includes the path to the pdf file for each bib-entry (where one exists). Unless you meant the path to the notes .org file, which I just tried to add to one entry to see if that helped -- same negative result.

One thing that slightly confuses me, and might be a red herring, is that some of these notes files have an additional :PROPERTIES: drawer, above the main org heading, looking like:

:PROPERTIES:
:ID:       2dabb44e-1132-4fb7-b9c5-4b3842c12510
:ROAM_REFS: cite:foo
:END:

Maybe this helps? Not sure.

Does all of this work with orb? If so, that indeed would be another thing to check; how they do it.

I think so. Here are some settings that might be useful (sorry to spam):

  (setq orb-preformat-keywords
        '("citekey" "title" "url" "author-or-editor" "keywords" "file")
        orb-process-file-keyword t
        orb-file-field-extensions '("pdf"))

  (citar-register-notes-source    'orb-citar-source
				  (list :items #'citar-org-roam--get-candidates
					:hasitems #'citar-org-roam-has-notes
					:open #'citar-org-roam-open-note
					:create #'orb-citar-edit-note
					:annotate #'citar-org-roam--annotate))
  (setq citar-notes-source 'orb-citar-source)

These help enable this capture template for notes:

  (add-to-list 'org-roam-capture-templates
	       '("r" "bibliography reference" plain
		 (file "~/org-roam/org-notes/template.org")
		 :if-new
		 (file+head "~/org-roam/org-notes/${citekey}.org" "#+title: ${title}\n")
		 )))

with template.org looking like this:

* %^{title}
:PROPERTIES:
:Custom_ID: %^{citekey}
:URL: %^{url}
:AUTHOR: %^{author-or-editor}
:NOTER_DOCUMENT: %^{file}  
:END:


- tags ::
- keywords :: %^{keywords}

[bdarcus]

I think so. Here are some settings that might be useful (sorry to spam).

I mean internally in ORB. How does it support org-noter, or how do users configure it to do so?

[deen1]

I'm not sure. I think the settings I provided are a way in which ORB gets the information it needs (from citar, or from helm-bibtex) to create fill in the note template. Apart from that, you seem to have already found one piece of the puzzle here.

[bdarcus]

You mean the "file" keyword?

I think we can already do this, though there there may be some wrinkles? Like, org-noter I assume requires a single path, but citar can return multiple.

[roshanshariff]

From briefly looking at org-noter, here's my understanding of the problem:

Finding the PDF file from a given notes file should work fine, because as @deen1 said, org-noter puts the PDF filename in a property drawer under each heading.

The converse, finding the note file associated with a PDF file, requires the notes file to have the same base name as the PDF file but with a ".org" extension; this is hard-coded into org-noter. With Zotero, PDF files can have arbitrary names, so if you want your notes to be named by citation key (for example) then there's no way to get org-noter to find your notes.

In principle, Citar could help here by locating the citation key associated with a given PDF (by reverse-lookup of the file field in the bibliography file exported from Zotero). The problem is, even if such a function existed, org-noter has no way of overriding its note filename lookup logic.

I think the first step would be to get org-noter to refactor this logic out (i.e. a function to locate a note filename given a PDF filename) and make it customizable. Then we could override that function with one that does the lookup through Citar. Ideally, org-noter would have a customizable list of functions it calls in order to get the note file associated with a given PDF document.

(That said, I don't have a lot of time to work on it these days because I'm trying to get my thesis done soon.)

[bdarcus]

I think the first step would be to get org-noter to refactor this logic out (i.e. a function to locate a note filename given a PDF filename) and make it customizable. Then we could override that function with one that does the lookup through Citar. Ideally, org-noter would have a customizable list of functions it calls in order to get the note file associated with a given PDF document.

@petermao - would you be open to this feature request?

PS - I see you're in Santa Monica; I grew up there!

(That said, I don't have a lot of time to work on it these days because I'm trying to get my thesis done soon.)

Best of luck with that.

[deen1]

Thanks @roshanshariff, that helped clarify my empirical understanding of org-noter's inner workings! I've shifted away from the standalone Zotero application recently, (though I still use it from time to time due to the sizeable library I'd built up over the years). The new Zotra package fits my current needs, and the maintainer there took the time to add some functionality so that I can now grab pdfs from the arXiv via org-roam-protocol and link them to a new bibentry. (If you haven't already, maybe worth checking out? Once you've defended of course :-)) The pdfs are then given the citar-key for their filename, so the issue I described about finding the corresponding notes file doesn't occur with these new entries as far as I can tell.

@bdarcus Would it then be possible to implement the "file" keyword before handling any changes to org-noter itself? The Zotero/non cite-key filename issue is a little annoying for me, but I can work around it as it's a) a problem in one direction only b) maybe not a problem for other people who haven't had the same circuitous journey through bibliography management? Obviously no urgent need on my part for any of this to get done (and thanks again for all your prior work on this package!), just a suggestion possible course of action.

[bdarcus]

Would it then be possible to implement the "file" keyword before handling any changes to org-noter itself?

Is it not already?

[deen1]

Sorry, I meant in terms of adding it to the PROPERTIES drawer. I can't seem to get this work via templates for some reason.

[bdarcus]

Not sure what I can do when, but in the meantime, are you aware of:

https://retorque.re/zotero-better-bibtex/
http://zotfile.com/