title | author | date | link-citations | urlcolor | url2cite |
---|---|---|---|---|---|
Automatic citation extraction from URLs |
phiresky
|
2019-12-13 |
true |
blue |
all-links |
pandoc-url2cite allows you to instantly and transparently cite most papers directly given only a single URL.
You simply add a URL of a publication, and it will replace that with a real citation in whatever CSL style you want. This means you can avoid dealing with Mendeley or Zotero and keeping your Reference Manager database and bibtex file in sync, especially when collaborating with others.
Here is a minimal example:
minimal.md
# Introduction
The GAN was first introduced in [@gan].
# References
[@gan]: https://papers.nips.cc/paper/5423-generative-adversarial-nets
Compiling this file with this command
pandoc \
--filter=pandoc-url2cite --citeproc \
--csl ieee-with-url.csl \
minimal.md -o minimal.pdf
This results in the following output:
minimal.pdf
For a longer example, you can look at the source of this file itself, which is both a blog post, GitHub Readme and LaTeX "paper":
Source README.md - Result README.pdf
Install this package globally using npm install -g pandoc-url2cite
.
Then, add --filter=pandoc-url2cite
to your pandoc command (before --citeproc
, see the minimal example above).
Alternatively, clone this repo somewhere, then install the dependencies using npm ci install
.
If you're not familiar with writing papers in pandoc, you can refer to e.g. this article. It's pretty flexible, you can use templates from whatever conference you want, and you can still use inline latex code if you need it (and you are ok with not being able to convert your document to nice HTML or EPUB anymore).
url2cite allows multiple ways to cite:
-
(PREFERRED) Use the pandoc citation syntax for citations:
The authors of [@alexnet] first introduced CNNs to the ImageNet challenge.
More information about referencing specific pages etc. is in the pandoc manual.
Then add the URLs with the usual "link reference" syntax to the bottom of your document in its own paragraph:
[@alexnet]: https://...
You can also use the URL directly inline by using the flexible citation syntax introduced in Pandoc 2.14:
PPO [@{https://github.com/jgm/pandoc/issues/6026}] is a policy gradient method.
-
Convert all links to citations
Add
url2cite: all-links
to your yaml front matter. This will cause all links in the document to be converted to references.You can still blacklist some links by adding
no-url2cite
to either the CSS class of the link (pandoc-only):[foo](http://example.com){.no-url2cite}
or to the link title:
[foo](http://example.com "no-url2cite")
.
The main idea is that usually every piece of research you might want to cite is fully identifiable by an URL - no need to manually enter metadata like author, release date, journal, etc. Citation managers like Zotero already use this and enable you to automatically fetch metadata from a website. But then you still have a citation database somewhere that you may or may not be able to synchronize with different computers, but probably won't be able to add to the version control of your paper. There's hacks such as better-bibtex to automatically generate and update diffable bibtex files -- But that means you now have two sources of truth, and since the export is one-way this leads to multiple contributors overriding each other's changes. pandoc-url2cite goes a step further: URLs are directly used as the cite keys, and the "bibliography file" is just an auto-generated intermediary artifact of those URLs.
pandoc-url2cite is based on the work of the Zotero developers. Zotero has a set of "Translators" that are able to extract citation info from a number of specific and general web pages. These translators are written in Javascript and run within the context of the given web site. They are made to be used from the Zotero Connector browser extension, but thankfully there is a standalone Translation Server as well. To avoid the effort required to automatically start and manage this server locally, pandoc-url2cite instead uses a publicly accessible instance of this server provided by Wikipedia with a public REST API.
All citation data is cached (permanently) as bibtex as well as CSL to citation-cache.json
. This is both to improve performance and to make sure references stay the same forever after the initial fetch, as well as to avoid problems if the API might be down in the future. This also means that errors in the citation data can be fixed manually, although if you find you need to do a lot of manual tweaking you might again be better off with Zotero.
You can see a list of all supported config options in config.d.ts.
Right now there's four ways you can use url2cite in combination with "manual" citations:
- Prefix the cite key with
raw:
. e.g.[@raw:foobar]
. These are ignored by url2cite, and you can add the reference however you want in your--bibliography=
file. - set
url2cite-allow-dangling-citations=true
. That suppresses theCould not find URL for @foobar.
error and makes url2cite just ignore any cite keys that aren't aliased to an url. - Use an URL as a cite-key. Doesn't need to have a DOI or be a paper, just any website that's relevant to the work is fine as long as Zotero understands it. Then manually adjust the CSL entry url2cite generates in
citation-cache.json
. - Use an URL as a cite-key like in (3), but directly add the bibtex in a code block with language
url2cite-bibtex
anywhere:see also @{https://github.com/DLR-RM/stable-baselines3}. ```url2cite-bibtex @misc{https://github.com/DLR-RM/stable-baselines3, author = {Raffin, Antonin and Hill, Ashley and Ernestus, Maximilian and Gleave, Adam and Kanervisto, Anssi and Dormann, Noah}, title = {Stable Baselines3}, year = {2019}, publisher = {GitHub}, journal = {GitHub repository}, howpublished = {\url{https://github.com/DLR-RM/stable-baselines3}}, } ```
pandoc-url2cite also supports ISBNs and DOIs:
The book [@isbn:978-0374533557, pp. 15-17] is interesting.
See this example.
If you don't want to use citeproc, you can set url2cite-output-bib=foo.bib
to make url2cite output a bibtex file for consumption by your preferred LaTeX tool.
- Currently, extracting the metadata from direct URLs of full text PDFs does not work, so you will need to use the URL of an overview / abstract page etc. I'm not sure why, since this does work in Zotero. More info might be here
- Some websites just have wrong meta information. For example, citationstyles.org has set "Your Name" as the website author in their Open Graph metadata. You can manually modify the
citation-cache.json
file to fix / change anything.
- Manubot is a more integrated and opinionated tool for creating scientific documents that has a similar method for creating citations without the hassle.
- pandoc-url2cite-hs is a Haskell port of this tool (mostly compatible)
AlexNet first introduced CNNs to the ImageNet challenge. [@vgg; @googlenet; @resnet] further improved on the results.