Access to BibTeX fields outside of details page layout?

[WoutDLN]

Hi there!

First of all -- thanks for a great plugin! I am using it for a disciplinary lexicon website I'm building on GitHub, and it has been a great help to automatically reference all the academic output it aggregates.

I was wondering: is there a way for us to use Liquid to access the BibTeX fields anywhere outside of the details page layout file? Basically, I if I have a bibliography.bib file with the following entry:

@book{ruby,
  title     = {The Ruby Programming Language},
  author    = {Flanagan, David and Matsumoto, Yukihiro},
  year      = {2008},
  publisher = {O'Reilly Media}
}

I want to be able to access {{entry.author}}. I imagine this would then probably output Flanagan, David and Matsumoto, Yukihiro. Even better would be if I could print that information already formatted according to my preferred citation style (e.g.: Flanagan, D. & Matsumoto, Y. in the case of style: apa). But any way to access the field would already be a great help.

Since that information is accessible in the details pages, my first thought was to try find a way to {%for%}-loop over the details_dir, so I can access /details_dir/ruby.md's {{page.entry.author}}. But I don't see how I can access that directory (since it's only generated when I'm already jekyll build-ing).

Thanks for your help!
All the best,
Wout

[inukshuk]

Where exactly would you like to access the entry's fields? In order to load an entry you need to either cite it, generate a bibliography with it, or generate a detail page. So in general, the only places where there can be a reference to an entry would be the detail page or the reference template (unless I'm missing something).

I think you could insert a given entry pretty much anywhere by using the reference tag with a custom bibliography template.

[WoutDLN]

Thanks for your feedback!

To answer your question first: preferably anywhere on the website. On my website I basically aggregate quotes from academic writing, that are grouped together by subject (these are: the 'lemmas' in the 'lexicon', which each have an individual page in the website). Each quote needs a reference (for which I use {% cite %}), and there's one bibliography page that uses {% bibliography %} to render all the references in my _bibliography/lexicon.bib file. This is all what your plugin is designed to do, and for the most part it works great -- so thanks again for your work! But as I'm adding more and more quotes, the need starts to rise to organise my pages better. And that is what I would like direct access to the fields of the entries in my BibTeX file.

Say, for example, that I have a lemma.html page, that {% include %}s a total of 15 quotes of varying sizes. This is quickly becoming a very long page. So in a sidebar, it would be great, for example, to generate a little Table of Contents of all the quotes, using something like this:

<ul>
{% for quote in lemma %}
  <a href="/#{{entry.key}}"><li>{{ entry.author }}</li></a>
{% endfor%}
</ul>

This is a simplified example, but I hope it gets the gist across: for different actions across my website, I need to access the fields of individual entries in my .bib file, process them, collect them into variables, etc. I know this is not what your plugin was designed to do, but I was hoping that there was a hack/loophole that I could use, because your plugin does access those fields, e.g. in the layout for the details pages. Having looked more closely at the other issues in your repository (which I should have done earlier, sorry for that), I think I'm having the same or similar issue as those described in #270 and #279. Which tells me that it's a feature a lot of people would be interested in -- but that it sadly isn't possible at this point. I would offer to have a crack at it myself, but I would need to learn Ruby first, which my current position doesn't really allow for.

I like your suggestion of using {% reference key %}, though! With "custom bibliography template", do you mean design my own .csl? I have designed my own .bst before, so I think I could hack my way around that. I could make a .csl style that basically ignores everything except for the author field; another one that ignores everything except for the year; etc. Then, I could call on them using the --style parameter, like so:

{% reference key --style myStyle-author %}

I checked, and changing the --style parameter (e.g. from apa to chicago-fullnote-bibliography) does work inside the {% reference %} tag, so that's my issue already almost resolved! The only problem then is, that I don't know where to save my custom .csl style (and load it from). In your documentation it says that 'Alternatively you can link to any CSL style'. Does that mean that if I host my own CSL style (say: at https://www.mywebsite.com/styles/myStyle-author.csl), I could link to it as such:

{% reference key --style https://www.mywebsite.com/styles/myStyle-author.csl %}

If so, I just need to see if I can make my own simple custom CSLs files and I'm golden.

Thanks again for your help!

[inukshuk]

Instead of an ID you can also set the style to the absolute path to the (local) CSL style; using an URL as in your example should work as well (though using a local file might be best, so that you don't need to be online to generate the site).

Anyway, so using a custom CSL style should work, but it might be complicated to get a good result. Instead, what I was suggesting is to use a custom bibliography template. You can override the template in the page's front-matter or using the -T parameter of the bibliography or reference tag. This means, you could create a custom template, for example lemma_sidebar.html which will be rendered with access to each entry and use it for instance like this: {% reference key -T lemma_sidebar %}. With interpolation it might even work in the loop like:

{% for quote in lemma %}
  {% reference quote --template lemma_sidebar %}
{% endfor%}

[WoutDLN]

Ah yes -- I see how that might be much more convenient, thanks for the tip! I haven't really been able to get that to work yet, though. Maybe I'm still missing or misunderstanding something? I tested the following MWE:

Given:

A: the following Jekyll-scholar configuration in the config.yml file:

scholar:
  bibliography: lexicon.bib
  bibliography_template: bib
  relative: "../bibliography.html"
  sort_by: author, editor, year
  style: apa

B: the following lexicon.bib file inside _bibliography:

@book{ruby,
  title     = {The Ruby Programming Language},
  author    = {Flanagan, David and Matsumoto, Yukihiro},
  year      = {2008},
  publisher = {O'Reilly Media}
}

C: the following author.html template inside _layouts:

---
---
{{ entry.author }}

D the following default.html template inside _layouts:

<html>
 [...]
  {{ content }}
 [...]
</html>

E: the following test.md page in the root:

---
title: test
layout: default
---
{% reference ruby -T author %}

The expected result would be:

<html>
 [...]
  <p>
   Flanagan, David and Matsumoto, Yukihiro
  </p>
 [...]
</html>

Right?

Or am I doing something wrong? Because it keeps formatting it as if I am just using {% reference ruby %}:

<html>
 [...]
  <span id="ruby">
   <p>
    Flanagan, David and Matsumoto, Yukihiro (2008). <i>The Ruby Programming Language.</i> O'Reilly Media.
   </p>
  </span>
 [...]
</html>

I'm sorry if I'm missing something obvious! Thanks again for your help!

[inukshuk]

Hm, on a quick glance this looks good. Can you check the it works if you change the default bibliography_template option to 'author' to see if it's only the -T parameter override that's not being applied?

[WoutDLN]

Sure! With the same setup as before, but changing bibliography_template: bib to bibliography_template: author in A, I get the same result (with the full reference in test.html). I am sure that the author.html template does work, because it has transformed my default bibliography page at bibliography.html into a list of author names.

Just for the heck of it, I afterwards tried changing the {% reference %} command in test.md into {% reference ruby --bibliography_template author %} -- but as expected, the results remained the same.

What this tells me, is that {% reference %} is not taking -T nor the config.yml's scholar: bibliography_template as arguments whatsoever? And that those it only work with {% bibliography %}?

[inukshuk]

Yes, that's right, my apologies for not thinking this through properly. The reference tag is actually one level deeper than the bibliography tag: the bibliography template even has access to the rendered reference -- so obviously this reference cannot, itself, be rendered using the template that depends on its rendition.

That said, you could still achieve what you're looking for using the bibliography tag. It might even be more convenient this way if you're looking to render more than one entry: for example, in the sidebar you could render a bibliography, using the custom template, if there is a query you can use to select the entries in question our of your bib file. If there are only a handful of items, you could also construct the query using their keys.

For example, this works:

  Scenario: A simple reference with a custom template
    Given I have a scholar configuration with:
      | key          | value             |
      | source       | ./_bibliography   |
    And I have a "_bibliography" directory
    And I have a "_layouts" directory
    And I have a file "_bibliography/references.bib":
      """
      @book{ruby,
        title     = {The Ruby Programming Language},
        author    = {Flanagan, David and Matsumoto, Yukihiro},
        year      = {2008},
        publisher = {O'Reilly Media}
      }
      """
    And I have a file "_layouts/custom.html":
      """
      <div>{{entry.year}}</div>
      """
    And I have a page "scholar.html":
      """
      ---
      ---
      {% bibliography --query @*[key=ruby || key=x] -T custom %}
      """
    When I run jekyll
    Then the _site directory should exist
    And the "_site/scholar.html" file should exist
    And I should see "<div>2008" in "_site/scholar.html"

The only downside here is that even if you select only a single item, you will always also get the bibliography container wrapped around it.

Having said all that, it should be very easy to create a new tag which works sort of like we sketched out earlier: takes a single id and renders the bibliography template.

[WoutDLN]

Thank you! For individual references (or references that can be grouped together by a query) your suggestion works great! I tried it, and the fact that {% bibliography %} will always produce a bibliography container didn't even bother me that much -- after I set both bibliography_list_tag and bibliography_item_tag to span in the config.yml so it wouldn't mess with my layout.

But sadly this approach doesn't really work for me for another reason: I would need to pass a variable through the {% bibliography %} tag's options -- which doesn't work. I can't use queries, because the only thing these references have in common is that they provide a quote on a specific subject. (I could add this subject to a dedicated field in the BibTex entry, such as note = {subject}, -- but that would imply a lot of duplicate and manual work that I am trying to eliminate). If I could pass a variable through the tag option, like in the following example, that would solve the issue:

{% capture lemma %} [An array of keys to references pertaining to a lemma in the lexicon.] {% endcapture %} 
{% for quote in lemma %}
{% bibliography --query @*[key=quote] -T author %}
{% endfor %}

Which is probably impossible, because Liquid will always take the value of key in this example literally (rather than processing the variable first).

I think that therefore in my case the CSL-workaround may be the best fix for now. It's awfully hacky, but I think it should get the job done. And the added advantage would be that it would allow me to manipulate the output string in the same way that CSL does. So for example, the following code:

{% capture lemma %} ruby {% endcapture %} 
{% for key in lemma %}
{% reference key --style author %}
{% endfor %}

could be used to not only output Flanagan, David and Matsumoto, Yukihiro, but also CSL variants like Flanagan, D. and Matusumoto Y. or Flanagan et al..

Here, passing a variable inside the tag does work -- I have verified this. The only problem is that I haven't been able to get the CSL rendition to work yet (when I'm using my own). If I download a random CSL style like this one, called 'advanced materials' (saved as a .csl file, but structured as an XML file), put it in a local folder like assets/csl/advanced-materials.csl, and change the test.md file in the root to:

---
title: test
layout: default
---
{% reference ruby -T assets/csl/advanced-materials %}

Jekyll gives me the following error:

  Liquid Exception: no CSL node: nil in test.md

The log printed below this error points to things like renderer, render_text, render_group etc.

Do you know what could be the issue here? Thanks!

[inukshuk]

The queries support liquid variable interpolation -- that's why you can select specific items by key in cases where there is no suitable query to select a specific group otherwise as I meant to show in the previous example.

Using a custom CSL style you need to use the --style option, not template. I also believe you need to use an absolute path in this case (to use relative paths you can override the CSL::Style.root path)

[WoutDLN]

Thanks so much for your help! I ended up being able to solve all my issues with the {% bibliography %} tag once I figured out how to interpolate variables in the key. (I had made a mistake with data types somewhere in the transformation chain, but once I solved that error, everything worked fine!) For the sake of being complete, this is a simplified version of the actual solution as I used it in my jekyll site :

{% for definition in site.definitions | where: definition.lemma == page.name %}
  {% bibliography --query @*[key={{definition.source}}] -T my-template %}
{% endfor %}

(FYI: you can see it in action here, in an {% include %} that is called on here. It's basically what's producing all the card-headers in the bootstrap accordions, like on the website here.)

So for my part, this issue is resolved and may be closed! Thanks again.

Two points of feedback (that I'd be willing to do some more testing for if it's helpful):

1) I never did get the CSL solution to work.

You were right of course that I should be using --style in {% reference %}, rather than -T, but that was an error in my comment in this thread, I did use the right option in my code when I was testing it. I also don't think that the path is wrong (and so that you can use relative paths), because if I use another path (that does not point to a .csl file), I get a different error (no such file or directory).

2) It might be useful to also support liquid variable interpolation in the -l and -L options of {%cite%}?

As far as I'm aware, this is not possible at the moment? I also have no idea how difficult it would be, but if it's an easy tweak, I think it could be really useful for a lot of use cases, because people may have saved locators and location in a _collection somewhere, and need to call on those YAML front matter values in more automatically generated pages.

For me, for a long time, I was looking for a solution to get something like this working:

{% for definition in site.definitions | where: definition.lemma == page.name %}
  {% cite definition.source -L {{definition.locator}} -l {{definition.location}} %}
{% endfor %}

Once I got the above {% bibliography %} solution to work, I ended up using that to construct my own template for citations. It's awfully hacky, but it works -- which is why it's no longer an issue for me. And it would have been necessary in my case anyway, because I needed to account for the fact that some of the quotes I'm using are 'quoted in' other sources, and I wanted to be able to mention (and link to) both in the same reference (you can find an example on the website in the Adelung 1786 reference here). Which is not something that would be possible with {% cite %}. But that's a rare occurrence, and for people who have less complicated references, a canonical {% cite %} option with interpolated variables would probably be ideal.

Anyway -- thanks again for answering all my questions, and thinking through this together! It's been a great help!

[sneakers-the-rat]

This was super helpful, and answered my question "how do i make urls and DOIs into actual links" I think it would be lovely to add a mention to the bibliography templates section (https://github.com/inukshuk/jekyll-scholar#bibliography-template ) that you can also use the {{entry.<FIELD>}} syntax <3