Allow individual items to override locale when rendering references

[elotroalex]

I'm using chicago-fullnote-bibliography as my style. Given the following entry in the .bib file:

@article{pollock_ideologia_1987,
	title = {{Ideología y narrativa en el Ramayana de Valmiki}},
	number = {22},
	language = {es},
	journal = {Estudios de Asia y Africa},
	author = {Pollock, Sheldon I.},
	year = {1987},
	pages = {336--54}
}

I get this citation:

———. “Ideología y Narrativa En El Ramayana De Valmiki.” Estudios De Asia y Africa, no. 22 (1987): 336–54.

In Spanish the rule would lower case "Narrativa En El" and "De." The correct result should be:

———. “Ideología y narrativa en el Ramayana de Valmiki.” Estudios de Asia y Africa, no. 22 (1987): 336–54.

EDIT: Adding an edit here for context since the title of the issue changed. Originally I indicated that I was using use_raw_bibtex_entry: true and that it was not working as expected for the double braces.

[JoostvanPinxten]

When use_raw_bibtex_entry:false, the liquid tags {{ }} in the bibtex input are evaluated.
On the contrary, setting it to false, just skips evaluating the input, and instead copies the bibtex entry 'raw'. You need to use this flag because you have double {{ braces in your bibtex:

{{Ideología y narrativa en el Ramayana de Valmiki}}

The capitalization of the words, however, is done according to the CSL file. However, I think the capitalization is overridden somewhat due to the protected bibtex field (i.e. the double braces). Normally, this would mean that the capitalization is kept the same as in the input.

In your case, I don't know exactly how it should be rendered, but the use_raw_bibtex is unlikely to be the problem here.

Try removing one of the pair of braces around the title, it might fix the issue. Otherwise, the issue might be in an updated CSL style for the chosen citation style.

[inukshuk]

Yes, this happens because the style uses title-case.

Looking at the CSL spec I just noticed that title-case conversion should be skipped for non-English references. That's something we missed in citeproc-ruby but it should be easy to fix there. I'll try to do that.

If you use only non English references or if you know that the casing in your BibTeX file is already correct, you could also remove title-casing from the style as a temporary workaround (on a quick glance, I believe the appropriate place might be here).

[inukshuk]

Hmm, turns out this is already implemented in citeproc-ruby after all, so you're probably right that this used to work already and we broke it somewhere.

[inukshuk]

Sorry for the noise but I think this is actually not implemented in jekyll-scholar. If all your references are Spanish, you could set locale to es in your jekyll-scholar configuration and that should turn off title-casing for all references.

Switching locale per item is currently not implemented in jekyll-scholar, but it should be easy to do if there is demand for it.

[elotroalex]

Thanks, Sylvester, for looking into this. I would guess the need would be at the item level for most scholars I work with (comparative and post-colonial literature, and philology). We cite from many traditions. In this bibliography alone I have Spanish, dutch, English, Italian and hindi. Item level would be a life saver. Or at the very least, figure out how we can get the CSL to respect what we do when we want to --force it.

…

On Sat, Jul 20, 2019 at 6:25 AM Sylvester Keil ***@***.***> wrote: Sorry for the noise but I think this is actually not implemented in jekyll-scholar. If all your references are Spanish, you could set locale to es in your jekyll-scholar configuration and that should turn off title-casing for all references. Switching locale per item is currently not implemented in jekyll-scholar, but it should be easy to do if there is demand for it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#268?email_source=notifications&email_token=AAB3HLS5AJ2FQLKKECFIZ6LQALRYXA5CNFSM4IFNENCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2NLN7Q#issuecomment-513455870>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAB3HLT4SEYXAXTAF3ZHOP3QALRYXANCNFSM4IFNENCA> .

[elotroalex]

Oh, I see what you mean by "this" in your last message. You are separating the two things: the implementation of use_raw_bibtex_entry and the locale issue. Sorry for my noise now.

…

On Sat, Jul 20, 2019 at 8:28 AM Alex Gil ***@***.***> wrote: Thanks, Sylvester, for looking into this. I would guess the need would be at the item level for most scholars I work with (comparative and post-colonial literature, and philology). We cite from many traditions. In this bibliography alone I have Spanish, dutch, English, Italian and hindi. Item level would be a life saver. Or at the very least, figure out how we can get the CSL to respect what we do when we want to --force it. On Sat, Jul 20, 2019 at 6:25 AM Sylvester Keil ***@***.***> wrote: > Sorry for the noise but I think this is actually not implemented in > jekyll-scholar. If all your references are Spanish, you could set locale > to es in your jekyll-scholar configuration and that should turn off > title-casing for all references. > > Switching locale per item is currently not implemented in jekyll-scholar, > but it should be easy to do if there is demand for it. > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#268?email_source=notifications&email_token=AAB3HLS5AJ2FQLKKECFIZ6LQALRYXA5CNFSM4IFNENCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2NLN7Q#issuecomment-513455870>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAB3HLT4SEYXAXTAF3ZHOP3QALRYXANCNFSM4IFNENCA> > . >

[inukshuk]

citeproc-ruby actually has a feature that will let each item override the locale using its 'language' field. This would affect not only title-casing but everything that can be localized in CSL (e.g., dates, ordinals, certain terms like 'ibid', and so on).

To get the same in scholar, we would just have to do something like this in jekyll-scholar around here: basically, set renderer.locale to the item's locale before renderer.render is invoked and re-setting the original locale in an ensure block.

Ideally we'd also cache locales (similar to styles) so that we don't keep parsing the same locale files over and over again.

[elotroalex]

I can give it a spin in the afternoon and send a PR. I'll let you know if I get stumped.

[elotroalex]

Ok. Gave it a shot for the past couple of hours. I built the gem locally and went to town tweaking. Now officially stumped. Everything is finally running without exceptions. Except, the citation is still rendering with the capital letters. Any ideas?

      def render_bibliography(entry, index = nil)

      entry.language = entry['language']  

      unless entry.language == ""
        if config['allow_locale_overrides'] == true && entry.language != config['locale']
          begin
            new_locale = CSL::Locale.load(entry.language) 
            unless new_locale.nil?
              original_locale, @locale = @locale, new_locale
            end
          rescue ParseError
            # locale not found
          end
        end

        renderer.render citation_item_for(entry, index),
            styles(style).bibliography
      end

      ensure
        unless original_locale.nil?
          @locale = original_locale
        end
    end

[elotroalex]

Also:

allow_locale_overrides is now a thing on config and it's set to true in my rig. The only predictable but minor annoyance if we get this to work is that we would have to make sure that the the .bib file is using the two letter standard for localization, i.e. 'en','es', etc.

[elotroalex]

Did we give up on getting the CSL to respect double braces? I couldn't understand if it was just impossible. The pipeline remains somewhat mysterious to me.

[elotroalex]

Fixed it! Doh. I was changing the @locale instead of renderer.locale. And it worked:

———. «Ideología y narrativa en el Ramayana de Valmiki». Estudios de Asia y Africa, n.º 22 (1987): 336-54.

Well, "worked," since it rendered the wrong type of quotes. Now gotta go see if I can get it to return to the original locale. ...

[inukshuk]

Yes, those are the quotes defined by the Spanish locale: like I mentioned, this change renders the whole citation using Spanish locale (notice also the different ordinals). If only the quotes are an issue, you could adjust the Spanish locale to use different quotes... but if you want to render the reference using English locale but skip title-casing I think we'll need to work around this differently.

[elotroalex]

Ok. First part in PR already. I'm not sure I feel comfortable doing the cache part. I went and looked and it looked a bit over my pay grade.

[elotroalex]

Yes, those are the quotes defined by the Spanish locale: like I mentioned, this change renders the whole citation using Spanish locale (notice also the different ordinals). If only the quotes are an issue, you could adjust the Spanish locale to use different quotes... but if you want to render the reference using English locale but skip title-casing I think we'll need to work around this differently.

I read up on the manual, and other styles, and it is perfectly legitimate to have different quote systems according to the languages if you're mixing it up. I say we should go with this solution.

[inukshuk]

Looks great! I added the locale cache and a test for this.

[inukshuk]

If you could document the allow_locale_overrides option that would be great (maybe also locale unless that's documented already). I'll push 5.15.0 to RubyGems later today.

[inukshuk]

You have also earned write access to the repo is you want it!

https://github.com/inukshuk/jekyll-scholar/invitations

[elotroalex]

Thanks! I'll push the documentation in the next couple of hours.

[elotroalex]

Done. Glad we got this working. Enjoyed studying your refactoring, and the test you wrote. Little by little I'm getting a sense of how it all ties together.

[inukshuk]

Awesome, thanks for getting this done!

Just pushed 5.15.0 to RubyGems.