Export of creator-type fields from embedded CSL variables + change authors to directors for CSL-JSON film+tvBroadcast + export tvBroadcast to biblatex video

[njbart]

citeproc-js recognizes {:label: lastName || firstName} in Zotero’s Extra field as a two-field name value. (See zotero/zotero#846.)

BBT could use the same syntax (not too pretty, but then again, we don’t have to invent one from scratch).

This would be great to add creator types for items that don’t currently support them (e.g., “Director”¹ for “Film”, “Editor” for “Report” and “Web page”, …).

¹ Zotero does have a “Director” field but does not export it properly.

[retorquere]

I currently see these as potential creator fields; what Bib(La)TeX fields to these correspond to?

[retorquere]

In what sense does Zotero not export the director properly?

[njbart]

I currently see these as potential creator fields; what Bib(La)TeX fields to these correspond to?

Overlap between CSL and biblatex is minimal here: author, editor, translator, director¹ .

¹ director={} is only supported by biblatex-apa and biblatex-fiwi; biblatex-chicago uses the standard biblatex editor={…}, editortype={director}. I’d favour using director={} by default, I think it’s easier to convert director={} to editor={…}, editortype={director} (using biblatex’s \DeclareSourcemap mechanism) than the other way around. (FWIW, pandoc-citeproc can map both formats back to CSL director.)

In what sense does Zotero not export the director properly?

It exports “Director” as CSL author; see https://forums.zotero.org/discussion/6112/incorrect-apa-style-film-citing/

Styles that support CSL director, e.g., chicago-author-date.csl, would work nicely with Zotero if “Director” was exported as CSL director.

[retorquere]

I can convert director to the appropriate field in the *TeX exports; if you have a testcase + expected output, that would be much appreciated.

WRT "director" not being exported correctly, since Pandoc JSON uses CSL JSON internally, and since creator order is lost in the transformation from Zotero item to CSL-JSON item, I cannot patch this behavior (without recreating the Zotero to CSL-JSON formatter myself). This problem should be fixed when it is fixed in Zotero. I wish I could do more than defer the problem, but there's nothing else I can do for this one.

[retorquere]

The current (expected) behaviour in the 365 branch should be that if you specify a creator-type multiple times, it will replace the normal Zotero creator type, but will stack other than that; so multiple {:editor: ...} fields will replace the usual editors, but the overrides will all be exported.

[retorquere]

I have a version that passes all existing tests at http://tempsend.com/CAA34AAEF7

[njbart]

"director" not being exported correctly … I cannot patch this behavior …

What about a bit of brute force then?

Just convert all authors in motion_picture items to directors before writing out “Pandoc JSON” – motion_picture items do not have (genuine) authors this could conflict with.

[retorquere]

In those cases where the behavior is unequivocal, then yes.

But I see that for a zotero "Film" reference, nothing but the director is exported; other creator types are simply silently dropped on the floor. Is that desirable?

[njbart]

I have a version that passes all existing tests at http://tempsend.com/CAA34AAEF7

Thank you – but:

editor: Doe||Jane
editor: Roe||Rob

returns just

"editor":{"family":"Roe","given":"Rob"}

[retorquere]

Well, it was untested :) Do you have a full reference for me? I can stick those straight in my test suite.

[njbart]

But I see that for a zotero "Film" reference, nothing but the director is exported; other creator types are simply silently dropped on the floor. Is that desirable?

Not at all. But there are no CSL variables for any of these.

For bib*tex there are at least producer and scriptwriter (biblatex-apa / -fiwi) – but never authors.

[retorquere]

New version at http://tempsend.com/22F7908B24 for (I think) editor fix & director fix

[njbart]

full references: PQPQFN3D

New version at http://tempsend.com/22F7908B24 for (I think) editor fix & director fix

director: looks good, editor: not yet, still: "editor": { "family": "Editor", "given": "Ed" }

[njbart]

http://tempsend.com/22F7908B24: It seems labels that do not match a valid CSL variable are no longer accepted when using the {:label: xxx} format, e.g., {:X-author: Doe||John} or {:X-genre: xxx}.

Result when trying to export to “Pandoc JSON”: Popup “An error occurred while trying to export the selected file.”

I don’t observe this error with the ``X-author: Doe || John` format.

I found this quite useful for “commenting out” variables when testing, so it’d be great to get the original behaviour back.

[njbart]

One more thing: for the CSL broadcast item type, too, nothing but the director is exported (and mapped to author), and again, there are no (genuine) authors; so here, too, authors should be converted to directors.

[retorquere]

It looks like http://tempsend.com/4CA88E0D7E addresses your needs, yeah? Source references here (I added the X-field to see what would happen), BibLaTeX output, CSL-JSON output

[retorquere]

@nickbart1980 I missed your last comment. You mean authors changed to directors, right?

[retorquere]

(collapsing arrays is fairly easy if you use vi(m); but vim isn't really sensible to use unless you're already used to it, and without that I don't know if I'd bother to collapse those arrays)

[njbart]

I have the discrepancies down to two lines …

Looks like genuine errors:

• friends:leia:tv should have booktitle = {Friends} (from the “Program title” field in Zotero)
• friends:leia:dvd should have number = {1} (from number: 1 in the “Extra” field in Zotero)

[retorquere]

Well that's the thing then, there is no programTitle or number: 1 in the friends:leia:tv reference, but they're in the friends:leia:tv BibLaTeX reference. Right?

[njbart]

Ok, see #368.

But number is in both :tv ("number": "1",) and :dvd ("extra": "bibtex: friends:leia:dvd\ncontainer-title: Friends\nnumber: 1\noriginal-publisher: NBC\noriginal-date: 1996-09-16",).

[retorquere]

I don't know what's going on then on my end; here it only appears once:

$ grep 'number:.*1' test/fixtures/export/*365.json
      "extra": "bibtex: friends:leia:dvd\ncontainer-title: Friends\nnumber: 1\noriginal-publisher: NBC\noriginal-date: 1996-09-16",

[njbart]

One comes from a proper Zotero field, the other is embedded in the Extra field. (Zotero is pretty inconsistent wrt fields it offers for video material.)

[retorquere]

Ah, hold on, it's in the "number" field! Let me see what's up with that.

[retorquere]

Wait, when you say that

director={} is only supported by [biblatex-apa] and [biblatex-fiwi]; [biblatex-chicago] uses the standard biblatex editor={…}, editortype={director}. I’d favour using director={} by default, I think it’s easier to convert director={} to editor={…}, editortype={director} (using biblatex’s \DeclareSourcemap mechanism) than the other way around. (FWIW, pandoc-citeproc can map both formats back to CSL director.)

do you mean that stuff will break for existing users unless they use "DeclareSourceMap"? In that case I'd prefer to stick this behind a preference. I don't know the prevalence of -apa/-fiwi, but unless they constitute the bulk of the use, I can't just silently break stuff.

[njbart]

I'd prefer to stick this behind a preference …

Fair enough.

[retorquere]

Does @movie support publisher? The biblatex manual doesn't seem to say anything about @movie other than "see @video", which in turn says "see @movie".

[njbart]

The biblatex manual lists @movie, @video and others as aliases of @misc, and indeed (plain) biblatex does not output the content of the publisher field of @misc items.

biblatex-chicago and biblatex-apa, on the other hand, recognize a @movie type that is not just an alias of @misc, and both do use the publisher field for the production company.

Finally, biblatex-fiwi, a style focusing on film studies, uses a production field.

My recommendation would be to use the publisher field for biblatex: This is what biblatex-chicago and biblatex-apa, two of the more popular styles, use. In plain biblatex, specific support for movies and videos is all but nonexistent, so anyone interested in citing such material will most likely use a different style anyway. And biblatex-fiwi, I feel, is not sufficiently widespread to justify using its variant field name. FWIW, Zotero is mapping its GUI fields Distributor, Studio, Network to the CSL variable publisher, too.

[retorquere]

OK, if I add the publisher field, I still end up dropping:

• scriptwriter
• editor
• booktitle
• journaltitle
• volume
• number

--- expected
+++ found
@@ -1,7 +1,6 @@
 @report{Author2015,
   title = {A Report},
   author = {Author, Ann},
-  editor = {Editrix, Emma and Editor, Ed},
   date = {2015},
   institution = {{Institution}},
   location = {{Place}},
@@ -39,36 +38,24 @@
 
 @video{friends:leia:dvd,
   title = {The One with the {{Princess Leia}} Fantasy},
-  booktitle = {Friends},
   editor = {Mancuso, Gail},
   date = {2003},
   origdate = {1996-09-16},
-  journaltitle = {Friends},
-  volume = {3},
   publisher = {{Warner Home Video}},
   location = {{US}},
-  editora = {Curtis, Michael and Malins, Gregory S.},
-  editoratype = {scriptwriter},
   editortype = {director},
   langid = {american},
-  number = {1},
   origpublisher = {{NBC}}
 }
 
 @video{friends:leia:tv,
   title = {The One with the {{Princess Leia}} Fantasy},
-  booktitle = {Friends},
   editor = {Mancuso, Gail},
   date = {1996-09-16},
-  journaltitle = {Friends},
-  volume = {3},
   publisher = {{NBC}},
   location = {{US}},
-  editora = {Curtis, Michael and Malins, Gregory S.},
-  editoratype = {scriptwriter},
   editortype = {director},
-  langid = {american},
-  number = {1}
+  langid = {american}
 }

[njbart]

This seems to be based purely on what the biblatex manual lists as required and optional fields, but I would argue this is too narrow: It’s perfectly possible for a report to have editors (and indeed biblatex-chicago formats this properly).

The video entries, too, rather closely match the following from the current version of biblatex-chicago (/usr/local/texlive/2019/texmf-dist/doc/latex/biblatex-chicago/dates-test.bib on my machine) and are required to format such entries properly with that package:

@Video{friends:leia,
  title = 	 {The One with the {Princess Leia} Fantasy},
  date = 	 2003,
  booktitle = 	 {Friends},
  booktitleaddon = 	 {season~3, episode~1},
  author = 	 {Curtis, Michael and Malins, Gregory~S.},
  eventdate = 	 {1996-09-19},
  editor = 	 {Mancuso, Gail},
  editortype = 	 {director},
  publisher = {Warner Home Video},
  type = 	 {DVD},
  address = 	 {Burbank, CA},
  annotation = 	 {This is a template for citing television shows.  The
                  \textsf{eventdate} is the original broadcast date,
                  while the \textsf{date} applies to the
                  \vadjust{\eject}medium you are citing.  As in other
                  audiovisual entries, the earliest date automatically
                  goes at the head of the entry.  Note that
                  information about the season and episode numbers
                  goes in \textsf{booktitleaddon}.}
}

(The examples friends:leia:dvd and friends:leia:tv might have come from an earlier version of biblatex-chicago, but I can’t easily check this right now.)

I general, I wouldn’t drop Zotero fields just because they are not listed in the biblatex manual, provided they are being used in at least one of the other more popular packages.

One problem is of course that different biblatex packages sometimes require different fields, and IIRC in the past I have sometimes argued in favour of BBT using/generating simple and clear field names and content, e.g.:

volume = {3},
number = {1}

rather than biblatex-chicago’s

  booktitleaddon = 	 {season~3, episode~1},

or biblatex-apa’s

  TITLE           = {Who Shot {Mr.} {Burns}? ({Part} {One}) ({Season}~6,
                     {Episode}~25)},

(the latter from /usr/local/texlive/2019/texmf-dist/doc/latex/biblatex-apa/biblatex-apa-test-references.bib).

As I have argued elsewhere repeatedly, I feel the least bad option in such cases is to have users sort this out themselves, e.g., via the \DeclareSourcemap mechanism.

[retorquere]

I general, I wouldn’t drop Zotero fields just because they are not listed in the biblatex manual, provided they are being used in at least one of the other more popular packages.

I have no idea what popular packages are, and what fields they use though.

[retorquere]

@njbart do you know how biblatex and biblatex-chicago relate? Is the latter aiming to supplant the former, is the latter a fork specifically aimed at CMoS citing? I've just found that chicago-biblatex has language and not langid like biblatex.

[njbart]

Well, the package called biblatex contains both an underlying bibliographic (for lack of a better word) toolkit as well as a few specific bibliographic styles (such as numeric, authoryear, …). While I think these styles are well thought out, consistent, and usable, they don't follow any of the standard style manuals like APA, MLA, Chicago, etc.

This is where biblatex-chicago, biblatex-apa and others come into play, which use the biblatex "toolkit" to implement the rules of the various style manuals. So this certainly is not a question of biblatex-chicago supplanting biblatex (the toolkit). If at all, biblatex-chicago adds to the list of styles available in the, if you wish, biblatex universe.

I've just found that chicago-biblatex has language and not langid like biblatex.

Not sure where you got that from. (If chicago-biblatex is a different package, I haven’t heard of it.) The biblatex-chicago example .bib files, e.g., https://mirrors.ctan.org/macros/latex/contrib/biblatex-contrib/biblatex-chicago/doc/dates-test.bib, do contain both language and langid, and use them in the exact same way biblatex itself does, as described in the biblatex manual. This is even explained in the annotation fields of various entries in dates-test.bib, e.g., garaud:gatine, chu:panda, or pirumova.

[retorquere]

I was just wrong on that, I hadn't looked through the whole manual.