Skip to content
This repository

Indexing more than one field #31

Closed
maieul opened this Issue August 19, 2012 · 32 comments

4 participants

Maïeul Rouquette Audrey Boruvka plk PaulStanley
Maïeul Rouquette

Hello,

I would like to index more than one fied, to have index entry like author!title!pages. I did'tn see how can I do.

For yet, I index these 3 fields separtly and concate them bt python script.

plk
Owner
plk commented August 22, 2012

I'm not sure this is currently possible as the indexing of say "author" uses an internal macro which has to look at the structure of the name - it's not as simple as just \index{field}. It may be possible to implement something but it looks tricky. @josephwright or @aboruvka?

Maïeul Rouquette

Yes, I understand it's not easy. Right now, I have just make a python scritpt maieul/indexation-sources, but I think it's not very easy to use for other people (it's need terminal)

Audrey Boruvka
Collaborator

One workaround you could try is to save the indexsorttitle, title and pages fields and use these in a bibliography macro like index:name. I can look into a real fix over the weekend.

Maïeul Rouquette
Audrey Boruvka
Collaborator

@maieul I'm not sure what you mean. Coincidentally Paul demonstrated a workaround similar to the one I mentioned here: http://tex.stackexchange.com/a/68208.

Audrey Boruvka
Collaborator

You can extend the existing bibliography macros in biblatex.def to index multiple fields. Care is needed in expansion and sorting. Index entries for titles, for example, should have the form <indexsorttitle>@<indextitle>. Here <indextitle> should not be fully expanded to ensure correct formatting in the "actual"/printed index entries. Control sequences like \TeX should be protected (\protect\TeX) unless you're using a robust indexing package like index. Indexing page references in the postnote field isn't straightforward. In the code below, page ranges are sorted only using the starting value.

\makeatletter
\def\thefirstpage#1{\thefirstpage@i#1-&}
\def\thefirstpage@i#1-#2&{#1}
\makeatother

\DeclareIndexNameFormat{title:pages}{%
  \ifboolexpr{ not test {\iffieldundef{title}} and test {\iffieldnums{postnote}} }
    {\usebibmacro{index:entry}{\index}{%
       \mkbibindexname{#1}{#3}{#5}{#7}%
       \mkbibindexsubentry
         {\thefield{indexsorttitle}}{\emph{\csfield{indextitle}}}%
       \mkbibindexsubentry
         {\usefield{\thefirstpage}{postnote}}{\usefield{\mkfirstpage}{postnote}}}}
    {}}

\renewbibmacro*{citeindex}{%
  \ifciteindex{\indexnames[title:pages]{labelname}}{}}

There are multiple ways to specify page ranges. This just handles range delimiters starting with a hyphen.

PaulStanley

I think it's likely to be very hard to offer a suitable interface, for a range of reasons. Among other things: the very wide range of possible different use cases; the different formats used by different multiple index packages; problems with expansion and so forth.

It seems to me that this may be more a documentation issue than a package coding issue. Any sophisticated indexing system (beyond that already provided, I mean) is going to need to be written by delving rather deeply into some rather difficult stuff, especially with expansion. It's hard (at least, I find it very hard). But it's not impossible, even as things stand, and for my own part I'd rather doubt that the effort that would have to go into developing a fully flexible system at the package level would be worthwhile, because I think you'd either end up with something simply but inflexible or something immensely complex. So some modest development along the lines Audrey suggests seems exactly right to me, FWIW.

plk
Owner
plk commented August 23, 2012

I agree - it's impossible to do a generic interface - I think Audrey's documentation solution is a very good idea. I'm not even sure there is much point putting a new macro in biblatex.def, it's fine in the docs?

Audrey Boruvka
Collaborator

I think the author-title index is fairly common. This motivates putting a new indexing macro in biblatex.def. If you have a strong preference either way, I'll go with that. Just let me know. I can push the changes in the next few days.

Maïeul Rouquette
Audrey Boruvka
Collaborator

@maieul This is easier said than done. The \theindexentry needs to be expanded inside a formatting directive; you can't simply build an index entry from the output of three separate directives. I suppose we could define multi-field formatting directives, but we would need to know the field data types (e.g. name list, literal list, field) and iterating through multiple list fields would be messy. If we go through all this trouble and still put the onus of of proper sorting and expansion on the user, I'd have a hard time seeing an improvement of this new interface over the current one.

Maïeul Rouquette
plk
Owner
plk commented August 23, 2012

I will bow to Audrey on this - I am not as competent to judge it, so whatever you think is best.

PaulStanley

Just a quick note on Audrey's code above, which I have been playing with overnight (largely in order to correct an answer I wrote on tex/sx) ... with some frustration.

Audrey's code works perfectly if we are using index to create multiple indexes: but it fails for multind or imakeidx. If you run it through them, the indextitle field still seems to end up fully expanded. I only have a suspicion about why that's happening (there's a comment in the code in biblatex.def which gives a hint that different systems expand differently); and I don't really understand why the "ordinary" title indexing system (which "expands" the indextitle under a \DeclareIndexFormat control and then passes it to the indexing macro) doesn't. Even simple expansion issues tend to puzzle me, and while I think I understand Audrey's code I can't pretend to have a clear grasp of everything that's going on.

At the end of this comment I've posted an example file which can be used to play around with: basically you can comment or uncomment some lines at the start and end, and see the results using either index, multind or imakeidx.

I do have a tentative workaround, which I think will be "index-backend-agnostic": if, instead of trying to (partially) expand the indextitle field one simple replaces it in the indexing command with an unexpanded \citefield{\thefield{entrykey}}{indextitle}, then we avoid expansion trouble: in other words, we seem to be OK with zero expansion, but even partial expansion in the edef trips us up. I think this works, though it vexes me to get there by trial or error rather than logic. (The technique is one I have used in oscola, where I needed to print very complex index entries, effectively full citations; but that's another story.) I've also put this alternative in the example, but commented out, if anyone wants to play with it.

I guess what this does underline is the difficulty of arriving at GENERAL solutions, given the need to interface with other packages with their own quirks. Good luck deciding what to do!

\documentclass{article}
\usepackage[style=verbose,indexing=cite,backend=biber]{biblatex}
\addbibresource{biblatex-examples.bib}
% using index
%\usepackage{index}
%\newindex{authortitle}{adx}{and}{Author Title Index}
% using imakeindex
 \usepackage{imakeidx}
 \makeindex[name=authortitle, title={Author Title Index}]

% using multind
%\usepackage{multind}
%\makeindex{authortitle}
\makeatletter
\def\thefirstpage#1{%
  \expandafter\expandafter\expandafter\firstpage@i
  \expandafter\expandafter\expandafter{\csname abx@field@#1\endcsname}}
\def\firstpage@i#1{\firstpage@ii#1-\@nil}
\def\firstpage@ii#1-#2\@nil{#1}
 \newbibmacro*{index:name:title:pages}[5]{%
  \begingroup
  \ifuseprefix
    {\def\abx@tempa{%
       \ifblank{#4}{}{#4 }%
       \@firstofone #2% remove spurious braces
       \ifblank{#5}{}{ #5}%
       \ifblank{#3}{}{, #3}%
       \actualoperator
       \ifblank{#4}{}{\MakeCapital{#4} }%
       #2%
       \ifblank{#5}{}{ #5}%
       \ifblank{#3}{}{, #3}}}
    {\def\abx@tempa{%
       \@firstofone #2% remove spurious braces
       \ifblank{#5}{}{ #5}%
       \ifblank{#3#4}{}{,}%
       \ifblank{#3}{}{ #3}%
       \ifblank{#4}{}{ #4}}}%
  \savefield{indextitle}{\abx@tempb}%
  \iffieldpages{postnote}
    {\protected@edef\theindexentry{\unexpanded{#1}{%
       \abx@tempa!\thefield{indexsorttitle}%
       \actualoperator\emph{\expandonce\abx@tempb}%
       !\thefirstpage{postnote}\actualoperator\thefield{postnote}}}}
    {\protected@edef\theindexentry{\unexpanded{#1}{%
       \abx@tempa!\thefield{indexsorttitle}%
       \actualoperator\emph{\expandonce\abx@tempb}}}}%
  \theindexentry
  \endgroup}
% ALTERNATIVE APPROACH
%\renewbibmacro*{index:name:title:pages}[5]{%
% \begingroup
%  \ifuseprefix
%   {\def\abx@tempa{%
%       \ifblank{#4}{}{#4 }%
%       \@firstofone #2% remove spurious braces
%       \ifblank{#5}{}{ #5}%
%       \ifblank{#3}{}{, #3}%
%       \actualoperator
%       \ifblank{#4}{}{\MakeCapital{#4} }%
%       #2%
%       \ifblank{#5}{}{ #5}%
%       \ifblank{#3}{}{, #3}}}
%    {\def\abx@tempa{%
%       \@firstofone #2% remove spurious braces
%       \ifblank{#5}{}{ #5}%
%       \ifblank{#3#4}{}{,}%
%       \ifblank{#3}{}{ #3}%
%       \ifblank{#4}{}{ #4}}}%
%  \iffieldpages{postnote}
%    {\protected@edef\theindexentry{\unexpanded{#1}{%
%       \abx@tempa!\thefield{indexsorttitle}%
%       \actualoperator\unexpanded{\citefield}{\thefield{entrykey}}{indextitle}%
%       !\thefirstpage{postnote}\actualoperator\thefield{postnote}}}}
%    {\protected@edef\theindexentry{\unexpanded{#1}{%
%       \abx@tempa!\thefield{indexsorttitle}%
%       \actualoperator\unexpanded{\citefield}{\thefield{entrykey}}{indextitle}}}}%
%  \theindexentry
%  \endgroup}
 \global\newtoggle{usingmultind}
 \AtBeginDocument{%
  \@ifpackageloaded{multind}
    {\@ifpackageloaded{imakeidx}
       {\global\togglefalse{usingmultind}}
       {\global\toggletrue{usingmultind}}}%
    {\togglefalse{usingmultind}}}

\togglefalse{usingmultind}
\makeatother
\DeclareIndexNameFormat{title:pages}{%
  \iftoggle{usingmultind}
   {\usebibmacro{index:name:title:pages}{\index{authortitle}}{#1}{#3}{#5}{#7}}
   {\usebibmacro{index:name:title:pages}{\index[authortitle]}{#1}{#3}{#5}{#7}}}
 \renewbibmacro*{citeindex}{%
  \ifciteindex{\indexnames[title:pages]{labelname}}{}}

    \begin{document}

 \cite{knuth:ct:a}

 \cite[234]{knuth:ct:a}

\cite{knuth:ct:d}

%multind
%\printindex{authortitle}{Author-Title Index}
% imakeidx and index
\printindex[authortitle]
\end{document}
Audrey Boruvka
Collaborator

@PaulStanley Thanks. I've since simplified the code, but need to do more testing. So this is very helpful. Does PL's indextitle formatting directive work here?

PaulStanley

@aboruvka It works perfectly with the existing macros where you simply index the title, and that pumps the indextitle out using a \DeclareIndexFieldFormat... and that in turn gets passed through to the protected@edef wrapped in \unexpanded: then we get "one level" of expansion, whichever indexing package we use (i.e. we "see" the field as written in the .bib file, without any further expansion).

But that deals only with one field, and I couldn't work out how to deal with multiple fields that way (in other words, there is no analogue for an index of the driver that applies to formatting bibliography entries) -- so we are either working on the \IndexNameFormat for author OR on the \IndexFieldFormat for title, but I couldn't work out how to get both at the same time; there's a very complex flow of data through various macros at this point and that, coupled with my own obtuseness about expansion and the interaction with the indexing packages, means I can't pretend to have got to the bottom of it. I'm sure you will.

It's an additional curse that there are so many indexing packages with their little quirks and some voodoo of their own, but I suppose that's just a fact to be faced.

Audrey Boruvka
Collaborator

@PaulStanley Works perfectly with every package? imakeidx fully expands \TeX in the titles on my end.

PaulStanley

@aboruvka You might be right. I don't think I tested with imakeidx last night; I think I did test with multind, which was OK. I did no testing at all with splitindex as such. (My code above does work with imakeidx ... unless I'm going mad!)

Audrey Boruvka
Collaborator

@PaulStanley Sorry for the confusion - I didn't test your code. I'm just trying to see if example 21 runs with packages other than index. The reasoning being that if PL's directives don't work with some packages, we may not want to bother pursuing a solution for them. imakeidx seems problematic because it doesn't generate "self-protecting" entries like index. Do you mind if I bother you via email when I look at this again in the next few days? I think I need some help with testing. We should also look at what you're doing with oscola in considering any new indexing features. @plk I'll copy you on any correspondence.

plk
Owner
plk commented August 24, 2012

I really appreciate this - it's beyond my TeX abilities. Maybe @PaulStanley would consider joining the biblatex development team? My emphasis is usually on biber and it would be good to have another TeX person sine Joseph is often very busy with latex3 things.

PaulStanley

@aboruvka By all means -- though I have a very heavy week of teaching next week so I don't guarantee to be very immediately responsive. I'd rather you didn't look too hard at oscola, because the indexing is rather a mess and needs cleaning up, though it appears to be working. I had difficulty getting neat expansion of macros which "pick apart" references like 1(2) so as to produce \index{blah@blah!1!2} and the result was what I think Andrew Stacey called "cargo-cult" code which would shock your sensibilities. But the basic idea of dealing with difficult formatting by writing an unexpanded \cite command to the index file, and actually doing the typesetting when it comes back in may not be totally mad.

(Probably all you really need to know is that I define a citation command (\citeinindex) which never indexes itself or bothers with prenotes and postnotes, and which calls a variant driver for each entry-type to print suitable text. The primary sorted part of the index I deal with using indexsorttitle or other fields that get constructed by biber in some cases. I then write the \citeinindex completely unexpanded via the \index macro, so that all the indexing software ever sees is {blah@\citeindex{foo}!w@x!y@z. And do bear in mind that of necessity this is (a) designed for a specialised use where macro problems are unlikey to occur and (b) only tested so far on a limited set of data. I certainly wasn't trying to solve the problem you are dealing with here.)

There is an advantage to supporting at least one of imakeidx or splitindex, given the hardwired limitation on output files. It might seem absurd to suppose that one might need nine or ten separate indexes ... but I'm afraid legal writing is replete with absurdity, and I don't think index will do the job. But it might be reasonable to put the onus on style authors who want to support them to do the legwork. Biblatex is a Swiss army knife, to be sure, but even then there are just so many blades and widgets you can support.

Audrey Boruvka
Collaborator

@PaulStanley Great. Thanks. The indexing syntax is generic so it'll support the standard indexing packages; the results just may not be as robust as they are with index. I think we can make up for that with some additional documentation. imakeidx users for example will occasionally have to protect fields used in the index: indextitle = {\protect\TeX book}. Regarding oscola - The code I have going right now adds more auxiliary commands for low-level access to unformatted fields. You might find them helpful. I'll pickup this discussion via email soon.

PaulStanley

@plk I doubt that I would contribute usefully as a member of the team in any formal sense: I don't have Audrey's mastery, just a bunch of war wounds, acquired in a rather obscure battle! But I'm certainly happy to help as much as I can. FWIW my next biblatexy project was to try to produce some sort of tutorial introduction, perhaps especially on adaptation, because quite a lot of people find the comprehensive documentation, excellent as it is, rather forbidding.

plk
Owner
plk commented August 24, 2012

Well, that tutorial introduction has been very much needed and requested so if you would tackle that, it would be great and we'd certainly include it with biblatex.

Audrey Boruvka
Collaborator

The referenced commit provides some extensions to the indexing macros. You can build index entries using the bibmacro index:entry{<index command>}{<index command argument>}. Additional macros aid in formatting the entry. I've updated the code I posted above to give an example.

Maïeul Rouquette

thank a lot. I will made test after closing some ticket in maieul/ledmac. Thank

Maïeul Rouquette

Hello, I juste made test.

with a single example :


\renewbibmacro*{citeindex}{%

\ifciteindex
{\indexnames[name-title]{labelname}}{}}
\DeclareIndexNameFormat{name-title}{
\usebibmacro{index:entry}{\index[name-title]}{%
\mkbibindexname{#1}{#3}{}{#5}%
\subentryoperator%
\mkbibindexfield{\thefield{indexsorttitle}}{\emph{\thefield{title}}}
\subentryoperator\mkbibindexfield{\thefield{titleaddon}}{\thefield{titleaddon}}}
}

(i will add test for empty field later and for sorting).

I think the issue can be closed.

Maïeul Rouquette

and so think a lot !

Maïeul Rouquette maieul closed this September 02, 2012
Maïeul Rouquette maieul reopened this September 02, 2012
Audrey Boruvka
Collaborator

@maieul Are the new macros sufficient? Can we close this one?

Maïeul Rouquette
Audrey Boruvka aboruvka closed this September 06, 2012
Audrey Boruvka
Collaborator

@maieul OK. Thanks for this feature request. I think we've made some improvements here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.