Sorting by title is broken

[matthiasgeiger]

JabRef version 3.6 and current master

As reported in the forum sorting by title does not sort ALL entries correctly:

In fact there seem to be two independent lists which are sorted correctly. Without a look at the good I could not find a pattern which entries are assigned to which list...

[matthiasgeiger]

Okay I found the reason... { } are not ignored during sort. Thus all titles with curly braces appear before the remaining entries.

[mzagorowska]

Thank you for explanation about the title. At least it can sort two lists separately. Unfortunately, it doesn't explain sorting by 'times-cited' or 'year' (fields with numbers).

Every entry which has this field comes from the same source and every one is in {}. It seems that Jabref cannot sort the number inside the brackets, because all entries without them are sorted correctly.

[mlep]

It looks like JabRef consider the 'times-cited" as a text and not as a number.

[mzagorowska]

But it managed somehow to sort the titles inside {} (just takes { as first letter), yet it cannot do the same with 'numbers treated as text'. The same things happens with year which is much more popular field than times-cited.

[lenhard]

Indeed, @mlep is right, times-cited and year are sorted based on their textual representation.

@mzagorowska In this representation, the sorting is absolutely fine. You can reproduce the same behavior in the folder system of your operating system (at least in windows). If you create folders with names '2', '201', etc. they will be sorted in exactly the same fashion as above.

This is just to explain the behavior, not to justify it. Of course it would make sense to interpret these columns as numbers and sort them according to their numerical order. It might make sense for a few more fields that typically store numerical values, such as the volume of an article.

Since there are no mandatory type definitions for fields in BibTeX, JabRef does not really take this into account so far and I cannot say how easy or difficult it will be to implement this. But I think this would be a sensible enhancement.

[matthiasgeiger]

Year sorting seems to be numeric - at least in current master.

[mzagorowska]

Not really:

(every year here is in {}).

[matthiasgeiger]

Just to make sure:
If you switch to the source panel in the editor is format shown there for year:
year = {{2000}},

or

year = {2000},?

[mzagorowska]

The first one. Full example of an entry:
@InProceedings{Adaptive_Wang2010,
author = {Wang, Wei and Wen, Changyun},
title = {{Adaptive Output Feedback Controller Design For A Class of Uncertain Nonlinear Systems with Actuator Failures}},
booktitle = {{49TH IEEE CONFERENCE ON DECISION AND CONTROL (CDC)}},
year = {{2010}},
language = {{English}},
series = {{IEEE Conference on Decision and Control}},
note = {{49th IEEE Conference on Decision and Control (CDC), Atlanta, GA, DEC 15-17, 2010}},
organization = {{IEEE}},
publisher = {{IEEE}},
isbn = {{978-1-4244-7746-3}},
pages = {{1749-1754}},
doi = {{10.1109/CDC.2010.5717733}},
abstract = {{In this paper, we aim to design adaptive output feedback controllers for
a class of uncertain nonlinear multiple-input single-output (MISO)
systems in the presence of stuck type actuator failures. The class of
MISO system has a characteristic that the relative degrees with respect
to each of the inputs are not necessarily the same. We first design a
pre-filter for each input and then derive the state space representation
of a newly constructed plant. The design of control variable will be
provided by making certain modifications on traditional backstepping
technique. It will be shown that with our proposed scheme, the effects
due to actuator failures can be compensated and global boundedness of
the closed-loop signals is still ensured. Further, the system output can
asymptotically track a given reference signal.}},
address = {{345 E 47TH ST, NEW YORK, NY 10017 USA}},
affiliation = {{Wang, W (Reprint Author), Nanyang Technol Univ, Sch Elect & Elect Engn, Nanyang Ave, Singapore 639798, Singapore. Wang, Wei; Wen, Changyun, Nanyang Technol Univ, Sch Elect & Elect Engn, Singapore 639798, Singapore.}},
author-email = {{wang0336@ntu.edu.sg ecywen@ntu.edu.sg}},
book-group-author = {{IEEE}},
doc-delivery-number = {{BWV28}},
file = {:Adaptive_Output_Feedback_Controller_Design_Wang2010.pdf:PDF},
groups = {Reconfiguration},
issn = {{0191-2216}},
keywords-plus = {{FAULT-TOLERANT CONTROL; RECONFIGURABLE FLIGHT CONTROL; TRACKING CONTROL; COMPENSATION; ACCOMMODATION}},
number-of-cited-references = {{22}},
research-areas = {{Automation & Control Systems; Engineering}},
researcherid-numbers = {{Wen, Changyun/A-5018-2011}},
times-cited = {{5}},
type = {{Proceedings Paper}},
unique-id = {{ISI:000295049102016}},
usage-count-last-180-days = {{0}},
usage-count-since-2013 = {{3}},
web-of-science-categories = {{Automation & Control Systems; Engineering, Electrical & Electronic}},
}

[matthiasgeiger]

Okay, thank you!

The problems are caused by the double-braces {{ }} used by you in various fields. These braces are fine for text fields whose casing should be protected, e.g., title, but they should not be used in fields like "times-cited" or "years" are in this fields just numbers are expected.

Nonetheless, the incorrect sorting of title still is a bug.

Have you put those fields manually or have you configured JabRef to do this?

[mzagorowska]

OK, this might be a problem. Unfortunately, such formatting was not my idea, such entries are taken directly from Web of Science database. Apparently, I cannot just copy and paste the .bib file from their website. Thank you for your help!

But as you have said, the title problem remains.

[matthiasgeiger]

Find-ReplaceAll should solve most of your problems: Just replace {{ with { and }} with }

However, please make a backup of the bib-file first 😉

[stefan-kolb]

To get a numeric sorting for any custom field, we would need to make the field type numeric inside our implementation (I think there are several field types like month etc pp that are treated differently). I don't think this is possible for the user at the moment but only inside the code.

[mlep]

@matthiasgeiger @mzagorowska The find-replace may corrupt your database.
example: title={{C}hina and the {P}acific ocean},

[stefan-kolb]

Regarding the sorting bug, we need to use the latex free version of the entry title in case we want to have such a sorting.

[mzagorowska]

@mlep Exactly, I think that {{ is sometimes necessary (as in your example). We have a choice between correct sorting or correct title. The only solution I can think of is to have every title in {{ }}, but I have no idea how to change existing entries (not manually).

[mlep]

@mzagorowska Generally, there is no reason to have a whole content between {{ and }}. Maybe you can ask on the forum how to change this issue without trouble.

[stefan-kolb]

@JabRef/developers What is our decision on this? Should we ignore LaTeX commands for sorting? I tend to say so.

[koppor]

👍 for ignoring the LaTeX commands - think, we should just use the latex-free version of the BibEntry, shouldn't we?

[matthiasgeiger]

Yes. However, this sorting stuff is organized using the GlazedLists lib - thus, requires some time to dig into the details how to change the current behavior.

[lenhard]

Does the LaTeX-free version really solve the problem here? I mean does that actually remove the double braces? Because the bracing isn't really LaTeX-specific, is it?

[koppor]

JavaFX? 😇

[matthiasgeiger]

At least running "Latex To Unicode" removes the braces...

[lenhard]

Just a note regarding sorting (that we might want to discuss in another issue maybe): The sorting of titles in libraries and also in other reference managers, such as EndNote, avoids the words "A", "An", or "The" at the beginning of titles for the sorting. The reason for this is that you usually have a huge amount of entry titles starting with this and if you take into account "A" the sorting becomes less useful. We might want to consider doing the same in JabRef.

Personally, I like it more if we take these words into account for the sorting, but I have been told that the other way is the professional one.