BibTeX/RIS: Add DOIs to non-supported item types #1233

adam3smith · 2017-01-22T22:09:17Z

BibTeX: also add additional test, udpated tests, and add better
handling of additional identifiers from notes (which were in my local
bibtex either from an old version by Emiliano or from an attempt by me;
either way they look good).

BibTeX: also add additional test, udpated tests, and add better handling of additional identifiers from notes (which were in my local bibtex either from an old version by Emiliano or from an attempt by me; either way they look good).

Also, some minor touch-ups like cleaning out tabs and spaces from empty lines, adding semicolons, removing deprecated code.

update tests (manually)

zuphilip

Some comments are below. This looks good for me in general.

zuphilip · 2017-01-23T06:42:28Z

CrossRef.js

@@ -331,13 +335,10 @@ function doSearch(item) {
 	} else {
 		var co = Zotero.Utilities.createContextObject(item);
 	}
-	
+
 	Zotero.Utilities.HTTP.doGet("http://www.crossref.org/openurl/?pid=zter:zter321&"+co+"&noredirect=true&format=unixref", function(responseText) {


This function call looks also a bit strange. Normally we just have ZU.doGet and I would not be sure that the ZU.HTTP functions are (always) available in translator code.

zuphilip · 2017-01-23T06:53:23Z

CrossRef.js

 	item.DOI = ZU.xpathText(refXML, 'c:doi_data/c:doi', ns);
+	//add DOI to extra for unsupprted items
+	if (item.DOI && item.itemType != "journalArticle" && item.itemType != "conferencePaper") {
+		item.extra = "DOI: " + item.DOI;


How about switching here also to the more future-proof version instead of the one-liner as you have in the others already?

if (item.extra) { item.extra += '\nDOI: ' + value; } else { item.extra = 'DOI: ' + value; }

zuphilip · 2017-01-23T06:55:55Z

RIS.js

@@ -1451,6 +1452,19 @@ function applyValue(item, zField, value, rawLine) {
 				item.extra = value;
 			}
 		break;
+		case 'DOI':
+			if (!ZU.fieldIsValidForType("DOI", item.itemType) && ZU.cleanDOI(value)) {
+				value = ZU.cleanDOI(value);


Nits: If you switch these two lines, then you only have to calculate cleanDOI once.

zuphilip · 2017-01-23T07:01:17Z

RIS.js

+			else {
+				item[zField] = value;
+			}
+		break;


Alternatively, you can delete the else above and just continue without a break, which means that for DOI all default things will be performed and (depending on the itemType) also some more, i.e. delete these lines

else { item[zField] = value; } break;

zuphilip · 2017-01-23T07:05:43Z

BibTeX.js

@@ -262,7 +263,14 @@ function setKeywordDelimRe( val, flags ) {
 function processField(item, field, value, rawValue) {
 	if(Zotero.Utilities.trim(value) == '') return null;
 	if(fieldMap[field]) {
-		item[fieldMap[field]] = value;
+		//map DOIs + Label to Extra for unsupported item types
+		if (field == "doi" && item.itemType != "journalArticle" && item.itemType != "conferencePaper") {


I like the (more general) function ZU.fieldIsValidForType("DOI", item.itemType) which you used in another translator. How about using it here as well?

zuphilip · 2017-01-23T07:06:16Z

CrossRef.js

 	item.DOI = ZU.xpathText(refXML, 'c:doi_data/c:doi', ns);
+	//add DOI to extra for unsupprted items
+	if (item.DOI && item.itemType != "journalArticle" && item.itemType != "conferencePaper") {


I like the (more general) function ZU.fieldIsValidForType("DOI", item.itemType) which you used in another translator. How about using it here as well?

zuphilip · 2017-01-23T07:09:18Z

BibTeX.js

@@ -262,7 +263,14 @@ function setKeywordDelimRe( val, flags ) {
 function processField(item, field, value, rawValue) {
 	if(Zotero.Utilities.trim(value) == '') return null;
 	if(fieldMap[field]) {
-		item[fieldMap[field]] = value;
+		//map DOIs + Label to Extra for unsupported item types


We could also let the first line as it was and just do some extra work for the doi, i.e. doing the else case below always.

I'll leave this here -- I'd like to leave the DOI in the generic field mapping (l. 84) and the only way to get to else would be to remove it there.

Not sure we understand each other here. My suggestion would look like this:

if(fieldMap[field]) { //default behaviour item[fieldMap[field]] = value; //additionally map DOIs + Label to Extra for unsupported item types if (!ZU.fieldIsValidForType("DOI", item.itemType)) { var label = extraIdentifiers[field]; item._extraFields.push({field: label, value: value.trim()}); }

Ah I see. I figured it'd be better if the item isn't mapped to item.DOI when it's written to Extra, so that we don't get DOI twice when using bibtex from a web translator that also adds DOIs (I know a couple do). What's the advantage of doing this your way around?

Unsupported fields will not be saved in Zotero such that there will never be two fields containing the DOI information. No?

Anyway, there is no big deal here. I just thought it might be easier to read (and maybe already work correctly in some future version of Zotero item types).

I will review your changes a little later, but I am sure everything is good.

but they're available in the translator object before saving, so something like this in a web translator:
https://github.com/zotero/translators/blob/master/Springer%20Link.js#L158
would cause DOI: 10.1234/123456 to appear twice in the Extra field.

@zuphilip

have another look, please @zuphilip

zuphilip · 2017-01-25T21:37:51Z

RIS.js

-			else {
-				item[zField] = value;
-			}
-		break;


This change I caused with my previous comment, might produce now that the doi data will be saved in item.extra as well as in item.DOI in the translator obejct as you outlined in #1233 (comment) . Thus, we might switch back here again to the else case and the break to prevent that.

I think I have an elegant solution now moving the break into the if condition

zuphilip · 2017-01-25T21:38:25Z

RIS.js

@@ -2069,7 +2065,7 @@ var testCases = [
 				"abstractNote": "Abstract",
 				"archive": "Name of Database",
 				"archiveLocation": "Accession Number",
-				"extra": "Publication Number",
+				"extra": "DOI: 10.1234/123456\nPublication Number",


Why did this value vanish now?

I don't understand -- it's still there after the line break.

Sorry, my fault. Somehow I switched the meaning of green and red here in my mind. The doi is here, which is good.

zuphilip · 2017-01-25T21:40:09Z

CrossRef.js

@@ -293,7 +293,12 @@ function processCrossRef(xmlOutput) {
 	item.DOI = ZU.xpathText(refXML, 'c:doi_data/c:doi', ns);
 	//add DOI to extra for unsupprted items
 	if (item.DOI && item.itemType != "journalArticle" && item.itemType != "conferencePaper") {
-		item.extra = "DOI: " + item.DOI;


How about using !ZU.fieldIsValidForType("DOI", item.itemType) here as well?

somehow forgot this; done.

zuphilip

Looks good!

adam3smith · 2017-01-26T13:29:28Z

great, thanks

zuphilip · 2017-01-26T15:25:56Z

Thank you @adam3smith for the work! This looks really great.
(And sorry for my sloppy reviews, I am a little distracted by other things which are going on...)

BibTeX: also add additional test, udpated tests, and add better handling of additional identifiers from notes (which were in my local bibtex either from an old version by Emiliano or from an attempt by me; either way they look good). * CrossRef: Also, some minor touch-ups like cleaning out tabs and spaces from empty lines, adding semicolons, removing deprecated code.

adam3smith added 3 commits January 22, 2017 17:01

BibTeX/RIS: Add DOIs to non-supported item types

320f93b

BibTeX: also add additional test, udpated tests, and add better handling of additional identifiers from notes (which were in my local bibtex either from an old version by Emiliano or from an attempt by me; either way they look good).

CrossRef: add DOIs to unsupported itemTypes

73398ef

Also, some minor touch-ups like cleaning out tabs and spaces from empty lines, adding semicolons, removing deprecated code.

CrossRef: Bump timestamp

81a9211

update tests (manually)

zuphilip reviewed Jan 23, 2017

View reviewed changes

zuphilip mentioned this pull request Jan 23, 2017

BibTeX import: "Discarding invalid field 'publisher' for type 4" #1228

Open

Address review comments

2b30896

have another look, please @zuphilip

zuphilip mentioned this pull request Jan 25, 2017

Translator für Katalogisierung von Aufsätzen aus Sammelwerken im GBV UB-Mannheim/zotkat#42

Closed

zuphilip reviewed Jan 25, 2017

View reviewed changes

Address more comments

9f39722

zuphilip approved these changes Jan 26, 2017

View reviewed changes

adam3smith merged commit 74ea87b into zotero:master Jan 26, 2017

adam3smith deleted the DOIs branch January 26, 2017 13:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BibTeX/RIS: Add DOIs to non-supported item types #1233

BibTeX/RIS: Add DOIs to non-supported item types #1233

adam3smith commented Jan 22, 2017

zuphilip left a comment

zuphilip Jan 23, 2017

zuphilip Jan 23, 2017

zuphilip Jan 23, 2017

zuphilip Jan 23, 2017

zuphilip Jan 23, 2017

zuphilip Jan 23, 2017

zuphilip Jan 23, 2017

adam3smith Jan 25, 2017

zuphilip Jan 25, 2017 •

edited

Loading

adam3smith Jan 25, 2017

zuphilip Jan 25, 2017

adam3smith Jan 25, 2017

zuphilip Jan 25, 2017

zuphilip Jan 25, 2017 •

edited

Loading

adam3smith Jan 26, 2017

zuphilip Jan 25, 2017

adam3smith Jan 25, 2017

zuphilip Jan 25, 2017

zuphilip Jan 25, 2017

adam3smith Jan 26, 2017

zuphilip left a comment

adam3smith commented Jan 26, 2017

zuphilip commented Jan 26, 2017

BibTeX/RIS: Add DOIs to non-supported item types #1233

BibTeX/RIS: Add DOIs to non-supported item types #1233

Conversation

adam3smith commented Jan 22, 2017

zuphilip left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zuphilip Jan 25, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zuphilip Jan 25, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zuphilip left a comment

Choose a reason for hiding this comment

adam3smith commented Jan 26, 2017

zuphilip commented Jan 26, 2017

zuphilip Jan 25, 2017 •

edited

Loading

zuphilip Jan 25, 2017 •

edited

Loading