Search translation issues

[dstillman]

Looking at D1129911041, for an Add by Identifier attempt, I'm seeing some odd things.

First, probably no actual problem, but the output is a bit alarming:

(+0000000): JSLOG: (3)(+0000003): Extracted identifiers: [{"DOI":"10.1007/s11704-017-6132-7"}]

(+0000002): JSLOG: (3)(+0000000): Get translators

(+0000000): JSLOG: (4)(+0000005): Translate: Binding sandbox to http://www.example.com/

(+0000000): JSLOG: (4)(+0000002): Translate: Parsing code for EIDR (79c3d292-0afc-42a1-bd86-7e706fc35aa5, 2017-06-03 11:41:00)

(+0000005): JSLOG: (2)(+0000002): Translate: Detect using EIDR failed: 
TypeError: undefined is not an object (evaluating 'item.DOI')

detectSearch@
@file:///private/var/containers/Bundle/Application/FD775D95-1298-4882-8ADB-49286A9C031B/Zotero.app/translation/translate/src/translation/translate.js:1712:51
_detectTranslatorLoaded@file:///private/var/containers/Bundle/Application/FD775D95-1298-4882-8ADB-49286A9C031B/Zotero.app/translation/translate/src/translation/translate.js:1702:44
@file:///private/var/containers/Bundle/Application/FD775D95-1298-4882-8ADB-49286A9C031B/Zotero.app/translation/translate/src/translation/translate.js:1692:45

(+0000000): JSLOG: (4)(+0000000): Translate: Parsing code for Crossref REST (0a61e167-de9a-4f93-a68a-628b48855909, 2018-06-16 12:00:00)

(+0000000): JSLOG: (4)(+0000001): Translate: Parsing code for WHO (cd587058-6125-4b33-a876-8c6aae48b5e8, 2022-12-06 12:21:28)

(+0000000): JSLOG: (2)(+0000001): Translate: Detect using WHO failed: 
TypeError: undefined is not an object (evaluating 'item.ISBN')

detectSearch@
@file:///private/var/containers/Bundle/Application/FD775D95-1298-4882-8ADB-49286A9C031B/Zotero.app/translation/translate/src/translation/translate.js:1712:51
_detectTranslatorLoaded@file:///private/var/containers/Bundle/Application/FD775D95-1298-4882-8ADB-49286A9C031B/Zotero.app/translation/translate/src/translation/translate.js:1702:44
@file:///private/var/containers/Bundle/Application/FD775D95-1298-4882-8ADB-49286A9C031B/Zotero.app/translation/translate/src/translation/translate.js:1692:45

(+0000000): JSLOG: (4)(+0000000): Translate: Parsing code for Library of Congress ISBN (c070e5a2-4bfd-44bb-9b3c-4be20c50d0d9, 2022-02-07 18:32:41)

(+0000000): JSLOG: (2)(+0000000): Translate: Detect using Library of Congress ISBN failed: 
TypeError: undefined is not an object (evaluating 'item.ISBN')

detectSearch@
@file:///private/var/containers/Bundle/Application/FD775D95-1298-4882-8ADB-49286A9C031B/Zotero.app/translation/translate/src/translation/translate.js:1712:51
_detectTranslatorLoaded@file:///private/var/containers/Bundle/Application/FD775D95-1298-4882-8ADB-49286A9C031B/Zotero.app/translation/translate/src/translation/translate.js:1702:44
@file:///private/var/containers/Bundle/Application/FD775D95-1298-4882-8ADB-49286A9C031B/Zotero.app/translation/translate/src/translation/translate.js:1692:45

(+0000000): JSLOG: (4)(+0000000): Translate: Parsing code for National Library of Poland ISBN (aa7f310e-10d3-4209-91dc-88301e7070c6, 2023-06-15 02:35:00)

(+0000000): JSLOG: (2)(+0000000): Translate: Detect using National Library of Poland ISBN failed: 
TypeError: undefined is not an object (evaluating 'item.ISBN')

detectSearch@
@file:///private/var/containers/Bundle/Application/FD775D95-1298-4882-8ADB-49286A9C031B/Zotero.app/translation/translate/src/translation/translate.js:1712:51
_detectTranslatorLoaded@file:///private/var/containers/Bundle/Application/FD775D95-1298-4882-8ADB-49286A9C031B/Zotero.app/translation/translate/src/translation/translate.js:1702:44
@file:///private/var/containers/Bundle/Application/FD775D95-1298-4882-8ADB-49286A9C031B/Zotero.app/translation/translate/src/translation/translate.js:1692:45

Instead of a TypeError and stack trace, it would be better to just gracefully handle an id of the expected type not being found.

I'm also confused by the EIDR one, though, since item.DOI should exist. Something with EIDR taking an object and DOI Content Negotiation and others taking an array (which I'm actually confused by to begin with).

And I guess I'm generally confused by the output for this operation. It seems to be doing the above process and then saying translate called without specifying a translator. Running detection first. and running detection again.

And then, the reason I was looking at this to begin with, 10.1007/s11704-017-6132-7 is failing on iOS, whereas it works on desktop.

(Perhaps related, there's a problem with the Airiti translator in that output — marked as a search translator but no doSearch. I've fixed that but wasn't getting the updated translator, so can't test.)

[dstillman]

(The DOI is from https://forums.zotero.org/discussion/106638/ios-add-by-identifier-fails-with-particular-doi)

[michalrentka]

@dstillman I checked the EIDR translator, added some logs and the whole item in function detectSearch(item) is undefined when called first time, but then it has an instance, when called again. Better description bellow.

And I guess I'm generally confused by the output for this operation. It seems to be doing the above process and then saying translate called without specifying a translator. Running detection first. and running detection again.

I'm slightly confused by this too, but let me explain briefly how lookup works on iOS and you'll probably have better idea what's wrong since you now the internals of translation too:

• First I load all translators and send them to web view -

  zotero-ios/ZShare/Assets/translation/lookup.js

  Lines 26 to 32 in 3c174ba

  	function initTranslators(encodedTranslators) {
  	const translatorData = JSON.parse(decodeBase64(encodedTranslators));
  	if (translatorData) {
  	Zotero.debug("Init translators");
  	Zotero.Translators.init(translatorData);
  	}
  	}

• Then I create Zotero.Translate.Search instance and setTranslator all translators -

  zotero-ios/ZShare/Assets/translation/lookup.js

  Lines 69 to 81 in 3c174ba

  	var translate = new Zotero.Translate.Search();
  	try {
  	Zotero.debug("Get translators");
  	// Get translators
  	var translators = await translate.getTranslators();
  	translate.setTranslator(translators);
  	} catch (e) {
  	// Continue with other ids on failure
  	Zotero.logError(e);
  	window.webkit.messageHandlers.failureHandler.postMessage(1);
  	return;
  	}

• What confuses me is that at this point we're getting those TypeError messages, so translation is already going on in background? But anyway next we go identifier by identifier and try to translate -

  zotero-ios/ZShare/Assets/translation/lookup.js

  Lines 90 to 99 in 3c174ba

  	try {
  	translate.setIdentifier(identifier);
  	let items = await translate.translate({ libraryID: false, collections: false, saveAttachments: true });
  	window.webkit.messageHandlers.itemsHandler.postMessage({"identifier": identifier, "data": items});
  	} catch (e) {
  	// Continue with other ids on failure
  	Zotero.logError(e);
  	window.webkit.messageHandlers.itemsHandler.postMessage({"identifier": identifier, "error": e});
  	}
  	}

• Now we're getting the Translate: translate called without specifying a translator. Running detection first.. and we get Translate: Parsing code for "TRANSLATOR" again and it seems to work fine now. The EIDR translator gets proper item instance here ({"itemType":"journalArticle","DOI":"10.1007/s11704-017-6132-7"})

So for some reason detectSearch is called after var translators = await translate.getTranslators(); translate.setTranslator(translators); which returns those TypeErrors. Then when actually calling translate those detectSearches seem to run fine, but we get the translate called without specifying a translator message first.

Instead of a TypeError and stack trace, it would be better to just gracefully handle an id of the expected type not being found.

I'm not sure I can improve this on my side. The translate.translate({ ... }) is in try/catch block and I log the error message and send it back to iOS to handle.