Make DOMParser and ZU.walkNodeDom available in background export #2094
Comments
|
It looks like you did not upload an support log. The support log is important; it gives If you did try to submit a support log, but the ID looked like This request is much more likely than not to apply to you too @yangfurong, even if you think it unlikely. Please trust
Once done, you will see a debug ID in red. Please post that debug id in the issue here. Thank you! |
I don't think that is the problem. It looks like the URL is available as metadata to the annotation, and that what Dan is saying is "if you want to use that metadata to add the URL to the exported annotation, you can". I've looked at If this is correct, I don't think it is within the scope of BBT to modify the annotation during export. It should be possible using a postscript, and we can look into that, but maybe the Zotfile author can be persuaded to patch the note-export to once again include the URI. It seems more within the scope of Zotfile. |
|
@retorquere: Not sure what you mean there. ZotFile no longer does annotation extraction and doesn't do anything with export. [Edit: I guess you mean monkey-patch? No. ZotFile is no longer maintained and shouldn't be taking on more functionality.] What are you saying is outside the note? |
|
I did mean ZotFile could do a monkey-patch, yes. I didn't know it was unmaintained and it seemed to align more closely with the scope of stuff ZotFile did than anything else. With "outside the node" I meant "outside the note body". I was attempting to simplify. In the
|
|
@retorquere Thanks a lot for your reply. In my use case, It is necessary for me to include the links in the note fields of BibTex items. The reason is that I need to import those notes with links in Obsidian that directly reads the note fields of exported BibTex items. It would be very helpful if you can provide some instructions to implement this function with a postscript. |
|
|
@retorquere Thank you so much. I have one more question. Could you suggest if it is possible to append another link, which looks like |
|
It is possible, but more work; you'd have to parse that out of the HTML. The body has which has the |
|
Thanks. I will figure out how to do it. |
will give you a DOM document that you can walk/inspect |
Hi. So, I was trying to find another way and found Zotero.Utilities. /**
* Walk the DOM and the contents of JSON data attributes in the HTML representation
* of a note, calling visitor functions that might modify it and returning
* the resulting HTML.
*
* Elements are visited in depth-first order. First the element itself is visited,
* then its data attributes, then the URIs in its JSON attributes, then its subtree.
*
* @param {String} note Note HTML
* @param {Object} visitors
* @param {Function} [visitors.visitContainer]
* @param {Function} [visitors.visitAnnotation]
* @param {Function} [visitors.visitCitation]
* @param {Function} [visitors.visitOtherElement]
* @param {Function} [visitors.visitDataAttribute]
* @param {Function} [visitors.visitURI] Return a replacement for the passed URI
* @return {String} Potentially modified note HTML
*/
walkNoteDOM: function (note, visitors) {
function visit(elem) {
if (elem.hasAttribute('data-schema-version')) {
visitors.visitContainer?.(elem);
}
else if (elem.hasAttribute('data-annotation')) {
visitors.visitAnnotation?.(elem);
}
else if (elem.hasAttribute('data-citation')) {
visitors.visitCitation?.(elem);
}
else {
visitors.visitOtherElement?.(elem);
}
if (visitors.visitDataAttribute || visitors.visitURI) {
for (let attr of ['data-citation', 'data-citation-items', 'data-annotation']) {
if (elem.hasAttribute(attr)) {
let json;
try {
json = JSON.parse(decodeURIComponent(elem.getAttribute(attr)));
}
catch (e) {
continue;
}
visitors.visitDataAttribute?.(attr, json);
if (visitors.visitURI) visitURIs(json);
elem.setAttribute(attr, JSON.stringify(json));
}
}
}
for (let child of elem.children) {
visit(child);
}
}
function visitURIs(json) {
if (Array.isArray(json)) {
json.forEach(visitURIs);
}
else if (typeof json === 'object') {
for (let [key, value] of Object.entries(json)) {
if (key == 'id' || key == 'attachmentURI') {
json[key] = visitors.visitURI(value);
}
else if (key == 'uris') {
json[key] = value.map(visitors.visitURI)
}
else {
visitURIs(value);
}
}
}
}
let wrappedNote = '<div id="note-body">' + note + '</div>';
let doc;
if (Zotero.isNode) {
let { JSDOM } = require('jsdom');
doc = new JSDOM(wrappedNote).window.document;
}
else {
let parser;
try {
parser = new DOMParser();
}
catch {
// Fx60 defines DOMParser but throws an error when you construct
// it from this context
parser = Cc['@mozilla.org/xmlextras/domparser;1'].createInstance(Ci.nsIDOMParser);
}
doc = parser.parseFromString(wrappedNote, 'text/html');
}
let root = doc.getElementById('note-body');
for (let child of root.children) {
visit(child);
}
return root.innerHTML;
}So, I tried to use this function with the script below. if (Translator.BetterTeX) {
if (item.notes) {
var my_visitor = {
visitContainer: function (obj) {
Zotero.debug("visitContainer");
},
visitAnnotation: function (obj) {
Zotero.debug("visitAnnotation");
},
visitCitation: function (obj) {
Zotero.debug("visitCitation");
},
visitOtherElement: function (obj) {
Zotero.debug("visitOtherElement");
},
visitDataAttribute: function (obj) {
Zotero.debug("visitDataAttribute");
},
visitURI: function (obj) {
Zotero.debug("visitURI");
return obj;
}
};
const notes = item.notes.map(note => {
const [ , kind, lib, key ] = note.uri.match(/^https?:\/\/zotero\.org\/(users|groups)\/((?:local\/)?[^/]+)\/items\/(.+)/);
const select = (kind === 'users') ? `zotero://select/library/items/${key}` : `zotero://select/groups/${lib}/items/${key}`;
Zotero.Utilities.walkNoteDOM(note.note, my_visitor);
return note.note + ` <a href="${select}">Go to note</a>`;
}).join('</p><p>');
reference.add({ name: 'note', value: notes, html: true });
}
}But, I got a new error. I was wondering if The output was like this. However, it seems that the two functions are the only supported functions. Because I tried several other functions in Could you please suggest what is the problem here? Thanks a lot. |
|
Any update on how to solve this issue? |
|
@yangfurong: |
|
@dstillman Ok, thanks for your comment. But other functions in |
|
Ok. I have changed the script, and now it works. postscriptif (Translator.BetterTeX) {
if (item.notes) {
var my_visitor = {
citation_uris: [],
visitContainer: function (obj) {
Zotero.debug("2094: visitContainer:" + obj + " " + typeof obj);
},
visitAnnotation: function (obj) {
Zotero.debug("2094: visitAnnotation:" + obj + " " + typeof obj);
json = JSON.parse(decodeURIComponent(obj.getAttribute('data-annotation')));
Zotero.debug(JSON.stringify(json));
const page_index = json["position"]["pageIndex"];
const annot_key = json["annotationKey"];
const [ , kind, lib, key ] = json["attachmentURI"].match(/^https?:\/\/zotero\.org\/(users|groups)\/((?:local\/)?[^/]+)\/items\/(.+)/);
pdf_uri = (kind === 'users') ? `zotero://open-pdf/library/items/${key}?page=${page_index}&annotation=${annot_key}` : `zotero://open-pdf/groups/${lib}/items/${key}?page=${page_index}&annotation=${annot_key}`;
Zotero.debug("2094: Add a PDF link:" + pdf_uri);
obj.insertAdjacentHTML("beforeend", ` |Open PDF, <a href="${pdf_uri}">Open PDF</a>|`);
Zotero.debug("2094: Link added:" + obj);
},
visitCitation: function (obj) {
Zotero.debug("2094: visitCitation:" + obj + " " + typeof obj);
},
visitOtherElement: function (obj) {
Zotero.debug("2094: visitOtherElement:" + obj + " " + typeof obj + " : " + obj.toString().charCodeAt(0) + " . " + obj.nodeType + " , " + obj.length);
Zotero.debug("2094: " + obj.innerHTML);
if (this.citation_uris.length != 0) {
Zotero.debug("2094: Add citation links:" + this.citation_uris);
if (obj.hasAttribute("class") && obj.getAttribute("class") === "citation-item") {
const uri_to_added = this.citation_uris.shift();
obj.insertAdjacentHTML("afterend", ` |Open item, <a href="${uri_to_added}">Open item</a>|`);
Zotero.debug("2094: Link added:" + obj);
}
}
},
visitDataAttribute: function (attr, json) {
Zotero.debug("2094: visitDataAttribute:" + attr + " " + JSON.stringify(json));
if (attr === "data-citation") {
// clear the array
this.citation_uris.length = 0;
for (let temp of json["citationItems"]) {
const temp_uris = temp["uris"].map(val => {
const [ , kind, lib, key ] = val.match(/^https?:\/\/zotero\.org\/(users|groups)\/((?:local\/)?[^/]+)\/items\/(.+)/);
const select = (kind === 'users') ? `zotero://select/library/items/${key}` : `zotero://select/groups/${lib}/items/${key}`;
return select;
});
this.citation_uris.push.apply(this.citation_uris, temp_uris);
}
}
},
visitURI: function (obj) {
Zotero.debug("2094: visitURI:" + obj + " " + typeof obj);
return obj;
}
};
const notes = item.notes.map(note => {
const [ , kind, lib, key ] = note.uri.match(/^https?:\/\/zotero\.org\/(users|groups)\/((?:local\/)?[^/]+)\/items\/(.+)/);
const select = (kind === 'users') ? `zotero://select/library/items/${key}` : `zotero://select/groups/${lib}/items/${key}`;
const modified_content = Zotero.Utilities.walkNoteDOM(note.note, my_visitor);
Zotero.debug("2094: " + typeof modified_content + " , " + modified_content);
return modified_content + `(Go to the note, <a href="${select}">Go to note</a>)<br/>..................................`;
}).join('</p><p>');
reference.add({ name: 'note', value: notes, html: true });
}
}@article{huang_IndepthStudyLTE_2013,
title = {An In-Depth Study of {{LTE}}: Effect of Network Protocol and Application Behavior on Performance},
shorttitle = {An In-Depth Study of {{LTE}}},
author = {Huang, Junxian and Qian, Feng and Guo, Yihua and Zhou, Yuanyuan and Xu, Qiang and Mao, Z. Morley and Sen, Subhabrata and Spatscheck, Oliver},
year = {2013},
month = sep,
journal = {ACM SIGCOMM Computer Communication Review},
volume = {43},
number = {4},
pages = {363--374},
note = {\section{Annotations\\
(5/10/2022, 6:59:51 PM)}
\par
``However, the interactions among applications, network transport protocol, and the radio layer still remain unexplored.'' |Open PDF, \href{zotero://open-pdf/library/items/8SU8CCQA?page=0&annotation=EZFZEMQV}{Open PDF}| (Huang et al., 2013, p. 363 |Open item, \href{zotero://select/library/items/L2KWDK96}{Open item}|)
\par
``We observed that LTE has significantly shorter state promotion delays and lower RTTs than those of 3G networks.'' |Open PDF, \href{zotero://open-pdf/library/items/8SU8CCQA?page=0&annotation=WD3P2JKD}{Open PDF}| (Huang et al., 2013, p. 363 |Open item, \href{zotero://select/library/items/L2KWDK96}{Open item}|)
\par
(Huang et al., 2013, p. 363 |Open item, \href{zotero://select/library/items/L2KWDK96}{Open item}|) test
\par
(Go to the note, \href{zotero://select/library/items/XMZP86RD}{Go to note})\\
..................................
\par
\section{Manually Generated Notes}
\par
This is a reference to (Atxutegi et al., 2018 |Open item, \href{zotero://select/library/items/8FC6JVQ6}{Open item}|)
\par
(Atxutegi et al., 2018 |Open item, \href{zotero://select/library/items/8FC6JVQ6}{Open item}|; Croitoru et al., 2015 |Open item, \href{zotero://select/library/items/IR9GYRYW}{Open item}|; Dong et al., 2018 |Open item, \href{zotero://select/library/items/BGMMKVIF}{Open item}|)
\par
\href{https://bing.com}{aaa}
\par
(Go to the note, \href{zotero://select/library/items/Y4J279RR}{Go to note})\\
..................................}
}
But, if I turn on the |
github-actions
bot
added
awaiting-user-feedback
and removed
awaiting-user-feedback
labels
Ok. I did not know this before. |
|
I'm still testing, so no new log is needed yet, but could you replace your postscript with this one? It does the same, but the logging is more easy to find, and it outputs regular keys unless I'm testing. That way we can share this postscript while we're figuring this out. |
|
🤖 this is your friendly neighborhood build bot announcing test build 6.7.1.2629 ("insertAdjacentHTML") Install in Zotero by downloading test build 6.7.1.2629, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...". |
It seems that you are using the old script. I would like to note that it does not work as
Please use this latest script in which I have added your testing code.if (Translator.BetterTeX) {
if (item.notes) {
var my_visitor = {
citation_uris: [],
visitContainer: function (obj) {
Zotero.debug("2094: visitContainer:" + obj + " " + typeof obj);
},
visitAnnotation: function (obj) {
Zotero.debug("2094: visitAnnotation:" + obj + " " + typeof obj);
json = JSON.parse(decodeURIComponent(obj.getAttribute('data-annotation')));
Zotero.debug("2094: " + JSON.stringify(json));
const page_index = json["position"]["pageIndex"];
const annot_key = Translator.preferences.testing ? 'fixed-for-test' : json.annotationKey;
const [ , kind, lib, key ] = json["attachmentURI"].match(/^https?:\/\/zotero\.org\/(users|groups)\/((?:local\/)?[^/]+)\/items\/(.+)/);
if (Translator.preferences.testing) key = 'fixed-for-test';
pdf_uri = (kind === 'users') ? `zotero://open-pdf/library/items/${key}?page=${page_index}&annotation=${annot_key}` : `zotero://open-pdf/groups/${lib}/items/${key}?page=${page_index}&annotation=${annot_key}`;
Zotero.debug("2094: Add a PDF link:" + pdf_uri);
obj.insertAdjacentHTML("beforeend", ` |Open PDF, <a href="${pdf_uri}">Open PDF</a>|`);
Zotero.debug("2094: Link added:" + obj);
},
visitCitation: function (obj) {
Zotero.debug("2094: visitCitation:" + obj + " " + typeof obj);
},
visitOtherElement: function (obj) {
Zotero.debug("2094: visitOtherElement:" + obj + " " + typeof obj + " : " + obj.toString().charCodeAt(0) + " . " + obj.nodeType + " , " + obj.length);
Zotero.debug("2094: " + obj.innerHTML);
if (this.citation_uris.length != 0) {
Zotero.debug("2094: Add citation links:" + this.citation_uris);
if (obj.hasAttribute("class") && obj.getAttribute("class") === "citation-item") {
const uri_to_added = this.citation_uris.shift();
obj.insertAdjacentHTML("afterend", ` |Open item, <a href="${uri_to_added}">Open item</a>|`);
Zotero.debug("2094: Link added:" + obj);
}
}
},
visitDataAttribute: function (attr, json) {
Zotero.debug("2094: visitDataAttribute:" + attr + " " + JSON.stringify(json));
if (attr === "data-citation") {
// clear the array
this.citation_uris.length = 0;
for (let temp of json["citationItems"]) {
const temp_uris = temp["uris"].map(val => {
const [ , kind, lib, key ] = val.match(/^https?:\/\/zotero\.org\/(users|groups)\/((?:local\/)?[^/]+)\/items\/(.+)/);
if (Translator.preferences.testing) key = 'fixed-for-test';
const select = (kind === 'users') ? `zotero://select/library/items/${key}` : `zotero://select/groups/${lib}/items/${key}`;
return select;
});
this.citation_uris.push.apply(this.citation_uris, temp_uris);
}
}
},
visitURI: function (obj) {
Zotero.debug("2094: visitURI:" + obj + " " + typeof obj);
return obj;
}
};
const notes = item.notes.map(note => {
const [ , kind, lib, key ] = note.uri.match(/^https?:\/\/zotero\.org\/(users|groups)\/((?:local\/)?[^/]+)\/items\/(.+)/);
if (Translator.preferences.testing) key = 'fixed-for-test';
const select = (kind === 'users') ? `zotero://select/library/items/${key}` : `zotero://select/groups/${lib}/items/${key}`;
const modified_content = Zotero.Utilities.walkNoteDOM(note.note, my_visitor);
Zotero.debug("2094: " + typeof modified_content + " , " + modified_content);
return modified_content + `(Go to the note, <a href="${select}">Go to note</a>)<br/>..................................`;
}).join('</p><p>');
reference.add({ name: 'note', value: notes, html: true });
}
}I have tested this latest script with test build 6.7.1.2629 ("insertAdjacentHTML"). The new error is The error log.A new support log is also reported (ID: |
|
I had a bug in my tools where it wouldn't save the postscript from the log ID, sorry. I have fixed that, but the postscript in You will have to wait for the new build to drop here for further tests. I have fixed the edit: I have it reproducible now, so I should be able to fix it on my end. No further logs are needed. |
|
🤖 this is your friendly neighborhood build bot announcing test build 6.7.1.2635 ("Merge branch 'master' into gh-2094") Install in Zotero by downloading test build 6.7.1.2635, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...". |
|
With the testcase from |
|
I got the error below with the worker enabled.
Everything works fine with the worker disabled. A new support log is reported (ID: |
|
reproducible. It looks like I'll need to write a significant extension to xmldom. |
|
🤖 this is your friendly neighborhood build bot announcing test build 6.7.1.2682 ("zotero-only test") Install in Zotero by downloading test build 6.7.1.2682, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...". |
|
Great! Now it works with the worker enabled! Thanks a lot. |
|
That page has an "edit this page" at the top right, have at it 😄 . It's a hefty bit of code, so it'd require some explanation of what it does, and it's large enough that I'd stick it in a block to keep the page readable. |
Log ID: DMMHCUW6-refs-euc
Hi.
In Zotero 5.0, I used Zotfile to generate notes from annotations in PDF files. In those notes, Zotfile used to explicitly include Zotero-URI links pointing to the positions where the corresponding annotations were taken. Thus, when I exported some Zotero items to a Bibtex file, the Bibtex items included a "note" field where the Zotero-URI links were also included. An example is shown below.
However, in Zotero 6.0, Zotfile's "Extract Annotations" is replaced by the Zotero built-in function "Add Note From Annotations". Unlike the notes generated by Zotfile, the new-style notes do not explicitly include Zotero-URI links. Instead, they use embedded links. Therefore, if we export a Zotero item with a new-style note to a Bibtex file, we have no more Zotero-URI links in the note field of the Bibtex item. An example is given below.
But, as discussed in the Zotero forum (https://forums.zotero.org/discussion/95285/pdf-annotations-do-not-include-markdown-links#latest), Zotero 6.0 is able to export a new-style note to a markdown file that includes Zotero-URI links.
As @dstillman mentioned, in Zotero 6.0,
So, I guess that BBT currently does not decode the notes in the same way as Zotero 6.0's "Markdown export".
To verify this, I wrote a postscript in BBT to print the decoded notes in Zotero's debug console. The script and output are shown below.
The output may be a bit messy. But, if we translate it to HTML format, it looks like this.
So, I would like to ask if it is possible to change BBT's way of decoding notes in order to include Zotero-URI links in the note field of Bibtex items? Thanks a lot.
The text was updated successfully, but these errors were encountered: