ZoteroObjectUploadError: 'preprint' is not a valid itemType"

[aitianshi]

Hi @uniuuu
While testing the new script for user/group management, I tried to use the sync into this group. However, whatever I try to sync I have this error 'preprint' is not a valid itemType". This ItemType is in fact not in the database but I don't know why ALL documents shows this.
Do you have nay ideas ? There are the full errors.

[JavaScript Error: "Error 400 for item U8R5IEV7 in TestGroup:

ZoteroObjectUploadError: 'preprint' is not a valid itemType"]

[JavaScript Error: "'preprint' is not a valid itemType" {file: "chrome://zotero/content/xpcom/sync/syncEngine.js" line: 1357}]

[JavaScript Error: "Error 409 for item BREK9FMT in TestGroup:

ZoteroObjectUploadError: Parent item U8R5IEV7 not found"]

[JavaScript Error: "Parent item U8R5IEV7 not found" {file: "chrome://zotero/content/xpcom/sync/syncEngine.js" line: 1357}]

[JavaScript Error: "Error 409 for item BDBAKFJK in TestGroup:

ZoteroObjectUploadError: Parent item U8R5IEV7 not found"]

[JavaScript Error: "Parent item U8R5IEV7 not found" {file: "chrome://zotero/content/xpcom/sync/syncEngine.js" line: 1357}]

[JavaScript Error: "Made no progress during upload -- stopping" {file: "chrome://zotero/content/xpcom/sync/syncEngine.js" line: 1416}]

[JavaScript Error: "Made no progress during upload -- stopping" {file: "chrome://zotero/content/xpcom/sync/syncEngine.js" line: 1416}]

[JavaScript Error: "HTTP GET http://localhost:8080/retractions/list failed with status code 404:

<h1>Not Found</h1>
<p>The page you requested could not be found.</p>
"]

[JavaScript Error: "HTTP GET http://localhost:8080/retractions/list failed with status code 404:

<h1>Not Found</h1>
<p>The page you requested could not be found.</p>
"]

[JavaScript Error: "HTTP GET http://localhost:8080/retractions/list failed with status code 404:

<h1>Not Found</h1>
<p>The page you requested could not be found.</p>
"]

version => 6.0.26.SOURCE.f68dc7c90, platform => Win32, oscpu => Windows NT 10.0; WOW64, locale => fr-FR, appName => Zotero, appVersion => 6.0.26.SOURCE.f68dc7c90, extensions => Zotero LibreOffice Integration (6.0.4.SA.6.0.26.SOURCE.f68dc7c90, extension), Zotero Word for Windows Integration (6.0.3.SA.6.0.26.SOURCE.f68dc7c90, extension)

[uniuuu]

Hi @aitianshi
Thank you for your testing and finding.
Could you please share step by step details how I can reproduce the error ?

[aitianshi]

Sorry I forgot it. There are the steps to reproduce:

• Create a user
• Create group
• Download random paper from Arxiv: https://arxiv.org/abs/1902.00005
• Upload the PDF to the group
• Synchronize --> Never fully synchronize even if some items are added in the DB, throw the above error

EDIT: an object created from Zotero doesn't have any problem, only upload existing PDF has this issue so far. It seems most of PDF objects are inserted in the DB but cannot verify if everything's there. Sync is very long

[aitianshi]

This error also might cause the previous error. Is there a standalone way to test ws://localhost:9000/ ?

[JavaScript Error: "WebSocket connection closed: 1006 "]

[JavaScript Error: "HTTP GET http://localhost:8080/retractions/list failed with status code 404:

<h1>Not Found</h1>
<p>The page you requested could not be found.</p>
"]

[JavaScript Error: "HTTP GET http://localhost:8080/retractions/list failed with status code 404:

<h1>Not Found</h1>
<p>The page you requested could not be found.</p>
"]

[JavaScript Error: "HTTP GET http://localhost:8080/retractions/list failed with status code 404:

<h1>Not Found</h1>
<p>The page you requested could not be found.</p>
"]

[JavaScript Error: "The connection was refused when attempting to contact ws://localhost:9000/."]

[JavaScript Error: "WebSocket connection closed: 1006 "]

[JavaScript Error: "The connection was refused when attempting to contact ws://localhost:9000/."]

[JavaScript Error: "WebSocket connection closed: 1006 "]

[JavaScript Error: "S3 returned 0 (2/KTECA6ZU) -- retrying upload"]

[JavaScript Error: "The connection was refused when attempting to contact ws://localhost:9000/."]

[JavaScript Error: "WebSocket connection closed: 1006 "]

[JavaScript Error: "The connection was refused when attempting to contact ws://localhost:9000/."]

[JavaScript Error: "WebSocket connection closed: 1006 "]

[JavaScript Error: "S3 returned 0 (2/KTECA6ZU) -- retrying upload"]

[JavaScript Error: "S3 returned 0 (2/KTECA6ZU) -- retrying upload"]

[JavaScript Error: "The connection was refused when attempting to contact ws://localhost:9000/."]

[JavaScript Error: "WebSocket connection closed: 1006 "]

[JavaScript Error: "S3 returned 0 (2/KTECA6ZU) -- retrying upload"]

[JavaScript Error: "S3 returned 0 (2/KTECA6ZU) -- retrying upload"]

[JavaScript Error: "The connection was refused when attempting to contact ws://localhost:9000/."]

[JavaScript Error: "WebSocket connection closed: 1006 "]

[JavaScript Error: "Upload request 2/KTECA6ZU failed"]

[JavaScript Error: "Une erreur de synchronisation de fichier s'est produite. Veuillez redémarrer Zotero et/ou votre ordinateur et essayez de synchroniser à nouveau.
 
 Si l'erreur persiste, il peut y avoir un problème soit avec votre ordinateur soit avec le réseau : logiciels de sécurité – antivirus, pare-feu – VPN, etc. Essayez de désactiver les logiciels de sécurité que vous utilisez ou, si c'est un portable, essayez depuis un autre réseau." {file: "chrome://zotero/content/xpcom/storage/zfs.js" line: 702}]

[JavaScript Error: "Error uploading full text for item KTECA6ZU in TestDBEtienne:

ZoteroObjectUploadError: An error occurred"]

[JavaScript Error: "An error occurred" {file: "chrome://zotero/content/xpcom/sync/syncFullTextEngine.js" line: 180}]

version => 6.0.26.SOURCE.f68dc7c90, platform => Win32, oscpu => Windows NT 10.0; WOW64, locale => fr-FR, appName => Zotero, appVersion => 6.0.26.SOURCE.f68dc7c90, extensions => Zotero LibreOffice Integration (6.0.4.SA.6.0.26.SOURCE.f68dc7c90, extension), Zotero Word for Windows Integration (6.0.3.SA.6.0.26.SOURCE.f68dc7c90, extension)

[uniuuu]

Hi @aitianshi

The issue:

[JavaScript Error: "S3 returned 0 (2/KTECA6ZU) -- retrying upload"]

Has been fixed in v2.8.3

[aitianshi]

I noticed that .env file contains:

# Ip/host of the Minio.
S3HOST=10.5.5.1:9000

but docker-compose.yml shows that

 zotprime-minio:
    [...]
    networks:
      zotprime:
        ipv4_address: 10.5.5.7

[aitianshi]

@uniuuu , I checked Zotero repo, it might be linked to this opened PR zotero/zotero # 2401 to support Arxiv papers.
My second issue above is linked to Minio. I can make few PRs for Minio and DB management. Gonna check also what is missing to support Arxiv.

[uniuuu]

@aitianshi The line in error HTTP GET http://localhost:8080/ indicates that it tries to access localhost.
But in previous your post I saw you are using VM (or client- server setup where client and server are on different servers).
So to clarify the localhost setup is meant for server and client are running on same single computer. This idea inherited from original ZotPrime repository.
The VM setup is meant for such where server and client are sitting on different computers.
May I check if your setup done as VM Install described in README.md ?
If not then could you please try to rebuild both server and client while using respective configurations for both with VM IP Address and not localhost and then test again.
If this is the case of the issue could you please let me know what improvement of README.md you see that will make more clear the steps ?

[aitianshi]

I test my setup on my public network and private computer before deploying. So everything is local.

As mentioned few minutes before your asnwer I managed to upload/sync supported PDF by changing Minio address.

Also, the preprint error is due to some librairies lack of support (PR which is still opened zotero/zotero#2401). So I don't know who is really using synchronisation with such objects. I'm gonna try the PR commits.

[aitianshi]

preprint is in itemTypes from file ./client/zotero-client/resource/schema/global/schema.json but is missing the DB.

EDIT: I saw the file where all entries of the DB are done and it is in fact missing preprint with its relationships. I am still not familiar with your repository (or Zotero in general).
If entries are added in system-107.sql with the schema relationships, will the backend code support it ?

[uniuuu]

Hi @aitianshi

I test my setup on my public network and private computer before deploying. So everything is local.

If I understood correctly by "local" you mean localhost. Please confirm your client and server are on the same single machine/server and no VM run there.

[aitianshi]

Hi @aitianshi

I test my setup on my public network and private computer before deploying. So everything is local.

If I understood correctly by "local" you mean localhost. Please confirm your client and server are on the same single machine/server and no VM run there.

Hi @uniuuu
Yes they are on the same machine

[uniuuu]

I'll check on this issue today.

[uniuuu]

Hi @aitianshi
There was a typo which has been fixed in the latest 2.6.2 which is not related though to the issue.
But for the main issue 'preprint' is not a valid itemType I confirm it was reproduced on my side too for this type of pdf file.
Have you encountered this error with other type of files?

[uniuuu]

@uniuuu , I checked Zotero repo, it might be linked to this opened PR zotero/zotero#2401 to support Arxiv papers. My second issue above is linked to Minio. I can make few PRs for Minio and DB management. Gonna check also what is missing to support Arxiv.

Hi @aitianshi
The issue quoted is not related as the preprint is missing in zotero client and dataserver.
I traced it and found in file chrome/content/zotero/xpcom/translate/modules/utilities/resource/zoteroTypeSchemaData.js

It shows proprint's id is 38 but in https://github.com/uniuuu/dataserver/blob/master/misc/coredata.sql it's not there and last id is 37 so seems it wasn't added yet.
Moreover in client's https://github.com/uniuuu/zotero/blob/main/resource/schema/system-107.sql there are only total 36 id.
I have tried a quick fix to add respective entries to both files.

But it didn't not fixed issue.
I have created a new issue in the zotero/zotero repository let's see is it will be fixed any time soon.

This issue #14 will have a status blocked/wont fix.
As this is the upstream issue I'll convert this one to discussion to monitor the upstream progress.

[aitianshi]

Sorry for late reply. In fact I also tried to add new preprint entry in the system-107 but it requires more than this line. As preprint, according to the schema contains some fields, they should also be added in the file.
A bit busy recently but I'm gonna try that !

[uniuuu]

Hi @aitianshi
Good morning.
I saw the request in upstream was bounced for both forum and github issues.
I wanted know if you wanted to do report based on https://www.zotero.org/support/reporting_problems . Do you ?
Note, we're using branch Zotero 6.0 https://github.com/zotero/zotero/tree/6.0 . The exact version after build is 6.0.28. There is also tag 6.0.27 available.
Because main branch points to Zotero 7 which is still have no official release.
In download page https://www.zotero.org/download/
The current version for Windows and Linux is 6.0.27
The current version for Mac is 6.0.29

What I am going to do is to patch downloaded prebuilt client as described https://github.com/gfacciol/zotero_dataserver-docker#patch-the-standalone-client-to-use-the-new-dataserver and test with it.

[aitianshi]

Good afternoon @uniuuu ,

As I didn't see any answer from Zotero maintainers on the issue while they were active on others. I checked the report issue information which states they only check the forum first. I opened a discussion on the forum to make this issue seen by the maintainers.

I still don't understand how the schema is complete but the DB (system-107.sql) isn't.

Hope you don't mind I posted the issue in their forum. Waiting for your test.

[uniuuu]

Hi @aitianshi

What I saw, is that in the forum the issue you posted has been bounced too.

Hi @aitianshi Good morning. I saw the request in upstream was bounced for both forum and github issues. I wanted know if you wanted to do report based on https://www.zotero.org/support/reporting_problems . Do you ? Note, we're using branch Zotero 6.0 https://github.com/zotero/zotero/tree/6.0 . The exact version after build is 6.0.28. There is also tag 6.0.27 available. Because main branch points to Zotero 7 which is still have no official release. In download page https://www.zotero.org/download/ The current version for Windows and Linux is 6.0.27 The current version for Mac is 6.0.29

What I am going to do is to patch downloaded prebuilt client as described https://github.com/gfacciol/zotero_dataserver-docker#patch-the-standalone-client-to-use-the-new-dataserver and test with it.

Good afternoon @uniuuu ,

As I didn't see any answer from Zotero maintainers on the issue while they were active on others. I checked the report issue information which states they only check the forum first. I opened a discussion on the forum to make this issue seen by the maintainers.

I still don't understand how the schema is complete but the DB (system-107.sql) isn't.

Hope you don't mind I posted the issue in their forum. Waiting for your test.

[uniuuu]

@aitianshi By the way you're pointing to minio IP change fixed issue you've had. Could you write details?
If it's behaving on your side differently I need to try to reproduce what you've faced to see why changing minio has IP reflected on your side. I assume you're referring to 10.5.5.1 IP to change as you've posted before.
On which Linux do you run server ? What's docker version?

[uniuuu]

@aitianshi By the way you're pointing to minio IP change fixed issue you've had. Could you write details? If it's behaving on your side differently I need to try to reproduce what you've faced to see why changing minio has IP reflected on your side. I assume you're referring to 10.5.5.1 IP to change as you've posted before. On which Linux do you run server ? What's docker version?

Hi @aitianshi
Good day.
I have given details here regarding how to run the zotprime without issue you've got. #17 (comment)
Please check if it also will clarify for your case.

Regarding ZoteroObjectUploadError: 'preprint' is not a valid itemType" issue. Have you got any update for it?
From my side it's still pending to try to patch downloaded prebuilt client.

[aitianshi]

Hi @uniuuu , sorry for my (very) long absence. I have a way to fix that. It just needs to parse the real schema and introduce the new elements in the PHPMYADMIN.
I made a script which generate all database items according to the database schema in zoteroTypeSchemaData.js.
How do you want to proceed ? I make a PR so you can test it ?

[uniuuu]

Hi @aitianshi
Same here, I apologize for delay. Please make the PR.
I hope to find the reason why we have this issue however we're using the active Zotero Client and Dataserver repositories.

[MarkOWiesemann]

@aitianshi @uniuuu

I probably found a solution: The docker container of the dataserver does not initialize the database. If you run https://github.com/uniuuu/dataserver/blob/master/admin/schema_update located under /var/www/zotero/admin/schema_update preprint as well as many other elements gets added to the database.

I am not able to test it yet though because I have other issues with the sync right now.
I might look into updating the dockerfile when I am free the next time.

[uniuuu]

@MarkOWiesemann It works!
Created issue for it #34