Heads-Up for API use: add new model

[binderth]

I'm primarily using Printables.com. I like the UI and I like, that in the Download, there's everything in there - including the PDF description. I'm also aware of the GRAPH QL and the printabels-cli-api.py project (see #4530).
So I'm playing around a bit, because I'm tired of downloading the ZIP from Printables, and copy/pasting everything into Manyfold! ;)
So I already wrote a wrapper for the Graph QL part and I'm now stuck at how to add a model via API.

Is there a working example of the payload to add a model? the one from swagger throws an 400 (bad request) error. Is there anything, where I can copy the JSON-Payload from? I'm trying to find it in the UI, but I don't get how the upload works there... ;)

thanks!

[Floppy]

Hm, I'll have a look at the swagger example, it should work - that's the whole point, after all!

Does anything useful come back in the 400 response body?

[Floppy]

You should be able to put it all in the api payload; Manyfold will then generate the datapackage file for you. You could upload a datapackage, but if you don't already have one, there's probably no point doing it that way.

[binderth]

So, Update! ;)
it works - kinda....

What I do is:

first:

1. download the model data - ZIP from printables (there's only the 3MF/STL files plus the PDF in there)
2. request meta-information via GraphQL, which includes descriptions, tags, etc. plus all images (JPG)
3. download all images and add them to the ZIP
4. add the datapackage.json to the ZIP

With that, I then use the API to add all of that to a model:

1. get token
2. prepare upload (POST /upload)
3. upload the ZIP file and check (PATCH /upload every 2 secs until ready)
4. finalize the upload (HEAD /upload every 2 secs) [1]
5. create the model (POST /models) including a simple JSON-Payload with name, file[], sensitive, tag_list

So, in 80 % of that I get everything I want: the model is created with all the data form the ZIP-file. Pretty much the same outcome as the GUI has, if I uploaded just the ZIP-Archive for "upload models" and only place the ZIP there.
But in 20 % of the cases, only the information form the /models - JSON-Payload plus all files make it into manyfold. Sometimes if I repeat the whole process it works, sometimes I can repeat multiple times with the same outcome.
and to top it up: sometimes uploading the ZIP archive via the GUI also leads to that same outcome. To make it even more interesting, the preview thumbnail is not the same everytime... ;)

For the time being I'm more than ok with that. But perhaps there is something catching your eye.? I'm thinking of unfinished processess in the background? a zombie UpdateDatapackageJob... something like that? If something like that could be identified and thrown out?

[binderth]

Update #2:
I was fast enough and just discovered the Sidekiq dashboard. I think this error is the culprit:
Job: Scan::Model::ParseMetadataJob
error class: ActiveRecord::RecordInvalid
error message: Validation failed: Handle has already been taken

edit1: there is another error with the same job and error class
error message: Validation failed: Creator Name has already been taken, Handle has already been taken

after I manipulated the contributor to a new title, it worked for uploading manually.
after I deleted the contributor it also worked via API.

is there a way to tell manyfold to use an existing author if that happens?

edit2: I'm not sure what happened, but as I was testing, an existing author was just used. A fast guess would be, that creators with special characters in their title or handle are not recognized correctly:

working authors: "Anefecious" with handle "printables.com/@Anefecious"
not working authors: "Juggernaut_Print" with handle "printables.com/@Juggernaut_Pr_533637"

[binderth]

Update #3:
sorry for all that spamming... ;)

but I now confirmed it and can reproduce it as I said. the "ParseMetadataJob" does well with "normal" names and handles of authors, but crashes with an error, if the author already exists, but with special characters.

So I'll close this discussion, my initial point is met: I can add model via API! Yeah!!

[Floppy]

Congratulations, that's awesome, thanks for persisting with it! I don't know of many users of the API yet, so this is super useful for me!

Sounds like there's an error in the creator matching when special characters are involved. Could I ask you to please open a bug report issue with a summary of what happened and an example if you can?

Great to hear about the Printables GraphQL API; I've not looked at that, because they don't particularly advertise it, a bit like Thangs and their API.

[binderth]

Yes, I'll open a bug report over the weekend.

And I could document the GraphQL queries somewhere, the printables-cli-api wrapper was a good start, but lacks quite a bit of functionality needed for collecting the images and some meta information. So I used it as inspiration, but replaced it completely with simple Query-functions.

[binderth]

adding models via API works!