Adding an attachment via API doesn't work

[frank-f]

Describe the bug
I'm trying to add an attachment to a component via API and no matter how I'm sending my data, I always get an error 500 back

To Reproduce
This is a very simple example written in Python

#!/usr/bin/env python
import requests
import yaml

partdb_api = 'https://my.part.db/api'
partdb_bearer = 'tcp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' # Edit

headers = {'Accept': 'application/json', 'Content-type': 'application/json', 'Authorization': 'Bearer '+partdb_bearer}
data = """{
  "name": "front",
  "attachment_type": 1,
  "element": {
    "id": 212
  },
  "url": "https://assets.lcsc.com/images/lcsc/224x224/20221231_Littelfuse-0437-750WR_C99536_front.jpg"
}"""

response = requests.post(partdb_api + '/attachments', data=data, headers=headers)
print(yaml.dump(response.json()))

Expected behavior
A new attachment for part 212

Screenshots
Result looks like this:

$ ./attachment.py
detail: Internal Server Error
status: 500
title: An error occurred
type: /errors/500

Server Side

• Part-DB Version: 1.10.5-dev (master/ee3ad40)
• PHP Version: 8.1.27
• Database Server: MySQL 8.0.35

Script host

• Python 3.11.7

Additional context
Since the example in the API documentation is throwing the same error, I'm opening this as a bug, but if I'm just missing some key element in my JSON, some heads up would be nice.

var/log/prod.log reports a bunch of lines, but I think this one looks most helpful

2024-01-26T19:52:00.129717+01:00","extra":{"token":{"authenticated":true,"roles":["ROLE_USER","ROLE_API_AUTHENTICATED","ROLE_API_READ_ONLY","ROLE_API_EDIT"],"user_identifier":"myuser"},"url":"/api/attachments","ip":"37.201.c.d","http_method":"POST","server":"my.part.db","referrer":null}}
{"message":"Uncaught PHP Exception Error: "Cannot instantiate abstract class App\Entity\Attachments\Attachment" at AbstractItemNormalizer.php line 347","context":{"exception":{"class":"Error","message":"Cannot instantiate abstract class App\Entity\Attachments\Attachment","code":0,"file":"/usr/local/www/partdb/vendor/api-platform/core/src/Serializer/AbstractItemNormalizer.php:347"}},"level":500,"level_name":"CRITICAL","channel":"request","datetime":"2024-01-26T19:52:00.140329+01:00","extra":{"token":{"authenticated":true,"roles":["ROLE_USER","ROLE_API_AUTHENTICATED","ROLE_API_READ_ONLY","ROLE_API_EDIT"],"user_identifier":"myuser"},"url":"/api/attachments","ip":"37.201.c.d","http_method":"POST","server":"my.part.db","referrer":null}}
{"message":"Stored the security token in the session.","context":{"key":"_security_main"},"level":100,"level_name":"DEBUG","channel":"security","datetime":"2024-01-26T19:52:00.145967+01:00","extra":{"token":{"authenticated":true,"roles":["ROLE_USER","ROLE_API_AUTHENTICATED","ROLE_API_READ_ONLY","ROLE_API_EDIT"],"user_identifier":"myuser"},"url":"/api/attachments","ip":"37.201.c.d","http_method":"POST","server":"my.part.db","referrer":null}}

[frank-f]

Digging through the API further I thought of another way to add an attachment: By posting a PATCH request to a part directly, but that also does not work. It always returns You must select an attachment type! (which I of course did).

Here's an example, this time more simple with cURL:

curl -X 'PATCH' \
  'https://my.part.db/api/parts/212' \
  -H 'accept: application/ld+json' \
  -H 'Content-Type: application/merge-patch+json' \
  -H "Authorization: Bearer tcp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
  -d '{"attachments": [{"name": "front1", "attachment_type": 1, "element": {"id": "212"}, "url": "https://assets.lcsc.com/images/lcsc/224x224/20221231_Littelfuse-0437-750WR_C99536_front.jpg"}]}' | python -m json.tool

Result

{
    "@id": "/api/validation_errors/ad32d13f-c3d4-423b-909a-857b961eb720",
    "@type": "ConstraintViolationList",
    "status": 422,
    "violations": [
        {
            "propertyPath": "attachments[0].attachment_type",
            "message": "You must select an attachment type!",
            "code": "ad32d13f-c3d4-423b-909a-857b961eb720"
        }
    ],
    "detail": "attachments[0].attachment_type: You must select an attachment type!",
    "hydra:title": "An error occurred",
    "hydra:description": "attachments[0].attachment_type: You must select an attachment type!",
    "type": "/validation_errors/ad32d13f-c3d4-423b-909a-857b961eb720",
    "title": "An error occurred"
}

[jbtronics]

The main problem, is that attachments (similar to parameters) can belong to different elements, and there are different classes to model that. However when trying to create a new attachment via POST, it can not be decided, which class to instantiate and this error occurs. I think this is not quite an easy fix and it might require some changes to the current API interface. I will look into that in the future.

For now as an workaround, at least the possibility to create new attachments (and parameters) via the PATCH operation should work now with something like:

{	
"attachments": [
  {"name": "front68", "attachment_type": "/api/attachment_types/1", "url": "https://invalid.invalid/test.url"}
],
"parameters": [
  {"name": "value", "unit": "Ohm", "value": 100}
]
}

See here for more infos: https://docs.part-db.de/api/intro.html#creating-attachments-and-parameters

[frank-f]

Thank you for fixing this so quickly.

It works, but when I add attachments like this it creates external links. I'd like to save all files locally, so I tried to add downloadURL: 1 to the JSON, because I saw that parameter in the Web UI, but it's not working. external: false and picture: true also don't seem to have any effect. Or do I need to download the file myself and send the content to the API?

My .env.local says

ALLOW_ATTACHMENT_DOWNLOADS=1
ATTACHMENT_DOWNLOAD_BY_DEFAULT=1

[jbtronics]

Currently its not possible to upload files via API and its also not possible to let Part-DB download files.
In the moment you can only do that via the WebUI.

[frank-f]

Okay. 😕
I noticed, that the attachments are not merged. If I submit a "Front" attachment in one request and "Back" in a second one, in the end the part has just one attachment and that is the last submitted one. Is that intended?

[jbtronics]

It's not "intended" by me, but that's what the json merge specification says, and how the API framework works. I tried out some stuff, and there seems to be no possibility to just refer existing elements.

Therefore I would only recommend to use PATCH to change scalar values on elements, not such nested stuff. However that's currently the only way to properly create new attachments.

If you really need to add additional attachments to an element with existing ones, then it should work with the following workaround:

Retrieve the current attachments of an element, store them somewhere. Then send an patch response with an empty array, to remove all existing attachments on the element (to prevent validation errors). Then send a patch array with containing the old attachments, and your new ones.

This will recreate the attachments. You will loose Infos like creation date, last modified and history. And it will only work if they contain an URL. You will not be able to recreate attachments containing a local file via API yet.

Depending on what you currently try to achieve, maybe directly accessing and modifying the database of Part-DB directly might be the better choice for now

[frank-f]

The merge behavior is not a major issue.

I wrote a script that converted my LCSC orders into Part-DB compatible CSV files. That worked fine, but they also have actual photos of each part and datasheets for most parts on their website, so I'd like to scrape those off their servers and store them in Part-DB. That said, my parts don't have any attachments that I'd lose and could not recover, but having them locally is important to me.

Since it's - more or less - a one-off task, I'll probably create the database entries via API and then take your advice, download the files myself and modify the rows in the database.

[jbtronics]

With the latest commits, the POST endpoint for attachments and parameters work fine.

The attachment type is normally autodetected on the IRI of the element property passed.

So no need for this weird workaround anymore.