backend: access.status not consistent in drafts in direct access vs query access

[apirogov]

Describe the bug

1. Create a draft and enable files, but don't upload any. Save the draft.

Notice that access.status == "metadata-only" and this kind of could make sense for now.

Edit the draft, upload a file, save the draft.

2. Now access the draft via GET /api/records/{id}/draft.

Notice that access.status == "open"

3. Access the same draft from GET /api/user/records.

Notice that access.status is here "metadata-only".

Expected behavior

It should not matter whether I get a record from the search results or via ID directly, I expect the data to agree.

Furthermore, I actually would only expect to see metadata-only if files.enabled == false.

Notes

After doing the above - for the same draft, you can switch access to fully restricted, save, and switch back to fully public, save.

Still, the problem persists that the two access.status values will disagree. No idea what is going on here. First I thought it was the search index not being updated fast enough, but changing visibility worked instantly, and going back and forth did not fix the problem, so it must be something else.

[github-actions]

This issue was automatically marked as stale.

[max-moser]

I've done some digging, and it is due to the service's ResultList loading every search result hit via record = service.record_cls.loads(...), even for drafts[1].
This results in the record.files.entries being empty ({}) for drafts, even though the draft has files attached to it.
This causes the status calculation logic in the access system field to return metadata-only [2].
If the draft is loaded properly via record = service.draft_cls.loads(...), then the records.files.entries are set correctly and the access.record is also set correctly.

However, the results in the user records are mixed and consist of both records as well as drafts, so it's not possible to just use either service.record_cls or service.draft_cls to load the records from search hits without errors or glitches.
So far I have a few ideas on how to fix this problem, but need to investigate further.

[1] https://github.com/inveniosoftware/invenio-records-resources/blob/master/invenio_records_resources/services/records/results.py#L188
[2] https://github.com/inveniosoftware/invenio-rdm-records/blob/master/invenio_rdm_records/records/systemfields/access/field/record.py#L69

---

Rough call stack for the search:

• RecordsResource.search_user_records()
• RecordService.search_drafts()
• Service.result_list()
• RecordList.hits()