Datastore Access Restrictions Removal on Cron Run

[marlubo]

Could you assist me in understanding the system architecture behind why a backend datastore periodically removes access restrictions, allowing all csv rows to be visible on the frontend? I'm particularly interested in how this process impacts data security, especially concerning sensitive information that becomes visible externally each time cron runs. Saving all datasets again helps until next cron import. Currently, our frontend is built using React.

[marlubo]

@janette Data Visibility Issue After Running Cron Job

I'm facing a challenge with my system where data visibility settings seem to reset after running a cron job.

Here's the issue:

Background: I have a system where I need to periodically import data into a store using a cron job.
Access Control Settings: When I add a new dataset, I select the access level as "partially open," which should hide certain CSV rows and columns on the frontend. I've configured it to have a "restricted public" access level with the option to hide CSV data.
Problem: However, every time I run the cron job to import data, all data becomes visible again on the frontend, overriding the access control settings I've configured.

Question: I'm seeking guidance on how to ensure that the data remains hidden permanently as per the access control settings, or if there's something wrong with the way the cron job is interacting with the system.
Any insights or suggestions on how to address this issue would be greatly appreciated. Thank you!

[dafeder]

Hi @marlubo in DKAN, which is mostly based on the DCAT-US metadata standard, the accessLevel field does not carry any kind of business logic or access control. It is simply a field that is part of the metadata standard, where data publishers can document whether the dataset is actually publicly available (in some cases government agencies must document the existence of a dataset in their catalog even though it is not publicly available).

DKAN's datastore is meant to provide API access to public datasets, and is not configured for fine-grained access controls. You would I think need to override DKAN's routes for the datastore module and add in your own logic to accomplish this I think.

If you do come up with a more general-purpose solution we will of course be happy to review any PRs.