Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delete and resources leaves unreferenced datastore tables #6675

Open
firstnevyn opened this issue Jan 28, 2022 · 2 comments
Open

Delete and resources leaves unreferenced datastore tables #6675

firstnevyn opened this issue Jan 28, 2022 · 2 comments

Comments

@firstnevyn
Copy link

CKAN version
2.9.4

Describe the bug
As noted at the end of #3422 deleting and purging a dataset still results in datastore tables that are unreferenced.

Steps to reproduce
on a configuration with datapusher enabled upload a dataset with csv tables delete and purge that dataset then recreate it with additional csv lines

Expected behavior
1 table in datastore

**Actual behavior *
current and previous tables in datastore

Additional details
We have a set of csv files from iot sensors the csv files have time series data with real time additions every x minutes every day we drop all these datasets and re-injest them into ckan.

the complete dataset of all data in the system is ~200Mb however our datastore_default is 12Gb after running for a month or so

@firstnevyn firstnevyn changed the title Delete and purging a dataset leaves unreferenced datastore tables Delete and resources leaves unreferenced datastore tables Jan 28, 2022
@wardi
Copy link
Contributor

wardi commented Feb 1, 2022

like in #3425 we could hook the IPackageController after_delete to handle package deletion, then chain the dataset_purge action to handle the purge case.

@wardi
Copy link
Contributor

wardi commented Feb 2, 2022

Possibly better option: make core call IResourceController after_delete for all resources in a dataset that is being deleted or purged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants