Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delete intermediate tables after use #89

Conversation

RasmusSkytte
Copy link
Contributor

@RasmusSkytte RasmusSkytte commented Jan 19, 2024

Intent

Fixes #86

Approach

  • A new function defer_db_cleanup was added which uses withr::defer to remove the specified table when the function exits.
    By inserting this function into our functions when we generate new tables, the intermediate tables created will automatically be removed once the function returns.

EDIT: to clarify, it will delete the table once the parent of defer_db_cleanup exits, not when defer_db_cleanup exits.

  • A new function unique_table_name() is added (inspired by dbplyrs internal function of the same name) which gives an easy way to create table names like SCDB_### for tables.
    By using this function, we can keep track of the tables that we explicitly need and which should be kept when the function returns (which we can give the "SCDB_" prefix) and those we implicitly create in intermediate computations and should not remain after the function returns (which will get the "dbplyr_" prefix).

  • A new testing helper connection_clean_up() is added that is used at the end of tests that use a data base backend.
    This function looks for the existence of dbplyr_### tables and gives a warning if they exists. Then it closes the connection to the data base.

Known issues

This PR also contains the code being merged in

Once merged, this PR will be rebased and opened.

Checklist

  • The PR passes all local unit tests
  • I have documented any new features introduced
  • If the PR adds a new feature, please add an entry in NEWS.md
  • A reviewer is assigned to this PR

@RasmusSkytte RasmusSkytte added the enhancement New feature or request label Jan 19, 2024
@RasmusSkytte RasmusSkytte self-assigned this Jan 19, 2024
@RasmusSkytte RasmusSkytte linked an issue Jan 19, 2024 that may be closed by this pull request
1 task
@RasmusSkytte RasmusSkytte requested review from a team, kaare-gr and LasseEngboChr and removed request for a team, kaare-gr and LasseEngboChr January 19, 2024 13:40
@RasmusSkytte RasmusSkytte marked this pull request as draft January 24, 2024 07:34
@RasmusSkytte RasmusSkytte force-pushed the fix/86-lacking-clean-up-of-temporary-tables-in-update_snapshot branch from f5de25e to 4926073 Compare February 1, 2024 10:52
@RasmusSkytte RasmusSkytte mentioned this pull request Feb 1, 2024
6 tasks
@RasmusSkytte RasmusSkytte force-pushed the fix/86-lacking-clean-up-of-temporary-tables-in-update_snapshot branch from 4926073 to 2ef6de3 Compare February 1, 2024 14:09
@RasmusSkytte RasmusSkytte marked this pull request as ready for review February 1, 2024 14:18
@RasmusSkytte RasmusSkytte requested review from a team, kaare-gr and LasseEngboChr and removed request for a team February 1, 2024 14:18
@RasmusSkytte RasmusSkytte changed the title Delete intermediate tables after use in update_snapshot Delete intermediate tables after use Feb 1, 2024
@RasmusSkytte RasmusSkytte force-pushed the fix/86-lacking-clean-up-of-temporary-tables-in-update_snapshot branch from 2ef6de3 to 2b5b8fb Compare February 5, 2024 10:59
@RasmusSkytte RasmusSkytte added this to the v0.4 milestone Feb 6, 2024
@RasmusSkytte RasmusSkytte merged commit d0d41ab into main Feb 8, 2024
23 of 24 checks passed
@RasmusSkytte RasmusSkytte deleted the fix/86-lacking-clean-up-of-temporary-tables-in-update_snapshot branch February 8, 2024 10:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Lacking clean up of temporary tables in update_snapshot
2 participants