Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix passing DBI::Id to update_snapshot and dbplyr messages #72

Merged
merged 50 commits into from
Dec 11, 2023

Conversation

marcusmunch
Copy link
Collaborator

@marcusmunch marcusmunch commented Nov 29, 2023

Intent

This fixes #68.

Additionally, while fixing, I managed to squash all the messages about in_schema caused by dbplyr v2.4.0 giving this warning VERY often1.

Approach

update_snapshot had some "leaky" logic, not properly catching all cases. Now, the addition of id.tbl_dbi allows for easy retrieval of a table ID to use onwards. As we now do not expect any db_table arguments to not be passable by id, this will catch all valid inputs.

Regarding the numerous messages from dbplyr, I hope that they will be changed to a more graceful form, but for now, all calls to dplyr::tbl and dplyr::copy_to in tests and code have been changed to include check_from = FALSE.

Also, some instances of above messages were caused by the Logger unnecessarily copying a data.frame twice in one log update.

Known issues

I concede that passing check_from = FALSE to all calls mentioned above is a somewhat lazy solution. One solution that I thought of just now (and that I could honestly see working better) is to implement some kind of check to see if the caller env is .GlobalEnv and give a warning at most ONCE per session.

The current state of logger$log_to_db has no race condition prevention. This is neither introduced nor fixed by this PR, but race condition prevention is an overall topic of #63 and therefore outside the scope of this PR.

Checklist

  • The PR passes all local unit tests
  • I have documented any new features introduced
  • If the PR adds a new feature, please add an entry in NEWS.md2
  • A reviewer is assigned to this PR

Footnotes

  1. Naturally, you will not find me missing an opportunity to brag that I managed to identify—and fix—the issue myself.

  2. In order to not mangle NEWS.md too much, this will be done after v0.2.1 is accepted on CRAN.

@marcusmunch marcusmunch added the bug Something isn't working label Nov 29, 2023
@marcusmunch marcusmunch added this to the v0.2.2 milestone Nov 29, 2023
@marcusmunch marcusmunch self-assigned this Nov 29, 2023
chore: Remove SQLite-specific functions from `get_tables.default`
R/connection.R Show resolved Hide resolved
R/connection.R Outdated Show resolved Hide resolved
R/get_schema.R Show resolved Hide resolved
R/connection.R Outdated Show resolved Hide resolved
@marcusmunch

This comment was marked as outdated.

Copy link
Contributor

@RasmusSkytte RasmusSkytte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Firstly, there a few places where it looks like you have made minor mistakes.

Secondly, I have some general comments, tips etc.

Finally, I have some pedantic comments you can take or leave as you please.

NEWS.md Outdated Show resolved Hide resolved
R/Logger.R Outdated Show resolved Hide resolved
R/connection.R Show resolved Hide resolved
R/connection.R Show resolved Hide resolved
R/connection.R Outdated Show resolved Hide resolved
R/get_schema.R Show resolved Hide resolved
R/get_table.R Show resolved Hide resolved
R/get_table.R Show resolved Hide resolved
R/get_table.R Outdated Show resolved Hide resolved
R/get_table.R Show resolved Hide resolved
Copy link
Contributor

@RasmusSkytte RasmusSkytte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine to me.

Just need the last check to work, but that should not be an issue.

@marcusmunch marcusmunch merged commit 2c2c727 into main Dec 11, 2023
16 checks passed
@RasmusSkytte RasmusSkytte deleted the update_snapshot_id branch February 6, 2024 09:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

update_snapshots fail when db_table has class DBI::Id
2 participants