Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove unused code and improve the Athena Linker #1775

Merged
merged 9 commits into from
Dec 2, 2023

Conversation

ThomasHepworth
Copy link
Contributor

Type of PR

  • BUG
  • FEAT
  • MAINT
  • DOC

Is your Pull Request linked to an existing Issue or Pull Request?

Should resolve #1686

Give a brief description for the solution you have provided

This PR:

  1. Addresses the issue above, removing any mention of _names_of_tables_created_by_splink.
  2. Changes how the athena linker performs drop_tables_in_current_splink_run. Now, it will look at tables in the cache and clean these.
  3. Streamlines and simplifies some of the existing athena tests.

Comments:

  • I wonder if it makes sense to generalise drop_tables_in_current_splink_run for other linkers. This would then simplify the clean-up process for all users, across all linkers.

PR Checklist

  • Added documentation for changes
  • Added feature to example notebooks or tutorial (if appropriate)
  • Added tests (if appropriate)
  • Updated CHANGELOG.md (if appropriate)
  • Made changes based off the latest version of Splink
  • Run the linter

@@ -1,23 +1,26 @@
import os

# Skip if no valid boto3 connection exists
# try:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is what's causing the tests to fail

Copy link
Member

@RobinL RobinL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy for you to merge this once the tests are passing

@ThomasHepworth ThomasHepworth merged commit 9d20750 into master Dec 2, 2023
10 checks passed
@ThomasHepworth ThomasHepworth deleted the remove_names_of_tables_created_by_splink branch December 2, 2023 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

_names_of_tables_created_by_splink is unused
2 participants