Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Added schema support for SQLite connections #67

Merged
merged 22 commits into from
Nov 27, 2023

Conversation

marcusmunch
Copy link
Collaborator

@marcusmunch marcusmunch commented Nov 13, 2023

Intent

This PR adds improved SQLite support to SCDB by allowing schemas in SQLite connections.

Approach

schema_exists and get_tables have been reworked to S3 classes.
schema_exists looks up the schema name in pragma_table_list and information_schema.schemata for SQLite and Postgres respectively.
get_tables currently only has a method (beside default) for SQLiteConnection, since the previous function was applicable in Postgres without using Postgres-specific features.

This in turn broke a lot of examples, likely due to schema-related functions being treated uniquely for SQLiteConnections.
These examples have been updated to only fetch the items in question if relevant schemata/tables exist.

An argument allow_table_only (default TRUE) has been added to id which returns a DBI::Id table = db_table_id if a schema cannot be parsed from the input.

Known issues

The return value of id with allow_table_only = TRUE changes depending on the presence of a matching schema, which may not be apparent to the user. For more roustness, DBI::Id should be used.
The default value of TRUE was chosen for backwards compatibility and may be changed pending a v1.0 release.

Checklist

  • The PR passes all local unit tests
  • I have documented any new features introduced
  • If the PR adds a new feature, please add an entry in NEWS.md
  • A reviewer is assigned to this PR

Marcus Munch Grünewald added 3 commits November 8, 2023 16:59
* schema_exists: Reworked to S3 class
  - Added methods for SQLite and Postgres using pragma_table_list and
    information_schema.schemata respectively
  - Previous method moved to schema_exists.default

* get_tables: Reworked to S3 class
  - Method for SQLiteConnection calls pragma_table_list, discarding
    "temp" schema and sqlite_stat* tables
@marcusmunch marcusmunch added the enhancement New feature or request label Nov 13, 2023
@marcusmunch marcusmunch added this to the v0.2.1 milestone Nov 13, 2023
@marcusmunch marcusmunch self-assigned this Nov 13, 2023
@marcusmunch
Copy link
Collaborator Author

I now notice that 605861c has become a part of this PR. However, it should not break stuff (it passes my local tests)

@marcusmunch marcusmunch marked this pull request as ready for review November 21, 2023 13:07
R/create_table.R Outdated Show resolved Hide resolved
R/get_schema.R Outdated Show resolved Hide resolved
R/get_schema.R Outdated Show resolved Hide resolved
R/get_schema.R Outdated Show resolved Hide resolved
R/get_table.R Outdated Show resolved Hide resolved
@@ -74,14 +74,15 @@ test_that("get_table returns list of tables if no table is requested", { for (co
test_that("table_exists fails when multiple matches are found", {
for (conn in conns) {
# This test exploits ambiguous notation requiring schemas to exits
if (!inherits(conn, "PqConnection")) next
if (!inherits(conn, "SQLiteConnection")) next
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You dont want this test to run on Postgres?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tests exploit the same case, so I would actually be in favor of keeping the test only on SQLite, which would in turn no longer require the schema test.one to exist in the Postgres database.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally, as the test now calls ATTACH [ expr ] AS [ schema-name ] — which is most likely SQLite only — I decided against writing a "universal" test, as any difference in results across backends would trace back to get_tables behaving inconsistently across backends.

Also, as this test is the only one which requires a specific schema ("test.one") to exist, it's better to quickly attach a temporary file in SQLite than to write a test for all supported backends.

For once, SQLite reigns supreme.

Copy link
Contributor

@RasmusSkytte RasmusSkytte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only a few comments / thing I want to double check.

  1. I am not sure the logic of table_exists.SQLiteConnection is correct. But I may be missing something...

  2. A lot of examples now has a call to schema_exists. I think you can make these more pedagogical by inserting a SCDB::id() in the example chan. (See specifc comment).
    If you agree, it should be changed throughout.

  3. Similarily, there are examples with calls to table_exists which I don't understand why are there. It seems the table is created just before?

Copy link
Contributor

@RasmusSkytte RasmusSkytte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only a few comments / thing I want to double check.

  1. I am not sure the logic of table_exists.SQLiteConnection is correct. But I may be missing something...

  2. A lot of examples now has a call to schema_exists. I think you can make these more pedagogical by inserting a SCDB::id() in the example chan. (See specifc comment).
    If you agree, it should be changed throughout.

  3. Similarily, there are examples with calls to table_exists which I don't understand why are there. It seems the table is created just before?

Copy link
Contributor

@RasmusSkytte RasmusSkytte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have linked the documentation for your S3 methods for table_exists but not for get_schema, schema_exists and get_tables.

The S3 methods for db_timestamp also has linked documentation.
Do you want to link get_schema, schema_exists and get_tables also?

I think it makes the documentation more obvious as to what methods exists.
That is, you can read it in the docs without using methods.

@marcusmunch marcusmunch merged commit 81a443d into ssi-dk:main Nov 27, 2023
13 checks passed
@marcusmunch
Copy link
Collaborator Author

I've merged the branch as-is. From what I can see, the Diseasy documentation guide and references do not contain any one guide as to whether or not S3 methods should always be linked with @rdname and @name and I think further discussion in this regard belongs in ssi-dk/diseasy#76.

As for SCDB, #69 has been opened to ensure all S3 methods are reviewed as part of the v1.0 milestone.

@marcusmunch marcusmunch deleted the sqlite_schema branch November 27, 2023 08:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants