-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DM-43079: Add metadata module for encapsulating sqlalchemy binding #47
Commits on Mar 13, 2024
-
Set autoincrement field to None by default
The current logic in Felis's sql module assumes that autoincrement will be None if it is not set explicitly so for now match this logic in the datamodel.
Configuration menu - View commit details
-
Copy full SHA for 08aa636 - Browse repository at this point
Copy the full SHA 08aa636View commit details
Commits on Mar 14, 2024
-
Add metadata module and implementation of SchemaMetaData class
Add a module with a new class that encapsulates the creation of sqlalchemy MetaData from a Pydantic Schema object, along with a test that dumps the corresponding SQL DDL statements to a file. The logic for translating into SQA was primarily derived from the existing visitor class in sql.py, which this is intended to replace eventually.
Configuration menu - View commit details
-
Copy full SHA for bc05ec4 - Browse repository at this point
Copy the full SHA bc05ec4View commit details
Commits on Mar 20, 2024
-
Add command line interface that uses metadata module
Add a "create" command that uses the metadata module instead of the sql module.
Configuration menu - View commit details
-
Copy full SHA for 0eba714 - Browse repository at this point
Copy the full SHA 0eba714View commit details -
Add schema name and create if not exists cli flags
Add flags for specifying an alternate schema name and creating the schema in the database if it does exist already. For MySQL, a new database will be created. In PostgreSQL, a new schema will be created within the current database.
Configuration menu - View commit details
-
Copy full SHA for 8ebd4f8 - Browse repository at this point
Copy the full SHA 8ebd4f8View commit details -
Add ability to drop and create schema using the command line interface
Add several new flags to the CLI for dropping a schema if it exists and creating one if it does not exist. Add methods implementing this functionality to the metadata module.
Configuration menu - View commit details
-
Copy full SHA for 64d23a8 - Browse repository at this point
Copy the full SHA 64d23a8View commit details -
Add simple YAML schema for testing
This is a generic schema representing sales data with customers and orders for testing purposes.
Configuration menu - View commit details
-
Copy full SHA for 460c805 - Browse repository at this point
Copy the full SHA 460c805View commit details
Commits on Mar 29, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 6e885ee - Browse repository at this point
Copy the full SHA 6e885eeView commit details -
Add test case for creating metadata from a schema and loading into a db
Add a test case which loads a simple YAML test file into a schema, creates SQLAlchemy Metadata from the object, creates the database objects within an in-memory SQLite database, and then checks that the reflected MetaData is the same as that created from the schema.
Configuration menu - View commit details
-
Copy full SHA for ca74c58 - Browse repository at this point
Copy the full SHA ca74c58View commit details -
Configuration menu - View commit details
-
Copy full SHA for ff56648 - Browse repository at this point
Copy the full SHA ff56648View commit details -
Comment out bad value in test data
This value is not handled correctly as it ends up being quoted. It will be fixed on another branch.
Configuration menu - View commit details
-
Copy full SHA for fe5b126 - Browse repository at this point
Copy the full SHA fe5b126View commit details -
Add wrapper class for executing SQL on an engine or mock connection
This is used by the metadata module to handle a SQA engine or mock connection. SQL is executed in a with statement for an engine but not for a mock connection, as it does not support this.
Configuration menu - View commit details
-
Copy full SHA for e2fb750 - Browse repository at this point
Copy the full SHA e2fb750View commit details -
Move sql variant utility functions into a new module
The sql module is being deleted but the variant handling utilities need to be kept.
Configuration menu - View commit details
-
Copy full SHA for dd95926 - Browse repository at this point
Copy the full SHA dd95926View commit details -
Remove sql module and its tests
Remove the sql module which is replaced by the metadata module. Remove the "create-all" command from the CLI which is replaced by "create". Remove the InsertDump class from the cli module, as it was moved into the metadata module.
Configuration menu - View commit details
-
Copy full SHA for 63070af - Browse repository at this point
Copy the full SHA 63070afView commit details -
Add builder class for creating SQLAlchemy MetaData from a Schema
Instead of having a class with internal functions for building a MetaData object, use a builder pattern for a cleaner interface.
Configuration menu - View commit details
-
Copy full SHA for 921831d - Browse repository at this point
Copy the full SHA 921831dView commit details -
Correct cli handling for sqlite and implement automatic dry run
Instead of not setting the schema name, use "main" instead which works for sqlite. In case a host name is not set on the database engine URL, force a dry run using the supplied variant.
Configuration menu - View commit details
-
Copy full SHA for e61dde1 - Browse repository at this point
Copy the full SHA e61dde1View commit details -
Add tests for indexes and constraints in MetaData test case
Test that the metadata from the builder and that reflected from the database have equivalent index and constraint objects. Add some additional constraints to the YAML test file.
Configuration menu - View commit details
-
Copy full SHA for b0a01f1 - Browse repository at this point
Copy the full SHA b0a01f1View commit details -
Add flags for applying the schema name to the metadata and tables
In some cases, it is not desirable to apply the schema name to either the metadata or the tables. Add two flags to the builder for controlling this behavior. The flags are true by default and can be set to false if the user does not want the schema to be applied.
Configuration menu - View commit details
-
Copy full SHA for 5ce2533 - Browse repository at this point
Copy the full SHA 5ce2533View commit details -
Prepend underscore to private global variable and change regexp
Change the regular expression to match on numbers rather than any character between the parentheses.
Configuration menu - View commit details
-
Copy full SHA for dd12e4d - Browse repository at this point
Copy the full SHA dd12e4dView commit details -
Use IO type so name attribute can be directly accessed
Fix mypy issue where TextIOBase does not have a visible name attribute.
Configuration menu - View commit details
-
Copy full SHA for 0d28a8a - Browse repository at this point
Copy the full SHA 0d28a8aView commit details -
Configuration menu - View commit details
-
Copy full SHA for bc5c4d2 - Browse repository at this point
Copy the full SHA bc5c4d2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9f8a844 - Browse repository at this point
Copy the full SHA 9f8a844View commit details -
Add utility method for getting column's datatype with variants
Provide a utility method for getting the SQLAlchemy datatype from a column object in the Pydantic model, including the variant type handling.
Configuration menu - View commit details
-
Copy full SHA for 88d5efe - Browse repository at this point
Copy the full SHA 88d5efeView commit details -
Add test comparing metadata to the schema from which it was built
This test reads in a test schema, creates a SQLAlchemy MetaData object from it, and then compares the schema to the metadata to check that they have the equivalent, expected information.
Configuration menu - View commit details
-
Copy full SHA for 05fd371 - Browse repository at this point
Copy the full SHA 05fd371View commit details -
Simplify handling of different constraint types in builder method
Remove exception checking for each constraint type as it is unnecessary. In the unlikely case that a constraint was not built properly, throw an exception.
Configuration menu - View commit details
-
Copy full SHA for 238ce62 - Browse repository at this point
Copy the full SHA 238ce62View commit details -
Configuration menu - View commit details
-
Copy full SHA for 01ecdc2 - Browse repository at this point
Copy the full SHA 01ecdc2View commit details -
Configuration menu - View commit details
-
Copy full SHA for c922293 - Browse repository at this point
Copy the full SHA c922293View commit details -
Configuration menu - View commit details
-
Copy full SHA for eaa3424 - Browse repository at this point
Copy the full SHA eaa3424View commit details -
Configuration menu - View commit details
-
Copy full SHA for ab96c6d - Browse repository at this point
Copy the full SHA ab96c6dView commit details -
Add 'postgresql:datatype' field to Column
This field is required so that a PostgreSQL datatype can be optionally provided in the column information.
Configuration menu - View commit details
-
Copy full SHA for e9afb14 - Browse repository at this point
Copy the full SHA e9afb14View commit details -
Configuration menu - View commit details
-
Copy full SHA for 410aa47 - Browse repository at this point
Copy the full SHA 410aa47View commit details -
Configuration menu - View commit details
-
Copy full SHA for 02bfc75 - Browse repository at this point
Copy the full SHA 02bfc75View commit details -
Configuration menu - View commit details
-
Copy full SHA for 73c5fb5 - Browse repository at this point
Copy the full SHA 73c5fb5View commit details -
Remove use_enum_values from Pydantic data model
This setting confuses mypy and the syntax seems clearer when using the enum object instead.
Configuration menu - View commit details
-
Copy full SHA for 3cd31a2 - Browse repository at this point
Copy the full SHA 3cd31a2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9eb8e95 - Browse repository at this point
Copy the full SHA 9eb8e95View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1f6a0a4 - Browse repository at this point
Copy the full SHA 1f6a0a4View commit details -
Change primaryKey to primary_key in Pydantic model for consistency
All of the other fields use snake case and aliases for camelcase variables, so change primary key to match.
Configuration menu - View commit details
-
Copy full SHA for 35a6c42 - Browse repository at this point
Copy the full SHA 35a6c42View commit details
Commits on Apr 1, 2024
-
Configuration menu - View commit details
-
Copy full SHA for b955513 - Browse repository at this point
Copy the full SHA b955513View commit details