Skip to content

New methods for DatabaseManager and improved performance!

Latest
Latest
Compare
Choose a tag to compare
@antonlydell antonlydell released this 06 Feb 20:25
v1.2.0
31837f5

Added

  • DatabaseManager.upsert_table() : Update a table with data from a pandas.DataFrame and insert new rows if any.

  • DatabaseManager.merge_table() : Merge data from a pandas.DataFrame into a table.

  • DatabaseManager.save_df() :

    • Added the 'drop-replace' option to the if_exists parameter, which drops the table, recreates it, and then writes the
      pandas.DataFrame to the table.

    • Added the parameters datetime_cols_dtype, datetime_format, localize_tz and target_tz, which allows adjusting the timezone
      and data type of datetime columns before saving the pandas.DataFrame to the database.

  • DatabaseManager.load_table() : Added the parameters datetime_cols_dtype and datetime_format, which allows adjusting the data type
    of datetime columns from the loaded pandas.DataFrame.

  • InvalidColumnNameError : Exception raised when supplying an invalid column name to a database operation.

  • SQLStatementNotSupportedError : Exception raised when executing a method that triggers a SQL statement not supported by the
    database dialect. E.g. a merge statement.

Changed

  • The DatabaseManager base class is now a proper class and not an ABC. It can now be initialized from a SQLAlchemy URL or Engine.
    This is useful if you want to use Pandemy, but there is no subclass of DatabaseManager implemented for the desired SQL dialect.
    It should only be used directly if there is no dedicated subclass since it has limited functionality.

  • DatabaseManager and its subclasses now use __slots__, which improves performance of attribute access and lowers memory usage.

  • SQLiteDb and OracleDb now support to be initialized with a SQLAlchemy connection URL or Engine.

  • Placeholder has been refactored from a namedtuple into a proper class that is using __slots__.

Fixed

  • DatabaseManager.save_df() :

    • The option 'replace' of the if_exists parameter now correctly deletes the data of the existing table before writing the pandas.DataFrame
      instead of dropping the table, recreating it, and finally writing the pandas.DataFrame to the table. The old behavior has been moved to the
      'drop-replace' option (see section Added above).

    • The method now correctly raises TableExistsError and SaveDataFrameError. A ValueError raised by the pandas.DataFrame.to_sql() method,
      that is not related to "table exists and if_exists='fail'", is now translated into a SaveDataFrameError and not a TableExistsError as before.

Deprecated

  • The methods OracleDb.from_url() and OracleDb.from_engine() are now deprecated since their functionality is now fulfilled
    by the url and engine parameters added to the original constructor.

  • The SQLiteDb.conn_str attribute is deprecated and replaced by the url attribute.

Assets 3