Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TST: Refactor sql test classes. #49757

Merged
merged 1 commit into from
Nov 18, 2022
Merged

TST: Refactor sql test classes. #49757

merged 1 commit into from
Nov 18, 2022

Conversation

cdcadman
Copy link
Contributor

  • closes #xxxx (Replace xxxx with the GitHub issue number)
  • Tests added and passed if fixing a bug or adding a new feature
  • All code checks passed.
  • Added type annotations to new arguments/methods/functions.
  • Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

After #49531 , the sql.SQLDatabase class only accepts SQLAlchemy Connections and not Engines. As a result, any test class that would have passed an Engine had to be turned off in that PR, even though it was still needed to support the test classes which passed Connections. The _EngineToConnMixin class bridged between the Engine test classes and the Connection test classes.

In this refactor, I remove the _EngineToConnMixin class and edit the setup and teardown methods so that the number of classes is reduced. The number of tests in test_sql.py and their outcomes remain the same. With sqlalchemy installed, 330 pass, 6 xfail, and 2 are skipped. Without sqlalchemy, 83 pass, and 255 are skipped. The tests seem to be slightly faster after the refactor.

This refactor will simplify the PR that I'm ultimately working towards (#48576).

@mroeschke mroeschke added Testing pandas testing functions or related to the test suite IO SQL to_sql, read_sql, read_sql_query labels Nov 17, 2022
Copy link
Member

@mroeschke mroeschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noting that ideally we would like the use the connection fixtures at the top of the file instead of the setup_method to set up connections. If you see an easy opportunity to incorporate them into these test happy to have that change as well (if not can be a followup for someone else)

@cdcadman
Copy link
Contributor Author

Unfortunately, that would take a while for me to figure out.


@classmethod
def teardown_class(cls):
cls.engine.dispose()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this still be in the SQLAlchemyMixIn so that the other test classes that inherit this also dispose the engine?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, I am changing this.

def setup_method(self, iris_path, types_data):
try:
self.conn = self.engine.connect()
self.conn.begin()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the commit or rollback handled in sql.SQLDatabase?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sql.SQLDatabase doesn't commit or rollback, but sql.SQLiteDatabase does both. I clarified this behavior in the to_sql documentation in #49531 after confirming that it's true in pandas version 1.5.1.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay great thanks for confirming

Copy link
Member

@mroeschke mroeschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. Just one question

@mroeschke mroeschke added this to the 2.0 milestone Nov 18, 2022
@mroeschke mroeschke merged commit 4f3c381 into pandas-dev:main Nov 18, 2022
@mroeschke
Copy link
Member

Thanks @cdcadman

@cdcadman cdcadman deleted the test_refactor branch November 19, 2022 01:08
mliu08 pushed a commit to mliu08/pandas that referenced this pull request Nov 27, 2022
Co-authored-by: Chuck Cadman <charles.cadman@standard.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
IO SQL to_sql, read_sql, read_sql_query Testing pandas testing functions or related to the test suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants