Skip to content

Db query evaluation#37

Merged
hardikgu23 merged 23 commits intomainfrom
DBQueryEvaluation
Oct 10, 2024
Merged

Db query evaluation#37
hardikgu23 merged 23 commits intomainfrom
DBQueryEvaluation

Conversation

@hardikgu23
Copy link
Copy Markdown
Collaborator

Seperate execution for dql, dml and ddl queries
Add flter by evalID range, tags
Add retry in setup-teardown

@mahyareb
Copy link
Copy Markdown
Collaborator

@hardikgu23 is this ready for review? Sorry if I missed it.

@hardikgu23
Copy link
Copy Markdown
Collaborator Author

@hardikgu23 is this ready for review? Sorry if I missed it.

Hi @mahyareb , yes it is.

IsmailMehdi
IsmailMehdi previously approved these changes Oct 7, 2024
return generate_ddl(rows, self.db_name)

def _execute(self, query: str) -> Tuple[Any, float]:
def _execute(self, query: str, rollback: bool = False, use_transaction: bool = True) -> Tuple[Any, float]:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it doesn't seem like we're doing anything with the transaction and rollback. Is that a TODO?

Copy link
Copy Markdown
Collaborator Author

@hardikgu23 hardikgu23 Oct 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, From discussion: plan is to have seperate execution path for dql, dml and ddl execution. So leaving this as it is so as to not further complicate the pr.

if rollback:
transaction.rollback()
else:
for query in queries:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you still need "with connection.begin():" here?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do not need the use_transaction parameter anymore for mysql. So this will be changed in different pr.

if rollback:
transaction.rollback()
else:
resultset = connection.execution_options(isolation_level="AUTOCOMMIT").execute(text(query))
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are we using AUTOCOMMIT for here?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Databases in postgres cannot be created within transaction block. Autocommit is being used here to execute queries in non-transactional mode.
From sqlalchemy documentation:
https://docs.sqlalchemy.org/en/20/orm/session_transaction.html#setting-transaction-isolation-levels-dbapi-autocommit.

if error:
logging.error(f"Error while creating user: {error}")
return
setup_file = os.path.join(os.path.dirname(__file__), f"schema_details/bat/{database}/setup.yaml")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This right now just points to bat in the directory, could you get this from the config?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great suggestion. To not further complicate the pr, will create a seperate pr doing the same.

Copy link
Copy Markdown
Collaborator

@mahyareb mahyareb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM Hardik! Great work. Left some comments in there.

@hardikgu23 hardikgu23 merged commit d97de66 into main Oct 10, 2024
@hardikgu23 hardikgu23 deleted the DBQueryEvaluation branch November 5, 2024 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants