Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch unit tests to use temp cohort tables to avoid collisions between runs #166

Closed
schuemie opened this issue Jun 3, 2022 · 2 comments

Comments

@schuemie
Copy link
Member

schuemie commented Jun 3, 2022

Currently the unit tests use a permanent table called cohorts_of_interest (see here and here, with SQL here). This means that if multiple processes try to create this table simultaneously there could be collisions. In fact, because the unit tests take over 2 hours to complete, it is not unlikely that this will happen!

A solution could be to use a temp table instead. This would require that the connection be kept alive across the various tests (or else the temp table is automatically dropped). Also remember that the temp tables need to be explicitly deleted after finishing the tests, for platforms where we merely emulate temp tables (like Oracle, Spark, and BigQuery)

@anthonysena
Copy link
Collaborator

A few notes on this issue:

First steps

  • Modify inst/sql/sql_server/cohortsOfInterest.sql to parameterize the target table from:

SELECT first_use.*
INTO @resultsDatabaseSchema.cohorts_of_interest
FROM (

to a temp table that either uses a dynamic name that is unique per session (so different OS unit tests do not collide) or add a suffix to the temp table name per OS. Modify all other dependent code to use the temp table and clean up when the test is complete so they are not lingering on platforms such as Oracle.

Next steps

@anthonysena
Copy link
Collaborator

Closed via #185

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants