New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Schema sqlalchemy engine caching is per-instance #280
Labels
affects: architecture
Improvements or additions to architecture
affects: technical debt
Improves the state of the codebase
ready
Ready for implementation
type: bug
Something isn't working
work: backend
Related to Python, Django, and simple SQL
Comments
kgodey
added
ready
Ready for implementation
work: backend
Related to Python, Django, and simple SQL
labels
Jun 24, 2021
Added this to milestone 2 because it seems important to resolve ASAP. |
Note: We have a temporary fix in place as of #265 which should be removed when this issue is handled. |
kgodey
added
affects: architecture
Improvements or additions to architecture
affects: technical debt
Improves the state of the codebase
labels
Aug 22, 2021
kgodey
modified the milestones:
[06] General Improvements #4,
[10] General Improvements #5
Oct 25, 2021
kgodey
modified the milestones:
[05.2] 2021-12 improvements,
[07.2] 2022-02 improvements
Dec 17, 2021
kgodey
modified the milestones:
[07.2] 2022-02 improvements,
[10.1] 2022-03 improvements
Feb 25, 2022
kgodey
modified the milestones:
[06.1] 2022-03 improvements,
[09.1] 2022-04 improvements
Mar 31, 2022
kgodey
modified the milestones:
[06.2] 2022-04 improvements,
[08.1] 2022-05 improvements
May 2, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
affects: architecture
Improvements or additions to architecture
affects: technical debt
Improves the state of the codebase
ready
Ready for implementation
type: bug
Something isn't working
work: backend
Related to Python, Django, and simple SQL
Describe the bug
The
_sa_engine
property of theSchema
model is cached using the@cached_property
decorator.@cached_property
caches on a per-instance basis, which means the engine is not cached across different instances of the same schema object, leading to the engine being created many times.For example, each table must access it's schema's engine which is then cached on that table's instance of the schema. However, as the engine is only cached on that instance, another table trying to access the same schema's engine will also create a new engine. As a result, something like the table list endpoint will create an engine for every table in the database.
Expected behavior
We should cache the schema's engine object at a higher level so it persists across instances.
Additional context
Issues with too many connections, specifically the
too many clients
error, is blocking backend work that adds additional tests (see #265, #268). A naive caching implementation:does resolve the error on #265. Unfortunately, I haven't been able to consistently reproduce the error as a single test.
I have also talked to @mathemancer about opening a broader issue on the where engines should be created. If this overlaps with that topic, I'm happy to close.
The text was updated successfully, but these errors were encountered: