Skip to content

Fix Postgres "idle in transaction" when getting current user #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Conversation

abionics
Copy link

When interacting with a Postgres database, particularly during the process of querying user info, I found that the database connection remains persistently open in an idle in transaction state. This issue may lead to inefficient resource utilization and potential database locks. So I propose to add explicit rollback command after the query operation. This will change connection state to a standard idle

Steps to reproduce

  1. Clone fastapi-users sqlalchemy examples
  2. Make breakpoint in some auth-only endpoint, for example in this one
  3. Open Postgres console and query active sessions:
SELECT state, query FROM pg_stat_activity WHERE client_addr IS NOT NULL;
-- | state                | query                 |
-- | -------------------- | --------------------- |
-- | idle in transaction  | SELECT "user".id, ... | <--- here
-- | active               | SELECT state, ...     |

@abionics
Copy link
Author

@frankie567 friendly ping on this

@frankie567
Copy link
Member

I'm not sure the behavior you describe is accurate. If you breakpoint inside the execution of an endpoint, then yes, your session will be idle. However, after the endpoint execution, FastAPI will release the SQLAlchemy session (that's the purpose of the yield):

async def get_async_session() -> AsyncGenerator[AsyncSession, None]:
    async with async_session_maker() as session:
        yield session

https://github.com/fastapi-users/fastapi-users/blob/ff9fae631cdae00ebc15f051e54728b3c8d11420/examples/sqlalchemy/app/db.py#L28-L30

@abionics
Copy link
Author

Yes, after the endpoint execution, FastAPI will release the SQLAlchemy session. This problem exists only during the execution of an endpoint. For example, when the endpoint has some long tasks that takes several seconds, especially IO-bound tasks that don't require a database connection

@frankie567
Copy link
Member

I see. Given that's it's a quite specific use-case, I'm not really confident into adding a rollback here in all cases. If those idle transactions really are a problem for you (TBH, with a standard PostgreSQL server, I don't think it's a real problem at standard scale), I would suggest to either run your IO tasks in the background or tweak SQLAlchemy settings to open a session that's not inside a transaction.

@frankie567 frankie567 closed this Dec 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants