-
Notifications
You must be signed in to change notification settings - Fork 13.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fixed PostgresToGCSOperator fail on empty resultset for use_server_side_cursor=True #21307
Fixed PostgresToGCSOperator fail on empty resultset for use_server_side_cursor=True #21307
Conversation
self.initialized = True | ||
self.cursor.fetchone() | ||
self.cursor.scroll(0, mode='absolute') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If .scroll(-1)
is more clear then I change
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like your current approach better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the misleading.
First of all server-side cursor doesn't support relative
mode by default. It should be initialized like conn.cursor(name='...', scrollable=True)
. But absolute mode worked without initialization.
But both scrolls cause a reset of .description
attribute. So, unfortunately, they could not be used.
This comment was marked as outdated.
This comment was marked as outdated.
80c506a
to
e6db1c7
Compare
Are these existing tests to verify the behaviour of |
Today I made few tests: Current version (code from
Updated version (this PR code):
The last one is not what is expected in the issue but I think creating the empty file is the right behavior
|
512f28b
to
1584d42
Compare
1584d42
to
6321d54
Compare
6321d54
to
ad1d827
Compare
Attribute
description
of simple psycopg2 cursor is available after.execute()
method.But for the named cursor (server-side cursor) any
.fetch*
method call is needed.In case of an empty result set like
.execute("SELECT 1 LIMIT 0")
:.description
and an empty iterator.description
and an empty original iterator.But also we have
self.rows=[None]
, as a result offetchone()
on an empty iterator of the server-side cursor.This
None
makes_PostgresServerSideCursorDecorator
non-empty iterator.I decided to removerows
attribute because it could consist of a maximum of one row, and also psycopg2 server-side cursor have.scroll()
method and it is safe to call.scroll()
on empty resultset even without.fetchone()
closes #20007