Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PQputCopyData's return value 0 should be considered fail #7152

Merged
merged 1 commit into from Aug 29, 2023

Conversation

zhjwpku
Copy link
Contributor

@zhjwpku zhjwpku commented Aug 28, 2023

PQputCopyData returns 0 when it failed to expand the output buffer in
nonblock mode. So 0 should be treating as fail.

DESCRIPTION: Fixes a bug that could cause COPY logic to skip data in case of OOM

PQputCopyData returns 0 when it failed to expand the output buffer
in nonblock mode. So 0 should be treating as fail.

Signed-off-by: Zhao Junwang <zhjwpku@gmail.com>
@zhjwpku zhjwpku force-pushed the PutRemoteCopyData_wrong_logic branch from bd18932 to 6f4c8ce Compare August 28, 2023 09:32
@codecov
Copy link

codecov bot commented Aug 28, 2023

Codecov Report

Merging #7152 (6f4c8ce) into main (d5d1684) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #7152      +/-   ##
==========================================
+ Coverage   93.40%   93.41%   +0.01%     
==========================================
  Files         272      272              
  Lines       58989    58991       +2     
==========================================
+ Hits        55101    55109       +8     
+ Misses       3888     3882       -6     

@@ -716,14 +716,14 @@ PutRemoteCopyData(MultiConnection *connection, const char *buffer, int nbytes)
Assert(PQisnonblocking(pgConn));

int copyState = PQputCopyData(pgConn, buffer, nbytes);
if (copyState == -1)
if (copyState <= 0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From docs:

The result is 1 if the data was queued, zero if it was not queued because of full buffers (this will only happen in nonblocking mode),

this does not seem like a failure

however, the current code does seem suspect

in practice, we might always end up in FinishConnectionIO below (?), but maybe we should force it to go in there if copyState == 0

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm, ok, on inspecting the code, the 0 seems to happen only case of out of memory.

so there's a choice here between failing immediately (current PR) or trying a FinishConnectionIO, but current code sems risky

Copy link
Contributor Author

@zhjwpku zhjwpku Aug 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From docs:

The result is 1 if the data was queued, zero if it was not queued because of full buffers (this will only happen in nonblocking mode),

this does not seem like a failure

however, the current code does seem suspect

in practice, we might always end up in FinishConnectionIO below (?), but maybe we should force it to go in there if copyState == 0

IMHO FinishConnectionIO may not help here, it tries to send all pending data(which is hold in conn->outBuffer) for the connection, but when copyState == 0 the data PQputCopyData try to send even not added to conn->outBuffer, we should either treat this as a failure or try PQputCopyData again, but should not treat this as a success.

I checked postgres upstream, all PQputCopyData are used with <= 0, the result 0 is returned when failed to enlarge conn->outBuffer.

@marcocitus marcocitus merged commit d97f786 into citusdata:main Aug 29, 2023
110 checks passed
@zhjwpku zhjwpku deleted the PutRemoteCopyData_wrong_logic branch August 29, 2023 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants