Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

database/sql: Go 2: Is context cancellation on QueryContext meant to cancel row.Scan? #28842

Closed
pjebs opened this issue Nov 17, 2018 · 8 comments
Closed

Comments

@pjebs
Copy link
Contributor

@pjebs pjebs commented Nov 17, 2018

ctx, cancel := context.WithTimeout(context.Background(), 250*time.Millisecond)
defer cancel()

rows, err := conn.QueryContext(ctx, stmt)
if err != nil {
   if err == context.DeadlineExceeded {
      // context canceled
   }
   return
}

for rows.Next() {
   if err := rows.Scan(&name); err != nil {
      if err == context.DeadlineExceeded {
         // Is this intended?
      }
   }
}

@pjebs pjebs changed the title database/sql: Is context cancellation on QueryContext meant to cancel row.Scan? database/sql: Go 2: Is context cancellation on QueryContext meant to cancel row.Scan? Nov 17, 2018
@pjebs
Copy link
Contributor Author

@pjebs pjebs commented Nov 17, 2018

If the context gets canceled after the QueryContext but during the scan loop, currently it errors out with context.DeadlineExceed.

Is this intended?
If so, is it wise?

@kardianos

@kardianos
Copy link
Contributor

@kardianos kardianos commented Nov 17, 2018

Is this intended?

Yes

If so, is it wise?

Yes?

@pjebs
Copy link
Contributor Author

@pjebs pjebs commented Nov 17, 2018

The issue I had was that I intended the context to cancel the query only. Once the query completed, and since the context hadn't cancelled while the query was in progress, I didn't expect it would interfere with gathering the results.

Perhaps the documentation should make clear that query cancellation will have possible after-effects after the actual QueryContext( ) returns.

One could argue that my interpretation is better because if I want the query to cancel, the status quo does what I believed it would do.

If I want the context to cancel the scan loop as well, I can easily check the ctx.Err() inside the loop.
That gives better flexibility. If I want the scan loop to continue (since I only wanted to cancel the query and not the scan process), then I have that option too.

@pjebs
Copy link
Contributor Author

@pjebs pjebs commented Nov 17, 2018

I also feel the current design conflicts with Go's API design with regards to context cancellaton.

When a context is passed into a function, I don't expect there to be side-effects after that function returns.

@kardianos
Copy link
Contributor

@kardianos kardianos commented Nov 17, 2018

A context cancelled means you've run out of your time slice and should stop and return. Context isn't just for remote functions. Within local services, I use ctx all the time, even without any remote functions.

@pjebs
Copy link
Contributor Author

@pjebs pjebs commented Nov 17, 2018

A context cancelled means you've run out of your time slice and should stop and return.

Agreed.

Context isn't just for remote functions.

Agreed.

Within local services, I use ctx all the time, even without any remote functions.

Agreed. So do I.

I disagree with your interpretation of Go API design with regards to context cancellation.
I feel your interpretation is too rigid.

My contention is when you provide a context to QueryContext( ), standard Go API design dictates that the function should respect the context sending a signal that it is canceled. If the context is cancelled after that function has already returned, it should not interfere any further. That is what I expect from the function. It is also what I believe most people expect (but albeit empirically untested).

Under the status quo, when the context is cancelled after the function returns - but during the scan-loop, it is interfering. It is preventing the scan from working during the scan loop.
What is better API design is to not interfere with the scan loop. IF the programmer desires the current implementation, it is trivial to check for context cancellation inside the loop.

for rows.Next() {
   if err := ctx.Err(); err != nil {
      return err
   }

   if err := rows.Scan(&name); err != nil {
      return err
   }
}

My suggestion makes it trivial to implement the current status quo implementation. My suggestion also gives the flexibility to treat the scanning process independently of the query execution.

I feel it is also more consistent with people's intuitive understanding of how context cancellation should work.

If I have successfully persuaded you, then the follow-up question is what to do with the Go 1 backward compatibility promise. Do you keep it the way it is and fix it for Go 2? Or do you consider it a mistake in the first place and update it for Go 1 to operate in line with what I believe is the correct interpretation?

Perhaps @robpike can shed light on how context cancellation should be interpreted. I looked in the specs and could not find a definitive interpretation.

Now on a separate but related note, if my interpretation is actually incorrect or deemed incorrect, then the documentation should state for QueryContext( ) that context cancellation can have this after-effect.

@kardianos
Copy link
Contributor

@kardianos kardianos commented Nov 17, 2018

A query function is not complete until all rows are scanned. While calling Scan, the database connection will still be sending results. In other words, if you consider Query a rpc call, it is still retuning.

@pjebs
Copy link
Contributor Author

@pjebs pjebs commented Nov 17, 2018

Maybe it should be added to documentation.

@kardianos kardianos closed this Nov 18, 2018
@golang golang locked and limited conversation to collaborators Nov 18, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.