Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

do not log an error on unset variable delete #3652

Merged
merged 6 commits into from
Apr 14, 2020

Conversation

jjhelmus
Copy link
Contributor

@jjhelmus jjhelmus commented Mar 27, 2020

If a Variable is never set or accessed no entries are made in the
tracking attributes of VariableExtension. Therefore there is no need to
raise or log an error when the variable is deleted.

closes #3492

If a Variable is never set or accessed no entries are made in the
tracking attributes of VariableExtension.  Therefore there is no need to
raise or log an error when the variable is deleted.
@jjhelmus
Copy link
Contributor Author

I've added a test here but it passes even without this change. An error is logged by the scheduler but I'm not sure how to turn this into a failure in the test.

@mrocklin
Copy link
Member

An error is logged by the scheduler but I'm not sure how to turn this into a failure in the test.

Try grepping for captured_logger

def test_delete_unset_variable(client):
x = Variable()
assert x.client is client
x.delete()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I encourage you to use gen_cluster whenever possible when testing. This is for a few reasons:

  1. If we need to debug we can enter a breakpoint anywhere. gen_cluster sets up everything to run in the main thread
  2. It's much faster than the client fixture, which while more comfortable because it uses the synchronous syntax, also sets up some processes
  3. It will force you to learn some async things if you don't know them already

https://distributed.dask.org/en/latest/develop.html#writing-tests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I modified the test to use gen_cluster and captured_logger, thanks for the suggestion.

I needed to trigger an action on the client after the delete to ensure the error was logged. I used c.close for this, was curious is there is a better way to do this.

distributed/variable.py Outdated Show resolved Hide resolved
@mrocklin
Copy link
Member

mrocklin commented Apr 5, 2020

Friendly ping @jjhelmus

Capture the log in the test_delete_unset_variable test to check that
no KeyError was emitted.
Copy link
Member

@mrocklin mrocklin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great. I'm happy to merge. Alternatively you might want to accept the suggestions below on async/await.

distributed/tests/test_variable.py Outdated Show resolved Hide resolved
distributed/tests/test_variable.py Outdated Show resolved Hide resolved
jjhelmus and others added 2 commits April 10, 2020 16:24
Co-Authored-By: Matthew Rocklin <mrocklin@gmail.com>
Co-Authored-By: Matthew Rocklin <mrocklin@gmail.com>
@mrocklin
Copy link
Member

Ah, it looks like the delete method isn't asynchronous. It just fires of a message and returns immediately.

Hrm, that kind of mismatch makes using these things a bit rough. Probably not something to resolve now, but something to keep in mind.

@mrocklin mrocklin merged commit 5495284 into dask:master Apr 14, 2020
@mrocklin
Copy link
Member

Thanks @jjhelmus

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Variable delete raises KeyError
2 participants