New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
A few changes to config.set
#5226
Conversation
The main intent of this PR is to support setting config values through keyword parameters instead of requiring a dictionary for nested config values. We follow the same convention as we do for environment variables, where double-underscores are interpreted as a ``.``, and indicate an increase in nesting level. Now the following are equivalent: ```python with dask.config.set({"foo.bar.baz": 1}): ... with dask.config.set(foo__bar__baz=1): ... ``` If a mapping argument and keyword arguments are both provided, the mapping is applied first, then the keyword arguments (the keywords are effectively overrides for the mapping). In the process, I discovered a bug in the rollback code when dealing with multiple keys interacting with the same nested value. To deal with this I rewrote the rollback code in a way that was simpler for me to understand, and hopefully others as well.
cc @djhoese, this may interest you. |
Failure is unrelated, and may be fixed by #5228. |
Thanks for the mention. This makes sense to me from a higher level. I'm not super excited about having to mirror the rollback changes to donfig, but that's my fault for not finishing my PRs getting dask on an object oriented config object. |
You mentioned a bug in the rollback, does your new test cover this case? |
Yeah, the issue had to do with multiple keys in a |
@mrocklin, quick review? Planning to merge this later today otherwise. |
Can I trade you for reviewing dask/distributed#2913 ? |
The support of __
looks good to me. The rewrite of writing and unrolling config I understand less, but it all seems reasonable. I'm +1
The main intent of this PR is to support setting config values through
keyword parameters instead of requiring a dictionary for nested config
values. We follow the same convention as we do for environment
variables, where double-underscores are interpreted as a
.
, andindicate an increase in nesting level. Now the following are equivalent:
If a mapping argument and keyword arguments are both provided, the
mapping is applied first, then the keyword arguments (the keywords are
effectively overrides for the mapping).
In the process, I discovered a bug in the rollback code when dealing
with multiple keys interacting with the same nested value. To deal with
this I rewrote the rollback code in a way that was simpler for me to
understand, and hopefully others as well.
black dask
/flake8 dask