Add a "no-op" (null) context manager to contextlib #54258
Comments
|
I find that I frequently need the "null" (no-op) context manager. For example, in code such as: with transaction or contextlib.null():
...Since there is no easy expression to create a null context manager, we must resort to workarounds, such as: if transaction:
with transaction:
... code ...
else:
... duplicated code ...(with the usual options of moving the duplicated code to a function—but still.) Or by creating ad-hoc null context managers with the help of contextlib.contextmanager: if transaction is None:
transaction = contextlib.contextmanager(lambda: iter([None])()
with transaction:
...Adding a "null" context manager would be both practical and elegant. I have attached a patch for contextlib.py |
hniksic
mannequin
added
stdlib
vstinner
changed the title
|
+1 Looks like a reasonable use case. |
|
Patch is missing tests and documentation. |
|
About your patch:
|
|
I also find the Null/_null/null affair confusing. |
@contextlib.contextmanager
def null():
yieldDo we really need to add this to the stdlib? |
|
Thank you for your comments. @michael: I will of course write tests and documentation if there is indication that the feature will be accepted for stdlib. @antoine: it is true that a null context manager can be easily defined, but it does requires a separate generator definition, often repeated in different modules. This is markedly less elegant than just using contextlib.null() in an expression. I'm not acquainted with the history of identity function requests, but note that the identity function can be defined as an expression, using simply lambda x: x. The equivalent expression that evaluates to a null context manager is markedly more convoluted, as shown in my report. @Éric: The Null/_null/null distinction is an optimization that avoids creating new objects for something that is effectively a singleton. It would be perfectly reasonable to define contextlib.null as Antoine did, but, this being stdlib, I wanted the implementation to be as efficient as (reasonably) possible. |
But you can use mymodule.null() where mymodule is a module gathering |
|
The difference here is the one pointed out in the original post: for a function, you usually only care about having a value, so if you don't want to call it, you can just swap in a None value instead. If you need an actual callable, then "lambda:None" fits the bill. The with statement isn't quite so forgiving. You need a genuine context manager in order to preserve the correct structure in the calling code. It isn't intuitively obvious how to do that easily. While not every 3-line function needs to be in the standard library, sometimes they're worth including to aid discoverability as much as anything else. However, I don't see the point in making it a singleton and the name should include the word "context" so it doesn't becoming ambiguous when referenced without the module name (there's a reason we went with contextlib.contextmanager over contextlib.manager). Something like: class nullcontext():
"""No-op context manager, executes block without doing any additional processing. with optional_cm or nullcontext():
# Perform operation, using the specified CM if one is given
"""
def __enter__():
pass
def __exit__(*exc_info):
pass |
|
That is what we are using now, but I think a contextlib.null() would be useful to others, i.e. that its use is a useful idiom to adopt. Specifically I would like to discourage the "duplicated code" idiom from the report, which I've seen all too often. The "closing" constructor is also trivial to define, but it's there for convenience and to promote the use of with statement over try/finally boilerplate. The same goes here: you don't miss the null context manager when you don't have it; you invent other solutions. But when it's already available, it's an elegant pattern. In my experience, if they have to define it to get it, most people won't bother with the pattern and will retain less elegant solutions. |
|
Actually, the singleton idea isn't a bad one, but I'd go one step further and skip the factory function as well. So change that to be: class NullContext():
... # as per nullcontext in my last message
nullcontext = NullContext()(with the example in the docstring adjusted accordingly) |
|
If you can supply a full patch before the end of the month, we should be able to get this in for 3.2beta1 (currently scheduled for 31 October) |
|
I considered using a variable, but I went with the factory function for two reasons: consistency with the rest of contextlib, and equivalence to the contextmanager-based implementation. Another reason is that it leaves the option of adding optional parameters at a later point. I know, optional parameters aren't likely for a "null" constructor, but still... it somehow didn't feel right to relinquish control. |
|
Here is a more complete patch that includes input from Nick, as well as the patch to test_contextlib.py and the documentation. For now I've retained the function-returning-singleton approach for consistency and future extensibility. |
|
Is there anything else I need to do to have the patch reviewed and applied? I am in no hurry since we're still using 2.x, I'd just like to know if more needs to be done on my part to move the issue forward. My last Python patch was accepted quite some years ago, so I'm not closely familiar with the current approval process. |
|
Unless Nick has further feedback I think you've done all you need to, thanks. |
|
I'm with Antoine. Why not just do this in the context function itself? from contextlib import contextmanager
CONDITION = False
@contextmanager
def transaction():
if not CONDITION:
yield None
else:
yield ...
with transaction() as x:
... |
|
Because hardcoding a particular condition into a context manager is less flexible? (I'm +0 on this thing myself, by the way.) |
|
Are you sure that this is useful enough to warrant inclusion in the standard lib? I don't know of anyone else who has used the same idiom. It seems crufty to me -- something that adds weight (mental burden and maintenance effort) without adding much value. I don't know that anyone actually needs this. |
|
To me, this is more a matter of conceptual completeness than one of practical utility (ala fractions.Fraction). That said, I *have* personally encountered the "I only sometimes want to wrap this code in a CM" situation, so it isn't completely impractical, either. Those two factors are enough to reach my threshold for it being worthwhile to declare "one obvious way to do it" through the contextlib module. There is a possible alternative approach that may be more intuitive to use and read than nullcontext() though: @contextmanager
def optional_cm(cm, *, use_cm=True): # See naming note below
if cm is None or not use_cm:
yield
else:
with cm:
yieldThe OP's original example would then look like: with optional_cm(transaction):
...I suspect readers would find it far easier to remember what optional_cm does than to learn to recognise the "or nullcontext()" idiom. It also plays better with nested context managers: with optional_cm(sync_lock), optional_cm(db_transaction), \
open(fname) as f:
...Naming Note: I nearly suggested "optional_context" as a name for this, but realised that would be subtly misleading (suggesting PEP-377 style functionality that potentially skipped the statement body, rather than the intended semantics of skipping use of the CM) |
|
I like your latest suggestion, except for the name. Given that we also have the (quite generic) "closing", what about just "optional"? |
Nick, you don't seem to be truly sold on the need. We need to focus on real problems in the standard |
|
I find Raymond's perspective persuasive in this case. Feel free to post either the original idea or my later variant as an ASPN cookbook recipe. (you could actually combine the two, and use NullContext as an implementation detail of an optional_cm() function) |
|
Not having this as a standard idiom makes it very tempting to just do copy-paste coding as in hniksic's example. Who likes to invent their own library for generic language-supporting idioms? What about an alternative of giving NoneType empty enter and exit methods? So instead of a 'null' CM you can just use "with None"? |
|
FWIW, it's likely I'll be adding contextlib.ContextStack (see [1]) for 3.3. While it's far from the primary use case, that API also serves as a "no-op" context manager (if you never register any contexts or callbacks, the __exit__ impl does nothing). [1] http://contextlib2.readthedocs.org/en/latest/index.html#contextlib2.ContextStack |
|
That's very reassuring. Thanks, Nick! |
|
After seeing a context manager named like "TempfileIfNeeded(..., cond)", whole sole purpose is to handle the conditional case, I'm firmly +1 on this proposal. It's much easier to just read "with Tempfile() if cond else nullcontext():" than to read through another level of indirection every time someone wanted some conditional logic on a context manager. Is there any chance that this issue could be reopened? Perhaps a more elegant solution would be to modify the "with" statement so that any object can be given to it (then we could just use None directly), but I suspect that would be a tad more controversial. ;) |
|
No, an empty ExitStack() instance already works fine as a no-op context manager in 3.3: http://docs.python.org/3/library/contextlib#simplifying-support-for-single-optional-context-managers We're not going to add a dedicated one under a different name. |
|
I've been looking for this today; I would have used it. Instead of an obvious (and explicit) null context manager, I had to read through this entire thread to eventually find out that I can use something called ExitStack(), which is designed for another use case. When many users have to replicate the same boilerplate code time and time again, it's not cruft, it's just something that ought to be part of the stdlib. There are a number of such cases in the stdlib. I think nullcontext should be part of the included batteries Python aims to provide. |
|
ExitStack() already covers the "null ctx mgr" use case described in the first message. Original example: with transaction or contextlib.null():
...By using ExitStack: with transaction or ExitStack():
...You can push this further and do this, which is even more flexible: with ExitStack() as stack:
if condition:
stack.enter_context(transaction)
...So ExitStack really is better than the original proposal which could have made sense 6 years ago but not anymore. |
|
The problem Martin is referring to is the SEO one, which is that the top link when searching for either "null context manager python" or "no-op context manager python" is this thread, rather than the "Use ExitStack for that" recipe in the docs: https://docs.python.org/3/library/contextlib.html#simplifying-support-for-single-optional-context-managers We unfortunately have exactly zero SEO experts working on the CPython documentation, so even when we provide specific recipes in the docs for solving particular problems, they aren't always easy for people to find. I've at least added the "use contextlib.ExitStack()" note to the issue title here, so folks can find that without having to read through the whole comment thread. |
ncoghlan
changed the title
|
Adding nullcontext = ExitStack in the source file would solve this problem in a single line of code. |
|
No, it wouldn't, as ExitStack() does far more than merely implement a null context. It would be like adding "nulliterable = ()" as a builtin, rather than just telling people "If you need a null iterable, use an empty tuple". |
|
Well that just echoes exactly what I originally thought, but somebody else said it was not needed because ExitStack already exists and could be used for that purpose. If this were at work and/or it were all just to me, I'd just implement a brand new nullcontext and move on. |
|
Unfortunately, the redundancy doesn't buy enough to justify the permanent documentation and style guide cost of providing two ways to do exactly the same thing. |
|
It turns out that there's a variant on the "null context manager" idea that may *not* be redundant with ExitStack(), and hence could potentially counter the current rationale for not adding one. Specifically, it relates to context managers like click.progressbar() that are designed to be used with an "as" clause: with click.progressbar(iterable) as myiter:
for item in myiter:
...At the moment, making that optional is a bit messy, since you need to do something like: with click.progressbar(iterable) as myiter:
if not show_progress:
myiter = iterable # Don't use the special iterator
for item in myiter:
...or: with ExitStack() as stack:
if show_progress:
myiter = stack.enter_context(click.progressbar(iterable))
else:
myiter = iter(iterable)
for item in myiter:
...or: @contextmanager
def maybe_show_progress(iterable, show_progress)
if show_progress:
with click.progressbar(iterable) as myiter:
yield myiter
else:
yield iter(iterable)
with maybe_show_progress(iterable, show_progress) as myiter:
for item in myiter:
...The problem is that there's no easy way to say "return *this* value from __enter__, but otherwise don't do anything special". With a suitably defined NullContext, that last approach could instead look more like: if show_progress:
ctx = click.progressbar(iterable)
else:
ctx = NullContext(iter(iterable))
with ctx as myiter:
for item in myiter:
... |
|
Note that this indeed seems confusing. I just found this thread by search for a null context manager. Because I found that in TensorFlow they introduced _NullContextmanager in their code and I wondered that this is not provided by the Python stdlib. |
|
For what it's worth, we are still using our own null context manager function in critical code. We tend to avoid contextlib.ExitStack() for two reasons:
It doesn't surprise me that projects like Tensor Flow introduce their own versions of this decorator. Having said that, I can also understand why it wasn't added. It is certainly possible to live without it, and ExitStack() is a more than acceptable replacement for casual use. |
|
Reopening this based on several years of additional experience with context managers since I wrote https://bugs.python.org/issue10049#msg119514 when originally closing it. The version I'm now interested in adding is the one from https://bugs.python.org/issue10049#msg281556 - rather than being completely without behaviour, the null context manager should accept the value to be returned from the call to __enter__ as an optional constructor parameter (defaulting to None). That allows even context managers that return a value from __enter__ to be made optional in a relatively obvious way that doesn't involve fundamentally rearranging the code. I think the overhead argument against the use of ExitStack() for this purpose also has merit (so I'd be curious to see relative performance numbers collected with perf), but it's not my main motive for changing my mind. |
ncoghlan
changed the title
|
Reverting to "Needs patch", as the currently attached patch is for the "No behaviour" variant that always returns None from __enter__(). (hniksic, would you still be willing to sign the Python CLA? If so, then your patch could be used as the basis for an updated implementation. Otherwise I'd advise anyone working on this to start from scratch) |
|
I am of course willing to sign the CLA (please send further instructions via email), although I don't know how useful my original patch is, given that it caches the null context manager. |
|
Thanks to Jesse Bakker for the PR implementing this for 3.7! |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: