Prebuttal for Standardizing observer_ptr
Document number: D1495R0
Reply-to: Tony Van Eerd. cadged at forecode.com
In P1408, Bjarne wrote a rebuttal against standardizing
observer_ptr, but no one had yet written a proposal to standardize it. Thus the need for this "prebuttal":
observer_ptr (from Library Fundamentals 2 TS) should be standardized (but with a better name and better conversions).
Reasons to standardize
Reasons to standardize
observer_ptr (with more in-depth explanations to follow).
observer_ptris the safe common type for
shared_ptr, other smart pointers, and
observer_ptris a safer alternative to
observer_ptrextends the type safety of
unique_ptr(and other smart pointers)
observer_ptrmakes intent more clear
observer_ptrhelps prevent overuse of
shared_ptras a function param
observer_ptris a post-modern tool for transitioning a codebase to more modern C++.
Personally, I come across these uses weekly. Note that I listed the oft-sited "transitioning a codebase" as sixth in order of importance (IMO).
Details/Explanations of each:
observer_ptr is the safe common type for
shared_ptr, other smart pointers, and
observer_ptr for smart and raw pointers, is like
char const * and
ie a common base type that any pointer can be temporarily safely converted to.
shared_ptr sp = ...; unique_ptr up = ...; Foo * rp = ...; void f(Foo * pf); f(sp.get()); f(up.get()); f(rp);
shared_ptr sp = ...; unique_ptr up = ...; Foo * rp = ...; void f(observer_ptr<Foo> pf); f(observer_ptr(sp.get())); f(observer_ptr(up.get())); f(observer_ptr(rp));
shared_ptr sp = ...; unique_ptr up = ...; Foo * rp = ...; void f(observer_ptr<Foo> pf); f(sp); f(up); f(rp);
- The leftmost version has calls to
get()which require extra scrutiny in a code review (as you are removing a safety latch)
- We will likely never allow smart pointers to implicitly convert to raw pointers
- The leftmost function
f(taking a raw pointer) also requires extra scrutiny.
- The middle version is just noisy - for no good reason - the noise does not protect anything dangerous.
- The rightmost version is safe -
observer_ptrdoesn't change ownership semantics. Nothing extra should be required.
- In terms of P0705 conversion guidelines, an implicit conversion to
observer_ptris safe and sensible.
T* the common type?
You don't want smart pointers to implicitly convert to raw pointers
(as this can too easily lead to accidental misownership), but converting to
observer_ptr does not increase the risk of misownership.
It does increase risk of dangling - same as string_view does. Thus, similar to
observer_ptr is best used as a function param.
observer_ptr is a safer alternative to
As mentioned in recent emails on the reflector,
T* has some downsides:
[i]even when it shouldn't
T*allows derived/base conversions (and then more
--with the wrong size)
<is not a total order
T*opens up questions about ownership (in many codebases)
T*allows conversion to
observer_ptr extends the type safety of other smart pointers
A smart pointer, when used correctly, ensures safe lifetime management. The only (well almost only) way to break the safety of a smart pointer, is improper use of
get() exposes the underlying pointer, it exposes the responsibility of the smart pointer's invariants, breaking the "air-tight seal" of the smart pointer.
get() and allows the invariant to stay protected throughout more code. A function that previously used a raw pointer (so as to be used by both
unique_ptr clients), can now avoid
get() completely. This means
get() can be pushed to the edge of your codebase - only needed at the border with 3rd party or unchangeable APIs that require other pointer types.
observer_ptr extends the safety net of a smart pointer over more of your codebase.
observer_ptr makes intent more clear
From the original proposal (N4282) "it is intended as a near drop-in replacement for raw pointer types, with the advantage that, as a vocabulary type, it indicates its intended use without need for detailed analysis by code readers".
observer_ptr makes code more clear, easier to read, alleviating nagging lifetime questions. Some codebases can use
T* to mean non-owning, but many popular 3rd party libraries still use raw pointers with various meanings, thus adding ambiguity to even modern codebases.
observer_ptr helps prevent overuse of
shared_ptr as a function param.
It is very common, in a codebase using
shared_ptr (for sharing, hopefully), to pass
shared_ptr as a function param. Basically "I have a shared_ptr to Foo. I need to write a function that uses it. I'll pass it into the function:"
int count(shared_ptr<Foo> foo);
There are 2 problems with this:
- it does not show intent - there is no intent to share ownership (beyond the length of the call). ie vs a function like
setFoo(shared_ptr<Foo> foo)which probably does intend to share ownership
- it involves unnecessary atomic increment/decrement - Photoshop, for example, changed all functions like this to instead take
shared_ptr<Foo> const & foo, to avoid the atomic ops. Effective, but convoluted for a pointer.
observer_ptr<Foo> is a drop-in replacement for these functions. (
Foo * is a replacement that has other issues as mentioned elsewhere, and also requires updating all call-sites, whereas
observer_ptr does not require changes to callsites.)
I feel it will be easier to teach/train/convince/remind developers to use
observer_ptr here than
T * as
observer_ptr is basically "made to order" for this usecase, and doesn't require extra thought.
observer_ptr is a post-modern tool for transitioning a codebase to more modern C++.
This is the reasoning most often discussed. Note that it is listed fifth. The idea is to review all raw pointers in a codebase, replacing each case with the correct smart pointer (ie
unique_ptr hopefully, else
Since this replacement takes time, and not all cases will be fixed at once, "unowned" pointers need a thing to be replaced with (ie
observer_ptr) else it is hard to know which raw pointers have been reviewed, and which haven't.
(If all owned pointers in a codebase are wrapped with smart pointers,
then a raw pointer can mean "unowned". But most codebases are still mixed with new and old uses.)
Rebuttal rebuttal: There was concern about the potential proliferation of smart pointers in the standard. Although there are a few other potential smart pointers not (yet) in the standard (ie an intrustive_ptr, see P0468), smart pointers have been around for about 20 years, and there really isn't that much variation. Boost has shared_ptr and intrusive_ptr, many codebases have something like observer_ptr. There is also inout_ptr (P1132), which can be found in many codebases (and is not really a smart pointer, but more of a helper, working alongside smart pointers), and also a clone_ptr or value_ptr, now called polymorphic_value (P0201). But the list of common, widely used (and thus candidates for standardization) smart pointers is finite and short. Ironically, the more we have, the more useful
observer_ptr becomes as a common type that works seemlessly with all of them.
observer_ptr, we should make a few small changes (more in-depth below)
- Allow implicit conversions from smart and raw pointers
release()(as it does not transfer ownership)
1. Allow implicit conversions from smart and raw pointers
The original proposal did not include implicit conversions. Most coding guidelines now favour
explicit constructors - when in doubt; however, in the case of
observer_ptr, there is no danger in an implicit conversion, only benefit.
See the table earlier in this paper - implicit conversion is required to allow
f(observer_ptr<T>) to take smart pointers and raw pointers without calls to
get(). See P0705 for a more complete explanation as to when implicit conversion is acceptable.
observer_ptr checks all the right boxes. In particular:
- a smart/raw pointer and an
observer_ptrboth represent the same "platonic" thing (or
observer_ptris a strict subset of a pointer, since it offers a subset of functionality). Thus conversion (of some form) is worth considering.
- the conversion is safe. The
observer_ptrwon't delete the pointer, etc. For a smart-pointer, conversion to
observer_ptrdoes not break the smart-pointer's invariants. (whereas
get()on a smart-pointer does break (or expose for breakage) a smart pointer's invariants). There is a slight concern with dangling (the
observer_ptrdoesn't extend the lifetime of the pointer), but this is the exact same level of concern (and same risk/reward) as with
This change has been implemented a few implementations without issue.
release() (as it does not transfer ownership)
unique_ptr has a
release() function, which transfers ownership. Note that
shared_ptr does NOT have a
release function (as you don't gain exclusive ownership from the shared pointer).
observer_ptr::release() does not transfer ownership. It should be consistent with STL
shared_ptr, and not have this function.
In any programming language, it is important that Different semantics require different names but it is particluarly important in a language with templates that use compile-time duck-typing. If
release() is called in a template, the expectation would be ownership transfer.
release can be renamed
detach or just removed - the user can call
(P.S. same with
retain_ptr::release() (P0468), although its semantics are even subtler - the reference count lives with the raw pointer, not the smart pointer, and only responsibility is being transferred.)
observer_ptr is a bad name. It is bad because "observer" already has common meaning in programming (ie "the observer pattern" https://en.wikipedia.org/wiki/Observer_pattern).
observer_ptr is thus a great name to use throughout this paper, as it is a only a placeholder, not a suggestion, and thus doesn't bias to any of the other good names to follow.
There are some criteria to use when considering naming:
- Not understanding is better than MISunderstanding. (“It is better to know nothing than to know what ain’t so.” - Josh Billings, 1874)
- Coining a term is OK - it will forever have that meaning (ie "observer" means observer pattern). The hard part is finding a good term not already used - an arbitrary term (like iota) is not good "coinage" (but it does show how coining a term - attaching meaning - works). A coined term should at least contain a hint that our brains can cling to.
- Avoid negatives, as these quickly lead to double negatives in code (ie
- Avoid spoken ambiguity. ie
raw_ptrvs "raw pointer".
A list of names
Mostly the following names can be grouped into a few piles:
- ONWERSHIP: smart pointers tend to be about ownership. This one is lack of ownership. But the pointer is still owned (hopefully!), just not by you. So
unowned_ptr, for example, is not correct (as someone owns it);
notmy_ptris more correct.
- USAGE: Instead of defined-by-contrast (ie vs other smart pointers), we could focus on how it is meant to be used - it is best used as a param (like string_view) and is only temporary. Thus names like temp/brief/transient/sojourn/... It is also meant to grant access to an object, no-more-no-less, thus
- WHAT: We can define a class by what it is and what it offers, and let users decide how to use it.
- COINAGE: Picking a word that is currently unused, and give it meaning in the programming context. But it should at least hint at meaning.
|notmy_ptr||intent||cheeky, double negative|
|1||alias_ptr||aliases a ptr out there somewhere|
|1||cadged_ptr||very correct, coins a term||not well known|
|borrowed_ptr||but how do you give it back?|
|dang_ptr||dangling/danger, coins a term||cheeky|
|trust_ptr||I trust it will live long enough|
|exempt_ptr||ownership, obviously||exempt from what?|
|1||access_ptr||grants access, no more no less|
|brief_ptr||i before e|
|param_ptr||strongly suggest safe usage||unenforceable|
|1||object_ptr||Anthony Williams library|
|basic_ptr||basic_string?||captures functionality, but not intent|
|common_ptr||functionality, not intent|
|view_ptr||a pointer to a view?|
|ptr_view||like string_view||doesn't end in ptr?|
|naive_ptr||gives fair warning|
|1||klein_ptr||https://en.wikipedia.org/wiki/Minimalism#/media/File:IKB_191.jpg||not Klein bottle :-(|
|bum/freeload/mooch||(this is what 'cadge' means)||slang|
|see below||many creative names from original paper coin a term|
From the original paper (N3740)
Very Scientific poll
|121||pointy_mcpointface||because it's reddit|
|100||access_ptr||it grants access, not ownership|
|57||observer_ptr||don't care if it's misleading|
|40||unowned_ptr||a non-owning "smart" pointer|
|37||pointer or ptr||the word already describes it|
|37||use_ptr||you can use the object, but not destroy it|
|24||borrowed_ptr||its borrows something someone else owns|
|21||view_ptr||something to view an object|
- Anthony Williams - https://www.justsoftwaresolutions.co.uk/cplusplus/object_ptr.html - includes implicit conversions, removed
- Martin Moene - https://github.com/martinmoene/observer-ptr-lite
- (IIUC) Ville has tried these changes against libstdc++ and its testsuite
Thank you Walter for the original proposal. Thanks to Ville, Anthony, Martin, and others for their encouragement and implementation experience.
N4282 - A Proposal for the World’s Dumbest Smart Pointer, v4 - Walter E. Brown
P1408 - Abandon observer_ptr - Bjarne Stroustrup
P0705 - Implicit and Explicit Conversions - Tony Van Eerd
P0468 - An Intrusive Smart Pointer - Isabella Muerte
P1132 - out_ptr - a scalable output pointer abstraction - JeanHeyd Meneide, Todor Buyukliev, Isabella Muerte
P0201 - A polymorphic value-type for C++ - Jonathan Coe, Sean Parent