"Meaningful normal value"

[phoe]

In many contexts, nil is a special value meaning "no value" or similar. In many of those contexts, nil would not be a meaningful normal value anyway (or both meanings coincide), so there are no difficulties.

Mechanisms such as gethash already provide support for that behaviour, in form of a secondary return value that states whether the value was found (including a NIL that was found in the hashtable) or whether the value was not found.

What issue does your library solve that isn't already solved by the (much more standardized and popularized, I assume) mechanisms of returning an additional value for whether the object was found or not?

[Hexstream]

What the fuck? The documentation (which you quoted a part of) already addresses all of your points. I expected better from you.

[phoe]

The documentation states, "Reserving nil as the special value is the right call in most contexts" without stating why exactly it is "right". I claim otherwise, that having a deliberate value marker for a function that can return any value does not solve any problem that multiple values do not solve; also, multiple return values already have high adoption as a value-returning idiom in the Lisp world (e.g. assoc-value and rassoc-value from Alexandria).

Moreover, using a value to denote a lack of value is not sound. Reserving another value, let's say, nil* as a stand-in for a meaningful "lack of value" simply moves the famous (find nil '(1 2 nil 3 4)) problem elsewhere. If we declare that nil* is the lack of value instead of nil, then, instead of being unable to represent the value nil, we are now unable to represent the value nil*. This might cause issues for example for functions which use fakenil and then iterate over all symbols; they may then interpret fakenil:nil* as a lack of value instead.

[Hexstream]

The documentation states, "Reserving nil as the special value is the right call in most contexts" without stating why exactly it is "right".

That's right, I am relying on my 13+ years of Common Lisp experience, here. (I recognize that the documentation for this project has not yet reached a platonic ideal, but I have much more important projects to work on.)

One of my favorite design decisions in definitions-systems is using the simpler API for defsys:locate (returning nil to indicate no definition found) instead of cargo-culting gethash. The reality is that almost nobody would ever need to use nil as a definition, and if someone somehow does need to, then fakenil would probably be a suitable solution in most of those rare contexts. If that's still not good enough for that highly sophisticated user, then they can trivially implement their own alternative to defsys:locate which does behave like gethash (and can fallback to defsys:locate by default with a default method). defsys:locate is also a frequently used operation (relative to all other operations supported by definitions-systems), and I would hate to take a small performance penalty just for the sake of gratuitous inconvenience.

Another library I'm working on, and which prompted me to finally create fakenil after years of always encountering the same damn problem, is a trivial protocol for arriving at a "guess" by consulting some "advisors", some of which may not have a particular preference in the matter. Here again, after initially considering to cargo-cult gethash, I decided to not support nil directly as a "normal value" (so an advisor returns nil if they don't know or don't care, not to choose nil as their preferred value). Instead of leaving open the issue of how to represent nil as a normal value or documenting the design decision, both of which are painful in some ways, I decided to reify and document the pattern once and for all with fakenil.

I claim otherwise, that having a deliberate value marker for a function that can return any value does not solve any problem that multiple values do not solve

Assuming that users indeed almost never need to represent nil as a "normal value" in that particular context, then using the single-value approach makes the API slightly simpler to use for users, and much simpler for implementors when you're designing an extensible protocol. While user code can normally trivially ignore the second returned value if it doesn't care about it, all implementors of extensions to your protocol do need to deal with that second return value explicitly, which can be quite a hassle. For instance, in my trivial consensus protocol one can just use some to return the guess of the first advisor who has an opinion on the matter. Having to deal with a second return value for each guess would be annoying.

multiple return values already have high adoption as a value-returning idiom in the Lisp world

No shit? I've been saying that multiple values is an underrated feature since at least 2012 (quote: "probably Common Lisp's most underrated feature, by the way"), and some of my libraries use multiple values in exposed parts of the API, such as its, parse-number-range and cartesian-product-switch, and I have some libraries entirely dedicated to the concept of multiple values such as enhanced-multiple-value-bind and multiple-value-variants. Needless to say, my code also heavily uses multiple values internally, but I don't care to dig examples of that...

(e.g. assoc-value and rassoc-value from Alexandria)

I extensively don't care about Alexandria, I haven't used it in probably at least a decade. I don't dispute that it's a fairly well-designed "personal utilities library", but I've come to hate that very concept, and from my personal perspective Alexandria is already mostly obsolete, and with a few of the libraries I'll be releasing this year I think I should be able to build a pretty good case to announce that Alexandria is "officially" obsolete. I think it's useful to remember that Alexandria is so damn popular in large part because of the official Quicklisp download statistics which count both direct and indirect downloads. I think it would be very interesting to see how libraries stack up if only direct downloads are counted...

Moreover, using a value to denote a lack of value is not sound.

It is extremely practical and entirely or mostly sound in most contexts, and sometimes you just don't have a choice to conform to existing API's. Note that using a distinguished value to represent "no value" is a classic technique notably to implement unbound slots, although fakenil would certainly not be an appropriate value to use in that context. fakenil is not an approach that makes sense in all contexts, but then all approaches to all problems don't make sense in all contexts. My recommendation would be to use fakenil in contexts where it makes sense, and not use it in contexts where it doesn't make sense.

the famous (find nil '(1 2 nil 3 4)) problem

What the fuck? Using member is the classical, obvious solution. I know it's just an example, but it's particularly ridiculous.

Moreover, using a value to denote a lack of value is not sound. Reserving another value, let's say, nil* as a stand-in for a meaningful "lack of value" simply moves the famous (find nil '(1 2 nil 3 4)) problem elsewhere. If we declare that nil* is the lack of value instead of nil, then, instead of being unable to represent the value nil, we are now unable to represent the value nil*. This might cause issues for example for functions which use fakenil and then iterate over all symbols; they may then interpret fakenil:nil* as a lack of value instead.

I know all this, aren't you happy that we now have an obvious place where to document all these caveats? By reifying this extremely common pattern, we are better able to reason about its strengths and shortcomings and advise others about them.

[equwal]

My view is that it is better to use either 1) a non-local exit, possibly by signaling a condition or 2) using a value that is outside the logical codomain of the function to signal. Using nil* just creates another arbitrary symbol to worry about in all contexts. In addition, your package seems to exist for no real purpose, and to be all boilerplate except for the one line where nil* is declared a constant.

I'm no longer going to put in effort on this package. I suggest nobody else does either.

[Hexstream]

My view is that it is better to use either 1) a non-local exit, possibly by signaling a condition

Of course that's going to be the right solution in some contexts, but in others you would just be complicating the API for everyone (users, implementors, extension designers) for no reason.

or 2) using a value that is outside the logical codomain of the function to signal.

What the fuck? Isn't that what fakenil already provides? Of course, in some contexts you would indeed need to use a truly unique value such as a gensym or something, in which case, don't use fakenil if it's not suitable.

Using nil* just creates another arbitrary symbol to worry about in all contexts.

Actually, a big part of the point of fakenil is that most code will be able to pass along nil* without having to know or care about it, it can just be returned somewhere and then go through some oblivious machinery and then at the other end something can finally care to distinguish it as a special value and do something appropriate about it such as treating it as a "normal value" of nil.

In addition, your package seems to exist for no real purpose, and to be all boilerplate except for the one line where nil* is declared a constant.

Your nearly information-free commentary also seems to exist for no real purpose. (And I'm a nihilist.) The target market for fakenil is those who understand why it might be useful, or otherwise derive some value from it, not those who are convinced it is not useful.

Most of the value of this library is obviously in the documentation, not the defconstant. And now, we can track all instances of the use of this pattern when people care to take the time to use this library. (Which granted, may not always happen, but at least it's an option now.)

I'm no longer going to put in effort on this package. I suggest nobody else does either.

You are an obnoxious child. I invite you to reflect on the unnecessary costs your impulsivity may inflict on yourself and others.

[phoe]

I know it's just an example, but it's particularly ridiculous.

It's an issue that has been long recognized as a mistake in the Common Lisp standard. A secondary value of t returned from (find nil '(1 2 nil 3 4)) would make it obvious that a nil was found in the list, and therefore find would be usable with a nil as its first argument.

[Hexstream]

It's an issue that has been long recognized as a mistake in the Common Lisp standard.

By whom, when, where and why? This is the first time I hear of this, and I am not convinced at all.

A secondary value of t returned from (find nil '(1 2 nil 3 4)) would make it obvious that a nil was found in the list, and therefore find would be usable with a nil as its first argument.

So what? I almost never want to find nil in a list anyway, and member provides a complete solution for the rare cases that I do, and member has other useful use-cases that justify its existence as well, so it's not like if it was there just to compensate for a design flaw in find.

I also wonder if your proposed design might sometimes incur a certain performance penalty, which I would not be eager to pay for, and I'm saying that as someone who mostly doesn't care about performance. Again, I just don't want to make my code slower just for the sake of gratuitous inconvenience...

I'm open to being convinced, but you would have to provide more compelling arguments.

[Hexstream]

This issue is resolved to my satisfaction, so I'm going to close this.

Obviously, a cleaned up version of some of this content should be added to the documentation at some point, but this is a very low priority for me, as I have much more important projects to work on.

Those with further meaningful commentary (as demonstrated by phoe) are encouraged to append to this issue.