Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0, [], {}, (), and "" should not be falsey #373

Closed
noprompt opened this Issue Dec 14, 2013 · 31 comments

Comments

Projects
None yet
@noprompt
Copy link

noprompt commented Dec 14, 2013

I know hy is targeting Python and all but that doesn't mean it has to bring the historical notion of falseyness along with it. You know what I'm saying? This is the kind of lame crap I'm talking about:

>>> 'yay' if [1].index(1) else 'boo'
'boo'
>>> 'yay' if [1, 0][1] else 'boo'
'boo'

It turns out 0 is a useful value. Even Ruby gets this right. Do the right thing here. 😄

@noprompt

This comment has been minimized.

Copy link
Author

noprompt commented Dec 14, 2013

For context:

hy

hy 0.9.11
=> (if 0 1 2)
2

Other lisps

Emacs lisp

ELISP> (if 0 1 2)
1

Clojure

user> (if 0 1 2)
1

Common lisp

* (if 0 1 2)
1

Racket

-> (if 0 1 2)
1
@Willyfrog

This comment has been minimized.

Copy link
Contributor

Willyfrog commented Dec 14, 2013

It isn't targeting Python, but interoperability with it. Changing the falsy evaluation would render the whole ecosystem of Python libraries unusable since people expects consistency.

Do you have any idea how to implement it without breaking interoperability?

I agree with you: it makes sense in a Lisp to have 0 as truthy, but I believe Hy does need to be consistent with the Python way.

@paultag

This comment has been minimized.

Copy link
Member

paultag commented Dec 14, 2013

Yeah, this guy is going to be a patches welcome situation - I don't know about a few core things:

  • Will if no longer compile to ast.If, if PyObject_IsTrue isn't DTRT. What will we use? hy.core.lang.if ? I don't want to get back into function-call-a-paloza, double so when our LoL is broken because of scoping in Python
  • What happens if you give a Truethey 0 to Python? Should something I think is True be evaluated to False by Python?

Anyway. I'm going to close this because it's really hard, and gives us only a slight gain. If you come up with a brilliant way, I'd love to consider it.

❤️

@paultag paultag closed this Dec 14, 2013

@noprompt

This comment has been minimized.

Copy link
Author

noprompt commented Dec 14, 2013

@paultag You should not be so quick to close this issue. This is actually a real problem beyond the silly examples I shared above. It will add an extra layer of complexity and a source of subtle bugs in places where list processing is important. For example: macros.

Changing the falsy evaluation would render the whole ecosystem of Python libraries unusable since people expects consistency.

Wat? No it wouldn't. It wouldn't render them useless anymore than Clojure renders Java libraries useless by the same logic.

Do you have any idea how to implement it without breaking interoperability?

I do. Read on.

What will we use? hy.core.lang.if ?

@paultag @Willyfrog Create a simple utility function, call it hy.core.lang.truth or something, that returns False if the value of the expression is False or None and True otherwise. Then when you emit if in the AST wrap the expression with a call to this function (ie. you emit hy.core.lang.truth(expr)). This is similar to what ClojureScript does when it compiles if. That should be fairly trivial given complexity of some of the other code.

What happens if you give a Truethey 0 to Python? Should something I think is True be evaluated to False by Python?

@paultag You wouldn't give Python a "Truthy 0", you would give it True or False which will be based on the result of calling hy.core.lang.truth as I've suggested above.

Have an ice cream. 🍨

@paultag

This comment has been minimized.

Copy link
Member

paultag commented Dec 15, 2013

I don't hate it entirely. I think an if macro may solve some of this, but I'm still not absolutely sure. I'm going to need more @hylang/core on this

@paultag paultag reopened this Dec 15, 2013

@tuturto

This comment has been minimized.

Copy link
Contributor

tuturto commented Dec 16, 2013

I'm a bit unsure of this. In case of if for example it doesn't sound like a bad idea (except it's different than what a python programmer would expect). What I'm more worried about is when Hy code is mixed with python code and calls are made to both directions. Doesn't a programmer have to remember which function has been implemented which function in order to know how they treat the truth value of 0?

@tuturto

This comment has been minimized.

Copy link
Contributor

tuturto commented Dec 17, 2013

To move this forward, would it be possible to show some code that shows how this would work under the hood? I'm not sure I understand everything about this and it would help to have some code that shows what happens when calls are made to various directions (hy to hy, hy to python, python to hy) and how truth would be treated in each case.

(if 0 (fn1) (fn2))
is translated to
(if (truth 0) (fn1) (fn2))

The above is what we are discussing, right?

@bitemyapp

This comment has been minimized.

Copy link

bitemyapp commented Dec 17, 2013

Is this a transpiler for s-expressions into Python bytecode or is it a Lisp?

If you stick with these terrible truth semantics you're doing Lisp a disservice.

@noprompt

This comment has been minimized.

Copy link
Author

noprompt commented Dec 17, 2013

What I'm more worried about is when Hy code is mixed with python code and calls are made to both directions.

@tuturto You're worried about the wrong thing.

Doesn't a programmer have to remember which function has been implemented which function in order to know how they treat the truth value of 0?

No! Why would that matter? If you tell Hy or Python code to do something, how truthiness is interpreted on either side should be irrelevant. If you're lying in bed awake at night because you're afraid of what will happen when Hy finds a 0 in the predicate position of an if expression and returns the result of the first branch, then you should see a therapist because it's really not that big of deal. Instead what you should be doing is rejoicing that you will not have to babysit a language who's designer did not think for more than a second about the consequences of 0 having false semantics and how great it will be when you do not have to worry about defending yourself against the hoards of bugs this one decision causes.

To move this forward, would it be possible to show some code that shows how this would work under the hood?

This code should be generated in the AST as I've suggested above.

@olasd

This comment has been minimized.

Copy link
Member

olasd commented Dec 17, 2013

My first reaction to this issue was "what the fuck is wrong with 0 being falsey, and why is everyone so obnoxious about it?". Statements like "you should see a therapist" or "you're doing X a disservice" are not helping you to get your point across. Please calm down.

Other lisps do it that way so it must make sense. I tried, but I can't find why. I would just like to know what are the expected false values and why some "python-false" values should be taken as true.

Implementation-wise, this is trivial: in hy/compiler.py, scroll down to @builds('if'), and change

cond = self.compile(expression.pop(0))

to

cond = expression.pop(0)
cond = HyExpression([HySymbol('is_truthy'), cond]).replace(cond)
cond = self.compile(cond)

Open hy/core/language.hy, implement the relevant (defn truthy? [value]). Done. No need to be obnoxious about it. Just show us why we should do this.

@khinsen

This comment has been minimized.

Copy link
Member

khinsen commented Dec 17, 2013

I am trying to catch up with this thread - and my first reaction is to agree with @olasd: please, everyone, keep this a pragmatic issue rather than an ideological one.

Next, some background considerations. True and False are relatively new additions to the Python language, and are in fact little more than syntactic sugar for 1 and 0. Even in Python 3, you can use False as a substitute for 0 everywhere, and True for 1. This is a feature of the Python bytecode interpreter, so we are not going to change it. There is also the issue of dealing with None, which tests as False in Python. Implementing is_truthy in a sensible way is probably a non-trivial job. Which means that getting this right requires a lot of practical testing. And that raises the question if it's really worth the effort.

On the Lisp side, there are already different approaches. Scheme and Common Lisp have different conventions, and newcomers like Clojure add other variants. I don't see any philosophical problem with introducing a Lisp that adopts Python's conventions for booleans, given that they are consistent and reasonable.

@paultag

This comment has been minimized.

Copy link
Member

paultag commented Dec 17, 2013

Dayum, this thread fired up.

language who's designer did not think for more than a second about the consequences of 0 having false semantics

I re-opened the bug, didn't I?

I'm still with @olasd and @khinsen here - what's the usecase?

@agentultra

This comment has been minimized.

Copy link
Member

agentultra commented Dec 17, 2013

The use-case I've found in the past is unbounded input where 0 is still a value and the predicate is testing for the presence of a value vs. None (since both evaluate to False in a boolean context but we're interested in the value of one of them):

(ap-map (if it (* it 3)) [1 0 2 3 None 3]) ;; aw sadface

Obviously if you've been coding Python for any amount of time you're used to seeing:

map(lambda x: x * 3, (n for n in input if n is not None))

Which is a minor nitpick. Technically conflating a numerical value with a boolean is mixing apples with oranges (and is a legacy left to us from Algol). Despite having to test explicitly for the presence of None in our lists the world hasn't come apart. It just has sloppy code and has perhaps raked the hackles of a few programmers who didn't catch the edge case in their tests.

@noprompt

This comment has been minimized.

Copy link
Author

noprompt commented Dec 17, 2013

Statements like "you should see a therapist" or "you're doing X a disservice" are not helping you to get your point across. Please calm down.

@olasd I was sort of joking but my sense of humor can be weird. Sorry if I offended anyone. (@tuturto )

Other lisps do it that way so it must make sense. I tried, but I can't find why.

0 is a value

It should not be thought of as False anymore than 1 should be thought of as True. The fact it is considered as such in Python is baggage. Other than familiarity and old habits, I can't think of a good reason to defend this.

Technically conflating a numerical value with a boolean is mixing apples with oranges

This everyone. 👍

language who's designer did not think for more than a second about the consequences of 0 having false semantics

I was referring to the creator of Python. Not you. 😄

what's the usecase?

  • It prevents a ton of subtle bugs.
  • You can safely use predicates that might return 0 with if.
  • Code such as (if (first xs) ...), (if (index-of xs "pie") ...), or (if (get map key) ...)

Implementing is_truthy in a sensible way is probably a non-trivial job.

@khinsen Is it really that hard to write a function that returns False if the x is False or None and True otherwise? Tell me what I'm missing here.

I don't see any philosophical problem with introducing a Lisp that adopts Python's conventions for booleans, given that they are consistent and reasonable.

I wouldn't either except that virtually every Lisp in existence doesn't do this. Python's "conventions" are really bad in this instance. Just because Hy emits Python code doesn't mean it has to have the same semantics.

@tuturto

This comment has been minimized.

Copy link
Contributor

tuturto commented Dec 17, 2013

@olasd I was sort of joking but my sense of humor can be weird. Sorry if I offended anyone. (@tuturto )

None taken, I'm just trying to understand how this should work and what implications it will have.

@khinsen

This comment has been minimized.

Copy link
Member

khinsen commented Dec 18, 2013

@noprompt: The problem is not writing the implementation, but being sure what the best semantics are. You propose that every value other than False and None should test as True. How much code did you write to validate this choice in practice? We do have to interoperate with Python code, and the transition should be fluent. We don't want lots of "truth converters" at the Hy-Python interface.

Concerning your judgement of Python's conventions for booleans, I think you are too quick in extrapolating your Lisp habits to other languages. Kenneth Iverson, the inventor of APL, has made a good case for not having booleans at all and use 0/1 instead. He showed that the common boolean operators become just special cases of more general arithmetic operators with this choice, and that those arithmetic operators are useful in their own right. You can see this approach at work in a lot of NumPy code. For example. numpy.repeat is most often used as filter for arrays, but it is a useful generalization of filtering.

The choice we are discussing is not between a "good" and a "broken" definition, but between the conventions of our target platform (Python) and those of our source of inspiration (Lisp).

@noprompt

This comment has been minimized.

Copy link
Author

noprompt commented Dec 18, 2013

How much code did you write to validate this choice in practice?

In Hy or in general? I've mostly been evaluating Hy for scripting purposes so, admittedly, not much. In general, I've written a lot of code in other languages to verify it's value from a pragmatic stand point. I will not restate my position or provide further examples.

We don't want lots of "truth converters" at the Hy-Python interface.

Can you demonstrate (with code) an example of what you're worried about here? Everyone keeps talking about this and not providing examples.

Concerning your judgement of Python's conventions for booleans, I think you are too quick in extrapolating your Lisp habits to other languages.

It's not a Python convention anymore than it is a JavaScript or PHP convention. This "convention" has historical roots as has discussed above. It is also not a "Lisp habit" of mine. Ruby, for example, also thinks of 0 as truthy and I was programming with it long before coming to Lisp. I've had a couple years to think about the benefits of this property so, no, it is not a quick extrapolation.

The choice we are discussing is not between a "good" and a "broken" definition, but between the conventions of our target platform (Python) and those of our source of inspiration (Lisp).

I understand that. But consider meditating on this for a while. Clojure targets no less than three platforms, each of which considers 0 to be falsey, and yet maintains the Lisp tradition. That being said I am not disregarding your argument. If someone were to write a Lisp targeting Haskell it would be reasonable to expect the predicate position of the if expression to be valid only when that expression is of type Bool. In the case of Hy, however, I simply have not seen any evidence so far that warrants the sort of fear people are expressing (ie. code).

@rwtolbert

This comment has been minimized.

Copy link
Member

rwtolbert commented Dec 19, 2013

While i get the basic idea that 0 being a value is a good thing, i do worry about one case. Python is used to wrap a ton of C extensions. If these C extensions use 0 and 1 for False and True, how is that manifested in Python (and Hy) land? Will that cause issues if 0 doesn’t mean to False in Hy? Or can we head that off at the interface layer?

I don’t think we can know for sure. It may be defined as “broken” for C to do this, but we do need to think about how that interop would work?

We use SWIG a ton at work, but we no longer wrap C code, only C++ with proper boolean support that maps to Python bools and will not be affected by this 0 issue. But we are only one case.

And I think fear is too strong a word, but there is cause for care and consideration.

Bob

On Dec 18, 2013, at 1:48 PM, Joel Holdbrooks notifications@github.com wrote:

How much code did you write to validate this choice in practice?

In Hy or in general? I've mostly been evaluating Hy for scripting purposes so, admittedly, not much. In general, I've written a lot of code in other languages to verify it's value from a pragmatic stand point. I will not restate my position or provide further examples.

We don't want lots of "truth converters" at the Hy-Python interface.

Can you demonstrate (with code) an example of what you're worried about here? Everyone keeps talking about this and not providing examples.

Concerning your judgement of Python's conventions for booleans, I think you are too quick in extrapolating your Lisp habits to other languages.

It's not a Python convention anymore than it is a JavaScript or PHP convention. This "convention" has historical roots as has discussed above. It is also not a "Lisp habit" of mine. Ruby, for example, also thinks of 0 as truthy and I was programming with it long before coming to Lisp. I've had a couple years to think about the benefits of this property so, no, it is not a quick extrapolation.

The choice we are discussing is not between a "good" and a "broken" definition, but between the conventions of our target platform (Python) and those of our source of inspiration (Lisp).

I understand that. But consider meditating on this for a while. Clojure targets no less than three platforms, each of which considers 0 to be falsey, and yet maintains the Lisp tradition. That being said I am not disregarding your argument. If someone were to write a Lisp targeting Haskell it would be reasonable to expect the predicate position of the if expression to be valid only when that expression is of type Bool. In the case of Hy, however, I simply have not seen any evidence so far that warrants the sort of fear people are expressing (ie. code).


Reply to this email directly or view it on GitHub.

@noprompt

This comment has been minimized.

Copy link
Author

noprompt commented Dec 19, 2013

I've done a bit of thinking and realize there is a deeper problem here that goes beyond the scope of this issue; the reliance on 0 (and others) being equivalent to False. If one is writing a predicate function (one which returns a boolean value) and relies on this property they're doing it wrong. Consider:

# Bad!
def is_empty(xs):
  return len(xs)

# Better
def is_empty(xs):
  return len(xs) == 0

Although the behavior in the context of an if statement will be the same when xs contains no elements, the first is_empty is merely and alias for len and not a predicate. I'm being pedantic here, yes. But this is the sort of thinking that leads to blurry code, strange bugs, and inconsistent APIs.

I hope I'm in the ballpark here when I assert this is the type of thing I think people are worried about. And rightly so. It reflects poor craftsmanship and, I too, would be concerned if the APIs I were consuming relied on the falsiness of 0, {}, [], and so forth to determine the outcome of my program.


Hy has an opportunity here to minimize the JavaScript level of insanity with regard to truthiness by defining a simple notion of truth. 0, {}, [], (), and "" are all considered equivalent (==) to False in Python. Those are five places where a programmer has the opportunity to shoot themselves in the foot. It isn't necessary.

I'm updating the title of this issue.

@noprompt

This comment has been minimized.

Copy link
Author

noprompt commented Dec 19, 2013

@rwtolbert

If these C extensions use 0 and 1 for False and True, how is that manifested in Python (and Hy) land?

By writing wrapper code that handles these situations appropriately. Many Java APIs return -1, 0, and 1 to signify various things. In Clojure this is worthless so the API is wrapped in something that is meaningful.

Will that cause issues if 0 doesn’t mean to False in Hy?

Why would it? If 0 doesn't mean False in Hy it doesn't mean False. Where's the issue?

Or can we head that off at the interface layer?

No. This should only concern Hy's if expression (which should be the base for all conditional macros). Values should be interpreted as expected by the platform in consideration. If a Hy function returns 0 and hands that to Python, expect Python's semantics and vice versa.

I don’t think we can know for sure. It may be defined as “broken” for C to do this, but we do need to think about how that interop would work?

Yes, but also no. Hy simply needs to determine what it considers to be truthy and move forward.

This is more or less what I'm proposing Hy use to make this determination:

def boolean(x):
  return not (x is False or x is None):

Under the covers (if expr ...) would behave as (if (boolean expr) ...).

@Coaldust

This comment has been minimized.

Copy link

Coaldust commented Dec 19, 2013

Hy's current behavior is the correct behavior. I'm stating this as a fact, rather than a opinion, because making the suggested change will cause bugs.

A careless reading of the proposal makes it seem reasonable. Indeed, treating 0 and empty containers as a boolean value is wrong. Doing this leads to bugs, as can be seen frequently in C-syntaxed languages where "=" is accidentally used in a "if" statement. That's why some modern programming languages have a real boolean type, and do not automatically convert nonsense into boolean values. Popular programming languages that get this right include C# and Java.

A careful reading reveals that the author actually wants 0 and empty containers to be treated as True, because some other Lisps made this mistake.

So the proposal suggests replacing bad Python semantics with bad Lisp semantics. Note that I consider both to be bad semantics for reasons I've already stated. If I had my way, and it didn't result in overhead and weird looking code from hy2py (which it would), a exception would be thrown any time a test expression resulted in a type that was not "bool". It would, in fact, be possible to do this.

The reason changing Hy's current behavior is "wrong" is that Hy code may be called by Python programmers who do not know the code was written in Hy. These Python programmers will expect None 0 and empty containers to be considered False. Making use of the fact that non-existent and empty containers are considered False is unfortunately encouraged by the Python community. In other words, the bad Python semantics are often depended upon.

Being able to use Hy code 'invisibly' is a large part of its appeal. It is a 'stealth Lisp'. I realize "a transpiler for s-expressions into Python" was meant to be disparaging, but Hy should wear the label proudly.

A Python programmer doesn't have to go through any effort at all to use code written in Hy. They just import it and call it as if it was all written in Python.

hy2py can even generate very readable Python code that, with minor tweaking, you could pass off as hand-written in a workplace that forbid the use of any programming language but Python. At least you could avoid writing lots of boilerplate and "design patterns" that macros can automate.

This creates the opportunity to use it in the workplace. It could resurrect the, seemingly extinct, Lisp job.

Of course if Hy not replicating every last aspect of Common Lisp or Scheme, even the bad parts, really infuriates you, you can always use macros to re-implement them on top of Hy. Racket has used macros to embed a statically type checked Lisp, after all.

@paultag

This comment has been minimized.

Copy link
Member

paultag commented Dec 19, 2013

Alright debate has gone on long enough.

There are 2 ways to fix this, that I can see.

First is to shadow if, and to add a hy core function that tests if the
function is not either False or None.

Second is to create a new function (something like truth?) that you can use
in filter and friends. We can talk about adding if-truth? as well.

Our official stance, unless I find a way to convince myself otherwise
(which I might, but I don't think I will)

We're going to go with the second. I don't think the gain in syntax clarity
(which I think there is) is worth fighting Python over.

Let's get a PR including truth? and is-truth? (or whatever we'll call that)
in soon.

On Thu, Dec 19, 2013 at 10:11 AM, Coaldust notifications@github.com wrote:

Hy's current behavior is the correct behavior. I'm stating this as a fact,
rather than a opinion, because making the suggested change will cause bugs.

A careless reading of the proposal makes it seem reasonable. Indeed,
treating 0 and empty containers as a boolean value is wrong. Doing this
leads to bugs, as can be seen frequently in C-syntaxed languages where "="
is accidentally used in a "if" statement. That's why some modern
programming languages have a real boolean type, and do not automatically
convert nonsense into boolean values. Popular programming languages that
get this right include C# and Java.

A careful reading reveals that the author actually wants 0 and empty
containers to be treated as True, because some other Lisps made this
mistake.

So the proposal suggests replacing bad Python semantics with bad Lisp
semantics. Note that I consider both to be bad semantics for reasons
I've already stated. If I had my way, and it didn't result in overhead and
weird looking code from hy2py (which it would), a exception would be thrown
any time a test expression resulted in a type that was not "bool". It
would, in fact, be possible to do this.

The reason changing Hy's current behavior is "wrong" is that Hy code may
be called by Python programmers who do not know the code was written in Hy.
These Python programmers will expect None 0 and empty containers to be
considered False. Making use of the fact that non-existent and empty
containers are considered False is unfortunately encouraged by the Python
community.
In other words, the bad Python semantics are often depended
upon.

Being able to use Hy code 'invisibly' is a large part of its appeal. It is
a 'stealth Lisp'. I realize "a transpiler for s-expressions into Python"
was meant to be disparaging, but Hy should wear the label proudly.

A Python programmer doesn't have to go through any effort at all to use
code written in Hy. They just import it and call it as if it was all
written in Python.

hy2py can even generate very readable Python code that, with minor
tweaking, you could pass off as hand-written in a workplace that forbid the
use of any programming language but Python. At least you could avoid
writing lots of boilerplate and "design patterns" that macros can automate.

This creates the opportunity to use it in the workplace. It could
resurrect the, seemingly extinct, Lisp job.

Of course if Hy not replicating every last aspect of Common Lisp or
Scheme, even the bad parts, really infuriates you, you can always use
macros to re-implement them on top of Hy. Racket has used macros to embed a
statically type checked Lisp, after all.


Reply to this email directly or view it on GitHubhttps://github.com//issues/373#issuecomment-30935726
.

:wq

@paultag

This comment has been minimized.

Copy link
Member

paultag commented Dec 19, 2013

if-truth? *

@noprompt

This comment has been minimized.

Copy link
Author

noprompt commented Dec 19, 2013

@Coaldust

Hy's current behavior is the correct behavior. I'm stating this as a fact, rather than a opinion, because making the suggested change will cause bugs.

Everyone keeps saying this with no examples or code to back it up.

These Python programmers will expect None 0 and empty containers to be considered False.

As I've pointed out this should be a problem. Look at the boolean function I discussed. What if a Python library employed that technique privately in it's API, would you still be worried? Also I never said None should be considered truthy.

A Python programmer doesn't have to go through any effort at all to use code written in Hy. They just import it and call it as if it was all written in Python.

I've written ClojureScript code meant to be called from JavaScript and it works just fine with the semantics I have proposed.

So the proposal suggests replacing bad Python semantics with bad Lisp semantics.

Haha! Touche! But I'd prefer "better Lisp semantics." 😆

This creates the opportunity to use it in the workplace. It could resurrect the, seemingly extinct, Lisp job.

Have you heard of Clojure? It's pretty much filling that role. 😸

you can always use macros

Yes, I suppose there's always that escape hatch.

@noprompt

This comment has been minimized.

Copy link
Author

noprompt commented Dec 19, 2013

@paultag is right, there's likely no sense in furthering this debate.

First is to shadow if, and to add a hy core function that tests if the
function is not either False or None.

This is the Lisp tradition and what I personally think is the right way to go. No one has demonstrated an example where this could be a problem outside of a Python programmer's initial expectations. Beside no one should be using Hy's if directly from Python code.

Second is to create a new function (something like truth?) that you can use
in filter and friends. We can talk about adding if-truth? as well.

This solution will also work. If someone wants "real" Lisp semantics for if they could always write a macro using this function.

(defmacro if+ [test success fail]
  `(if (boolean ~test) ~success ~fail))

However, the downside to this is if the first solution is not the default, the behavior of if will be misleading when the core functions that verify truthiness have a different notion of what truth is. Hy needs to settle on it's interpretation of truthiness and implement if to satisfy it. Regardless of what decision is made, consistency is paramount.

@agentultra

This comment has been minimized.

Copy link
Member

agentultra commented Dec 20, 2013

@noprompt I like the idea and would review a PR implementing it.

@joehakimrahme

This comment has been minimized.

Copy link
Contributor

joehakimrahme commented Dec 20, 2013

@agentultra +1. I agree with the idea. If someone implements it, I could also review it.

@Coaldust

This comment has been minimized.

Copy link

Coaldust commented Dec 20, 2013

It might be suspected that given the heat to light ratio of this discussion that it's all bikeshedding. This may be partially true. If you don't depend on bad semantics then you won't notice any difference between other Lisps and Hy.

Depending on 0 and empty containers being considered True is like depending on the order of evaluation of arguments. Common Lisp happens to define the order of evaluation as left to right, but many other programming languages do not, because not doing so provides more optimization opportunities. Just because some Lisp did it, does that mean we have to too? If so, what do we do when two or more Lisps disagree? For example, Scheme does not define order of evaluation of arguments.

One of the ways Scheme supposedly improved on older Lisps was by not confounding nil, false, and empty. Indeed, eq?, eqv? and equal? return #f (false) when comparing these. I was unpleasantly surprised to find out that your if examples do work in Scheme. It had never occurred to me to write code that way, much less depend on it.

I will try to add some light to this discussion now...

noprompt demanded examples of where depending on bad Lisp semantics, instead of bad Python semantics, could cause bugs. Here are some examples.

Imagine I'm writing a predicate in Hy that is intended to be used to filter a container of containers. It should return True when a entry is "interesting", and False otherwise. It should be as robust as possible.

A concrete example is a predicate to filter key combinations within a certain range of frequency of use, to be used in a keyboard layout optimizer. The key combinations are stored as frozensets within a frozenset. I've actually written this, so it's not a contrived example.

The way a programmer calling this predicate from Python would expect it to be written is:

    def is_valid_frequency(key_set):
        if key_set:
            key_set_frequency = frequency(key_set)
            if key_set_frequency >= MIN_FREQ and key_set_frequency <= MAX_FREQ:
                return True
            else:
                return False
        else:
            return False

Python's official style guide http://www.python.org/dev/peps/pep-0008/#programming-recommendations says "For sequences, (strings, lists, tuples), use the fact that empty sequences are false." Although no reason is given, I believe it's because this results in shorter code.
A direct translation to Hy with its current semantics is:

    (defun valid-frequency? [key-set]
      (if key-set 
          (let [[key-set-frequency (frequency key-set)]]
            (if (and (>= key-set-frequency +min-freq+)
                     (<= key-set-frequency +max-freq+))
                True
                False))
          False))

A translation to Hy with the proposed semantics is:

    (defun valid-frequency? [key-set]
      (if (and (!= key-set None)
               (!= (len key-set) 0))
          (let [[key-set-frequency (frequency key-set)]]
            (if (and (>= key-set-frequency +min-freq+)
                     (<= key-set-frequency +max-freq+))
                True
                False))
          False))

A programmer who has grown accustomed to thinking of Hy as just another Lisp is more likely to write:

    (defun valid-frequency? [key-set]
      (let [[key-set-frequency (frequency key-set)]]
        (if (and (>= key-set-frequency +min-freq+)
                 (<= key-set-frequency +max-freq+))
            True
            False)))

It's shorter, easier to understand, and more idiomatic Lisp. However, this code will not tolerate missing or empty entries like Python programmers have grown to expect. They could be in for a nasty surprise when calling it from Python.

At this point I should admit to having simplified the previous code. In reality, for a keyboard layout optimizer, you would not use constants for the frequency range. This was done in the interest of shortening the argument, which I still feel is overlong, but it seems some people were having trouble following it, so I decided to be very explicit, while trying to avoid extraneous complexity.

I have also written a program that manipulates a binary format for skeletal animations used by a 3-D game. I won't be posting any code, because it's long, and the relevant code is very simple to explain. The binary format stores some True/False and On/Off values. True and On are represented as 1, while False and Off are represented as 0, as in the vast majority of programming languages. Using Lisp bad semantics, instead of common bad semantics, makes (de)serializing these values more complex and error prone, because extra code will need to be written to map between the two.

Although I have not done it, Python can call procedures in C. This is necessary at some level to be able to interact with the outside world, since all popular operating systems are written in C. C considers true to be 1 (or at least that's what the TRUE constant is, but technically any non-zero integer will work) and false to be 0. If Hy adopts bad Lisp semantics interfacing with C will become more error prone and complex, because extra code will need to be written everywhere booleans are passed.

Those are the examples I can think of where adopting bad Lisp semantics instead of bad Python semantics could create bugs. I'm sure there are more. What they have in common is interoperability with the rest of the world, which isn't written in Lisp.

Although Lisp fans may be bitter about it, it's important to recognize the reality that Lisp is not a popular programming language. Every programming language popularity index that I'm aware of show it, in any form (Common Lisp, Scheme, Clojure...), as accounting for less than 1% of usage.

The significance of Lisp's unpopularity is that it is very important for code written in Lisp to be safe and easy to call from popular programming languages. That means respecting the conventions of popular programming languages. Lisp fans should try to write useful, safe, easy to use, code in Lisp, that can be called from other programming languages, so other programmers will at least tolerate its usage, and ideally decide to use it. With popularity comes more libraries (e.g. of macros) and job opportunities. That's why Clojure and Hy are hijacking popular environments.

Take a "Lisp's way or the highway" attitude at your own peril. Lisp programmers are not in a position to demand other programming languages adopt their ways.

Another issue is overhead. If every if requires an extra function call to enforce Lisp's boolean type punning, it will be slower. This may be enough to matter when it occurs in loops, especially deeply nested ones. CPython is particularly slow, relative to most programming languages ( http://benchmarksgame.alioth.debian.org/u32q/benchmark.php?test=all&lang=all&lang2=python3&data=u32q ).

People wanting to use hy2py to generate code that can easily be cleaned up to look like handwritten Python code (something Clojure cannot do) will also have more work to do, reducing its utility. I may be in the minority in that I care about this though.

Yet another issue is the "slippery slope". If Hy reduces its usability and performance, just so the small number of existing Lisp programmers can do something they really shouldn't be doing anyway, where does this end?

Things that actually matter (unlike supporting something you shouldn't do) that others might then expect Hy to adopt:

  • full compatibility with Common Lisp defmacro, Scheme syntax-rule and syntax-case, Racket syntax-parse, and/or Clojure defmacro
  • full compatibility with Common Lisp, Scheme, and/or Racket reader macros
  • Scheme's full numeric tower
  • Scheme's tail-call elimination
  • continuations in delimited (e.g. Racket's) and/or non-delimited (e.g. Scheme's) form
  • Lisp Machine Lisp's "world" image dumping and restoration
  • Lisp Machine Lisp's versioning of all files

To think I was happy just having a "s-expressions to Python transpiler" (with macros)...

@emidln

This comment has been minimized.

Copy link

emidln commented Jan 30, 2014

I'd like to point out that if-let and when-let are enormously more useful in Hy because of pythonic truthiness vs clojure's truthiness.

https://gist.github.com/emidln/8589060

https://gist.github.com/emidln/8588652

@cwebber

This comment has been minimized.

Copy link
Member

cwebber commented Feb 24, 2014

I know nobody needs to add things to the debate, the debate is over. But there's something learned in this bug, and I want to just note it here for future reference.

I think after reading agentultra's comment here my feelings on this and related issues became really clear. @agentultra made things really clear there, so I recommend just reading above.

So a year ago when I wrote the first draft of the tutorial in Hy, I added a section somewhat jokingly phrased: "Hy is python flavored lisp (or vice versa?)"

It's obvious to me now which side of that it goes: Hy is a lisp flavored Python. But it's still Just Python™. This doesn't mean that it's not incredibly value-added... it is by a lot. By getting a "lisp flavored python" you get a lot of cool things in that flavor... being able to design new language features with macros being the most obvious.

If someone really wants scheme or common lisp, those languages exist, people can use them. But there's nothing else out there like Hy. I don't mind supporting some like-my-favorite-lisp in officially supported modules (probably external). But if we move away from Python semantics, we break expectations. What's the point of being able to import .hy files in .py files if they don't act like Python? Do we really want to go to great effort to lose the core of what Hy really is?

I think we should get in a if-truth macro, either within Hy core or in a contrib module, and close out this bug. But I suggest we also remember that lesson.

@cwebber

This comment has been minimized.

Copy link
Member

cwebber commented Apr 10, 2014

Since #519 is now closed and we have lisp-if support, I think we can now close this.

@cwebber cwebber closed this Apr 10, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.