Variable declaration vs assignment #712

Closed
akva opened this Issue Sep 23, 2010 · 71 comments

Comments

Projects
None yet

akva commented Sep 23, 2010

I figured that general discussion of CoffeeScript belongs to issue tracker. So here it goes.

It seems like coffee-script does not distinguish variable declaration and assignment. This means that, unless you are very careful with naming local variables, it's easy to unintentionally assign to a global variable. Consider this js code:

sys = require('sys');
var foo = 42;
(function() {
    var foo = 43
}());
sys.puts(foo);

Here variable foo inside a function is local to that function. It is not possible to reproduce this code in CoffeeScript. CoffeeScript 'equivalent' will override global variable foo.

sys = require 'sys'
foo = 42
(() -> foo = 43)()
sys.puts foo

This may lead to hard to find bugs. What's your thoughts on this?

Regards,
-- Vitali

Contributor

StanAngeloff commented Sep 23, 2010

It has been well documented and discussed on the tracker before. I am actually quite happy with how Coffee handles scoping at the moment.

akva commented Sep 23, 2010

I just read the documentation again more carefully. Indeed, I was too quick to post an issue. My bad. However, let me elaborate on this issue a little bit. This is a part of the documentation that explains my point in fewer words.

Because you don't have direct access to the var keyword, it's impossible to shadow an outer variable on purpose, you may only refer to it. So be careful that you're not reusing the name of an external variable accidentally, if you're writing a deeply nested function.

Consider this code

...
... many lines of code (likely not written by me)
...
func = () -> # a new function that I add
    foo = 42 # not clear if I declare a local var or assign a global one

JS has different syntax for variable declaration var foo = 42 and assignment foo = 42. The problem with it imho is that declaration syntax is more verbose than assignment syntax (and var is easy to forget). It seems that variable declarations, especially if one tends to write in functional style, appear in source code more often than assignments. Therefore declaration syntax should be more concise and assignment could be more verbose.

My preferred syntax would look something like this:

foo = 'foo'
bar = 'bar'
...
... many lines of code
...
func = () ->
    foo = 42  # declares local var. shadows global
    bar := 43 # assigns global var
    fooo := 44 # compile error

This approach is only slightly more verbose but makes programs more explicit and robust, imho. And again, in my own code I tend not to use destructive assignment that much.

What do you think?

P.S. Python uses nonlocal keyword but it's a little overkill for my taste.

Contributor

StanAngeloff commented Sep 23, 2010

How about referencing a variable from the global scope? If you simply do alert bar from a function, should it throw an exception if bar is not in scope? It seems like it should since assigning does so and we need things to be consistent.

When you are writing in Coffee, you will not always have full control over what files you interface with. While it may be possible to throw compile time errors when referencing a non-existing global variable, we cannot reliably do so when you are working on a file that just happens to be a small part of an application which in turn might have been written in pure JavaScript.

Coffee is also used on the server-side. In node.js it's common to have many callbacks, like so:

fs.readFile 'file.js', 'utf8', (err, contents) ->
  # do some other stuff
  process.nextTick ->
    throw err if err

Here we are referencing err in a function that is nested inside another function. Since err is not in scope, but it's also not part of the global one, how do you assign or reference it?

Keep in mind Coffee is a dynamic language just as JavaScript is. There is no type checking, no NullReference exceptions and no actual compilation.

If I have to point out one benefit of the existing scoping, it would be that it's consistent. Define once and use everywhere.

akva commented Sep 24, 2010

Maybe I didn't express myself clear enough. I guess I should have used the word outer scope instead of global.

I am not proposing to change the scoping rules. Quite the opposite, I propose to keep them exactly the same as in JavaScript, and therefore making Coffee more consistent with JavaScript (I am as big fan of consistency). One way to do it is to introduce the following syntax

  1. foo = 42 to mean JavaScript's var foo = 42;
  2. foo := 42 to mean JavaScript's foo = 42;

Right now foo = 42 can mean two different things in Coffee depending on the context. It can be a variable declaration, or an assignment to a lexically scoped variable. And this is not consistent.

So your example with fs.readFile will just work unchanged as expected. err is in lexical scope of the nested function so everything is all right.

goj commented Sep 24, 2010

How about keeping current semantics and re-introducing JavaScript's var keyword to force declaration?

v = 1
fun = () ->
    var v = 2
fun()
alert v # shows 1

akva commented Sep 24, 2010

If I understand correctly what you mean by forced declaration, it doesn't really solve the issue. v = 2 is still ambiguous (can be local declaration or assignment to outer scope) depending on the context, and the context is as big as the whole source file. What "forced" declaration says is - I know there is this variable v in the outer scope and I want to shadow it. What I am trying to achieve is when I see v = 2, I could conclude - This is a local var and I don't need to check whether it's already declared in outer scope or not

To give anouther example. Say you have a function somewhere at the bottom of the source file.

fun = ->
    v = 2

Here v is a local var. Fine. Then a few months later a new developer comes in and adds v = 1 at the top of the file. Now, he altered the behavior of fun without even touching it.
And this will be hard to debug.

JavaScript copied scoping rules from Scheme but imho it got the defaults wrong. var should have been implicit and assignment more explicit.

I realize that proposed foo := 42 syntax might look old fashioned. I don't insist on this particular one. It's just the most concise form I could think of.

goj commented Sep 24, 2010

Yes, you understood me correctly.

AFAIK there is no way to shadow variables in CoffeeScript, which
is, as you explained is a big problem.

Using var was a proposal to allow that in backward-compiatible way.
This is much better than only solution that we have today
("be careful with naming your variables") but it results in almost copying
the problem JavaScript have today
("always remember to put vars in your code or you'll be in trouble").

The more I think about your proposal the more I like it.
It really solves the problem and gives us more consistency with JavaScript.

It is a huge backward incompatibility, but hey, CoffeeScript is before 1.0 version
and this really seems worth it.

Speaking of syntax, := stands out a little, but it should stand out when we change
something from outer scope, so that's more than OK. The other way to go would
be to use some sigil, like &, which would play nice with destructuring
assignment:

[&im_from_outer_scope, declare_me] = stuff

Other way to word akva's proposal that would probably make Stan happier is:
you assign to local variable using = (as you do now) but if you want
to change something from the outer scope, then you have be explicit
(use :=). This makes both code easier to read and bugs harder to introduce.

Collaborator

satyr commented Sep 24, 2010

Note that [we do have a way](http://satyr.github.com/cup/#foo%20=%2042%0A((foo%29%20-%3E%20foo%20=%2043%29(%29%0Aputs%20foo) (+1 by the way) to shadow outer variables.

See also: #238

akva commented Sep 25, 2010

Speaking of syntax, := stands out a little, but it should stand out when we change something from outer scope, so that's more than OK.

Exactly. Destructive assignment is a dangerous operation and should stand out as a warning. And, to repeat myself, programs written in functional style won't have too many :='s. If one finds oneself using too many :='s it's probably a good sign that code needs refactoring.

you assign to local variable using =

Don't want to sound pedantic but just to make sure we're on the same page. By assignment I mean destructive assignment, or rebinding the earlier declared variable. In this sence you use := to assing/rebind local var too. That is:

local = initialize()
if something()
    local := new_val()

@satyr

Note that we do have a way to shadow outer variables.

Yes, but shadowing is not the problem.
I've been thinking about the best way to summarize my issue and I think it can be summarized into the following two problems:

  1. Delocalization (I am just making up the term)
    Every time you indroduce a new variable in scope A you have to make sure the variable name is not already used in any of the nested functions. Otherwise you'll break it.
  2. Ambiguity
    var = val is ambigues. To understand what it means you have to inspect all outer scopes.

It is a huge backward incompatibility, but hey, CoffeeScript is before 1.0 version
and this really seems worth it.

Totally agree. It took Python 3.0 to get the scoping right - http://www.python.org/dev/peps/pep-3104/ (though they had a different problem to start with). CoffeeScript has a chance to get it right from the very beginning. Especially considering that it's not hard to implement, since JavaScript already gets the scoping (almost) right.

Owner

jashkenas commented Sep 26, 2010

Sorry, folks, but I'm afraid I disagree completely with this line of reasoning -- let me explain why:

Making assignment and declaration two different "things" is a huge mistake. It leads to the unexpected global problem in JavaScript, makes your code more verbose, is a huge source of confusion for beginners who don't understand well what the difference is, and is completely unnecessary in a language. As an existence proof, Ruby gets along just fine without it.

However, if you're not used to having a language without declarations, it seems scary, for the reasons outlined above: "what if someone uses my variable at the top of the file?". In reality, it's not a problem. Only the local variables in the current file can possibly be in scope, and well-factored code has very few variables in the top-level scope -- and they're all things like namespaces and class names, nothing that risks a clash.

And if they do clash, shadowing the variable is the wrong answer. It completely prevents you from making use of the original value for the remainder of the current scope. Shadowing doesn't fit well in languages with closures-by-default ... if you've closed over that variable, then you should always be able to refer to it.

The real solution to this is to keep your top-level scopes clean, and be aware of what's in your lexical scope. If you're creating a variable that's actually a different thing, you should give it a different name.

Closing as a wontfix, but this conversation is good to have on the record.

akva commented Sep 27, 2010

I was going to write a long response, but instead decided to replace it with a short question. Here

foo = 42
(-> for foo in [1..2] then)()
alert foo # 42

loop variable foo (which is just a local variable?) shadows the top-level foo, which is inconsistent with the rest of the language. How do you explain this inconsistency?

Owner

jashkenas commented Sep 27, 2010

I'm not saying that shadowing isn't technically possible -- just that it's bad style, and CoffeeScript shouldn't make it easy for you to accomplish. It would be far better to use a different name for that index.

akva commented Sep 27, 2010

I totally agree with you that shadowing is a bad style. And instead of shadowing it's indeed better to just choose a different name.

But I am not talking about deliberate shadowing. I am talking about situations when you are not aware that top-level variable exists and assign it accidentally, or when top-level variable is added later. Check this example from my comment. The body of fibonacci function is supposed to be a black-box, but it's not. And what's worse, even unit test won't spot it - when run in isolation fibonacci function will work just fine, but will break in the context of the whole module. In other words the current scoping rules break encapsulation. This example is particularly illustrative since cache is such a common name and can easily clash.

Collaborator

satyr commented Nov 3, 2010

 foo  = 'foo'
 bar  = 'bar'
 func = ->
   foo   = 42  # declares local var. shadows global
   bar  := 43  # assigns global var
   fooo := 44  # compile error

Adopted this strategy on my coco branch and am quite happy with the decision.

akva commented Nov 3, 2010

@satyr Looks great. Would you like to share some more details about your experience of using it?

Honestly, I reckon Ruby scoping rules are just wrong. It seems that they may change in Ruby 2.0

Collaborator

satyr commented Nov 4, 2010

Would you like to share some more details about your experience of using it?

During the change among CS sources, I've found:

  • only 10 instances of = that needed conversion to :=.
  • a place where a variable on an upper scope was accidently modified.

akva commented Nov 4, 2010

@satyr Thanks

only 10 instances of = that needed conversion to :=

That's about what I expected.

a place where a variable on an upper scope was accidently modified.

It's true that these bugs are quite rare BUT they are extremely hard to find. Even unit tests won't help. I rather deal with the bugs that happen all the time but easy to find than with those that are quietly sitting around as a ticking bomb waiting to explode.

ggRalf commented Nov 25, 2010

I discovered the scoping problem on my own.
After a small chat i was pointed to this thread.
And finally I looked at the coco branch. Looks really nice!
Since this language is really young I will switch to the coco branch and hope it will be maintained.
Thanks for the really nice language and syntax!

So I've reached this thread also. In fact, it's very arguable whether the shadowing is a bad style. Behind and before this term "shadowing" stands the basic term of programming -- an abstraction.

A helper procedure should be really a black box. It's the main principle of substitution the arguments for a formal parameters even in math functions. The same stands for the local variables.

The abstraction barrier which separates the level of implementation of a procedure from the level of usage of the procedure should be the real barrier in a well-designed system. Which means -- it should not bother the level of usage with exact variable names used inside.

A designer of the procedure also shouldn't worry about which variables to use as just helper local variables. In general, it can be a casual case -- to reuse a some 3rd-party procedure in own project.

Exactly for this concept of a scope and in particular nested scopes (namespaces, modules, classes, etc) is invented.

That the Ruby have chosen to declare local vars without any keyword and refer these variables as outer from closures (lambdas, blocks, procs) is just the exact and the particular case -- and actually very arguable implementation.

The mentioned reason ("it's a completely lexical scope") in fact just breaks the concept of nested scopes. And moreover, it smells like a substitution of concepts. The lexical scoping is one when it's possible at parsing stage to determine in which scope a variable will be resolved in runtime. And this determination is made by the place of the variable's definition. I.e. we may have several nested definitions with the same name and it still will be the lexical scope.

If you don't like a special keyword for the definition, you may consider instead Python's way (which was mentioned also above in this thread). Though, less ugly keyword than nonlocal can be used. E.g. outer:

a = 10
b = 20

foo = ->
  outer a = 30
  b = 40

alert a, b # 30, 20

It requires to capture manually needed closured vars thought.

Or indeed, maybe nevertheless to return the definition keyword but not in the definition semantics but in semantics of localizing the scope. E.g. let. It's really like in math. First you say, that x is 10, but later, you decide that for this function, "let x be a string.

x = 10

foo = ->
  let x = "test"
  y = 30 # Syntax Error (undefined global/local var)

foo()

alert x # still 10

Which semantically, repeat, sounds even not as "define a variable", but as "assume for this scope name x to be with this value". Thus, in global scope a var can be created without keyword.

Or e.g. Lua's local keyword can be used also. That you can in general determine whether the var was already declared, right? However, it's hard with eval.

Anyway, current breaking of abstraction -- that is, breaking the black-box with all it's "offal" which belong to this black-box (and belong by the right) is very arguable.

Repeat, a 3rd-party programmer should be able to use any names of local (to the black-box) variables. And at the same time another 3rd-party programmer should be able to reuse this function written by the first programmer -- and without reviewing the code of the function. It's the main principle of the abstraction which seems just broken in Coffee.

Notice, that in Ruby (in contrast with Coffee) the picture is a bit different. There most frequently a user works with methods (defs) which doesn't capture (by default) local variables of the surrounding context. That is, if in the same example that 3rd-party programmer writes his method, he calm about writing a = 10 without thinking whether this name is already (or will be in unknown in advance environment -- that is worth) borrowed. And it's (probably completely) another case (from which you borrowed the design) are closures (lambdas, procs, blocks, etc) -- with they already a local user usually works and he usually knows which local vars he defined and that they will be captured. Though, in general, this also is not guaranteed that he remembers that -- in the same respect he can make a mistaking thinking that he uses a local var, but indeed he just forgot that has already defined it above).

Once again, Ruby's semantics in this respect is not the same as in JS/Coffee -- in JS all functions are closures (so the complete port of the semantics cannot be used as a best argument in explanations). And moreover, Ruby's design is not perfect in this question; consider it.

So Lua's way with local or proposed ES6 let or even Python's with proposed mine outer (i.e. even if we should "mark" needed vars "to be closured") seems more bugs- and fool-proof and without breaking the abstraction principle.

Dmitry.

odf commented Feb 9, 2011

It may help (or, as it were, end) this discussion to note that coffeescript now has the do construct.
a = 2
do (a) ->
a = 1
console.log a
console.log a
prints
1
2

odf how will it help to restore breaking principle of an abstraction? Should all programmers starts their functions with do?

So I still propose to consider Python's way but with outer keyword. Or Lua's / ES6 way -- with let keyword.

Dmitry.

odf commented Feb 9, 2011

Well, I'm hoping that eventually the behaviour of do will be fixed in such a way that we can write
a = 2
do (a = 1) ->
console.log a
console.log a
in the example above. Have a look at issue #960 for some discussion on do. I would have preferred let as the keyword, but it was decided to use one that's already a reserved word.

Yes, I'm ware about what do instruction does in Coffee. Actually it's analog of the let, yes (starting the semantics even from Scheme and desugars into immediately invoked function).

However, it's not the generic case. I mean, if you suggest to start every function with that do, I am sure it will be very annoying.

One more time. Currently Coffee isn't even consistent in chosen strategy. On one hand it says -- "no name shadowing" (i.e. there are no two frames in the environment with the same name binding), which by itself, as I wrote above, already breaks the main principle of the abstraction and nested scopes. On the other hand, it contradicts to the first chosen way, and nevertheless allows two frames with the same name bindings -- it's achieved via formal parameter names or with the same do.

And one more time. Arguing that "this is like in Ruby" isn't completely correct. Since repeat again -- in Ruby methods doesn't capture local vars of the surrounding context and create local vars via assignment (that's for the example with two 3rd-party programmers which share the code -- in Ruby and in contrast with Coffee they can do this safely). And Ruby closures (blocks, lambdas, procs, etc) do captures vars, but with closures usually already the user himself works and know his vars (though, again repeat, even this is not safe in Ruby and can be considered like a design flow).

In JavaScript as you know, all functions are closures. So the case with two 3rd-party programmers fails breaking the main principle of an abstraction.

Dmitry.

P.S.:

Exact proposals:

a = 10 
b = 20
c = 30
d = 40

foo = (a) ->

  outer b, c

  a = 100
  b = 200
  c = 300
  d = 400

foo()

console.log a, b, c, d # 10, 200, 300, 40

That is, a is local since it's a formal parameter, d is local since it's not marked as outer. OTOH, outer b and c are modified.

Another way:

a = 10 
b = 20
c = 30
d = 40

foo = (a) ->

  local b, c

  a = 100
  b = 200
  c = 300
  d = 400

foo()

console.log a, b, c, d # 10, 20, 30, 400

That is, only d was outer.

odf commented Feb 10, 2011

I don't see the point. Why would you have all those variables on the file level when the functions you're exporting are not supposed to use them? If you don't want those bindings to be visible everywhere within your file, don't put them there.

In general, if you don't pollute your scopes with unnecessary bindings in the first place, you'll have no problems with broken abstraction. I think the way to look at this is to consider the context in which a function is defined as a genuine part of it, not just an environment it was thrown into by accident.

First of all -- do you agree that Coffee isn't even consistent in its chosen way? That is, "no two bindings with the same name in the environment chain" vs. "allow nevertheless two bindings with the same name via formal parameter names and do"?

If "yes", what the difference do you see from defining a local variable via formal parameter name and just a local variable with the same name?

Why would you have all those variables on the file level when the functions you're exporting are not supposed to use them?

OK, let's take a simple example. We (you and me) work together and should support the same source.

I wrote a helper function somewhere above:

square = (x) -> x * x

You three month ago write your function 1000 lines below:

createWidget = ->
  square = new Square 10
  square.onResize = (e) -> console.log e
  square

You just used a local variable name, probably you didn't even know about my square function since that block of code was in my responsibility. What will be with my code execution then? How will we find the bugs?

We'll of course find the bug sooner or later (and moreover since we in one project, you can argue that you should know all the source and all the used identifiers above. By the way, why "should" you if to consider the principle of separation programmer responsibilities?).

But we can complicate the example, when I e.g. may reuse (in the simplest way just to copy-paste) some peace of code from completely another project to mine. Should I review all the "imported" sources to find out used variable names? If yes -- why "yes" since it's a direct breaking of the abstraction?

Dmitry.

odf commented Feb 10, 2011

I don't find your example convincing, at all. If I were unaware of which functions you defined on the file level, what would stop me from simply re-using the name square in the same scope?

Seems you just ignored my questions I wanted you to answer before your reasoning about the scope theory. Well, OK. Let's assume that you just agree with them.

For details, look at any general scope theory paper (and especially on environment frames concept) -- it will help us in discussion not to mix definitions and in particular the definition of the "same scope". E.g. http://bit.ly/esnkD6

So if you reuse square in the same scope as mine (and by this I mean the definition of the same frame of the environment), then you make an error. So, substitution and mixing of definitions is irrelevant here. We talk about different frames (different scopes), not the same -- and from this viewpoint you are able to use any identifier (and this action is even available in Coffee -- repeat via formal parameter names or via do).

However, if you use the same name in your own scope, it's completely your right as the author of this encapsulated abstraction.

Dmitry.

Just a small note. OTOH, e.g. Erlang warnings about shadowed variable:

X = 10,

Foo = fun(X) -> X + 1 end, % warning X is shadowed

Foo(20)

It's for that the shadowing can be dangerous. However, in contrast with Coffee/Ruby, Erlang's variables are immutable and just pattern matched (i.e. you can't assign to X inside the Foo function).

Dmitry.

You just used a local variable name, probably you didn't even know about my square function since that block of code was in my responsibility. What will be with my code execution then? How will we find the bugs?

Hi Dmitry, I agree with almost everything you said it. But some problems/bugs you'll find testing your code, doing TDD or something like that.
Of course when you do your tests you don't think all the possible cases however you can coverage good part of your code.

@cairesvs yes, I'm aware about TDD. Though, the question was specially to underline the design flow (it wasn't a real asking how we should find the bugs ;) But, thanks anyway for mentioning.

Dmitry.

@DmitrySoshnikov
Yes, I understand the problem. Just bring another perspective.
To me many of the problems and arguments you bring to the table could be solved with simple TDD and TDD could change the design of your project, help to find other ways to approach the same problem.
But, like I said it before, I agree with you with could be more easy to understand the code and make less bugs/problems if there is something like local/outer or let keyword. The solution brought by @akva and @satyr is similar to yours and good too.

Contributor

StanAngeloff commented Feb 10, 2011

This proposal talks about assignments, but how about accessing a variable:

x  = 10
fn = -> x
console.log fn()  # RefErr or 10?

Doesn't make sense to use outer to shadow global vars on assignment, yet still being able to access them without the modifier.

<?php
$x = 10;
function fn() {
  print $x;
}
print fn();  # Undefined $x

...and on that note, given how JavaScript scoping works, it would be insane to even try and implement non-descending scope.

@cairesvs

Yep, right, TDD may help to catch some bugs. However, the way "to use our language, you should program in TDD style or ... you've been warned - catch your bugs yourself, since our language may easily provide them " cannot be considered as the best way I guess ;)

@StanAngeloff

Nope, in this case x should be normally resolved in the global frame. Keyword global makes sense only when exactly an assignment is presented in the code. In this case it just won't add var in the generated code.

Regarding PHP example (you've added it later) -- in PHP casual functions aren't closures (the same as in Ruby methods aren't closures). So there we should use global keyword. In JS as said, all functions are closures with chained frames in the environment. So a free identifier should be resolved normally in outer frames.

Dmitry.

Contributor

StanAngeloff commented Feb 10, 2011

Yeah, I don't see how it makes sense to require outer for shadowing, but keep everything else working as-is.

@StanAngeloff

Approach with outer is not for shadowing, but vice-versa to open the door for outer frames, since the assignment always creates a local variable. If the variable instead were marked as outer, then in the generated code no var statement is added for it.

In contrast, the approach with local or let is already for shadowing. It vice-versa for outer does generate var keyword for a variable. And assignment just works as assignment -- if the binding exists in the own frame -- it assigns to it. In other case -- it continues the lookup of the identifier in parent frames (if found, then assign, if not -- ReferenceError).

Dmitry.

Contributor

StanAngeloff commented Feb 10, 2011

That's what I meant by shadowing, i.e., it's explicit.

Not to repeat myself, but I'd be against such a change.

@StanAngeloff

Please explain then your meaning about issues I described above. What can you suggest to solve these issues? You said you're against the proposal, but you don't mention any word about solving the existing issue.

Dmitry.

Contributor

StanAngeloff commented Feb 10, 2011

Dmitry, I don't find the current implementation to have any issues. I don't look at it from the point of view you have developed. Therefore I can't explain or suggest anything as to me it all makes sense.

Repeat, a 3rd-party programmer should be able to use any names of local (to the black-box) variables. And at the same time another 3rd-party programmer should be able to reuse this function written by the first programmer -- and without reviewing the code of the function. It's the main principle of the abstraction which seems just broken in Coffee.

Keeping all your modules separated and compiling them individually, you'll never run in the above. Coffee's scoping rules are file-based (as you probably know, no doubt). Stuffing a lot of code in one file and accidentally breaking the black-box by overwriting a global is most likely intentional.

Anyway, it's how I see things. While you, satyr and akva, etc. have the same view, I am entitled to have my own as well ☻

@DmitrySoshnikov

I think I did not express myself well. Ins't about bugs, is about find the best design, so you can define the correct scope for each part of your project. When you have problems like this on some project is about bad design or not? I think it is and it seems to me the real problem is on the way the language was designed.

@StanAngeloff

I also don't think it is an implementation issue however it seems to me that it would be a natural improvement of the language.

@StanAngeloff

Stuffing a lot of code in one file

It doesn't matter what is "a lot of code" means in your/my view. This use case can be completely real and in the small code. Then direct question -- should I before using any local variable check the all code in the file you wrote? Don't take a huge file, let it be 100-300 lines. Should I?

@cairesvs yes I see your point and also think that the language should be designed so that even no unit tests are needed to program without bugs. Though, repeat, TDD is a good addition to avoid bugs regardless the exact language.

Dmitry.

Collaborator

TrevorBurnham commented Feb 10, 2011

I'm with Stan on this one; the status quo feels more intuitive to me than the addition of an operator or keyword to make explicit whether you're using a local-scoped or outer-scoped variable. The ethos of CoffeeScript, as I understand it, is:

  1. Shadowing is bad—a variable name should only mean one thing within a file. True globals (those that are shared among files) should be attached to window or global.
  2. Projects should be broken up into modular files, each wrapped in a closure to prevent scope leaks. A file should be perhaps 200 lines, tops.

Now, could the language be more consistent with these principles? Sure. Shadowing within a file could be eliminated entirely by having the compiler emit a warning, at the least, when a function argument (or, worse, issue 1121-type order issue) causes an outer variable (other than true globals) to be shadowed.

But for the most part, CoffeeScript the language seems well-aligned with its ethos. As long as you use a variable name to mean just one thing within a file, automatic scoping feels to me like the best possible system.

Also, I'm curious how the outer system would work in this situation:

do ->
  outer x = 1
  x = 2

Would that be a compiler error? Would it be inferred that both x's refer to the outer x? Or would the compiler change the name of the local x so that the var declaration wouldn't interfere with the outer x assignment?

Collaborator

satyr commented Feb 10, 2011

Projects should be broken up into modular files

Why then does it provide --join option?

@TrevorBurnham

Yes, I see the goal and the initial wishes of the chosen principles. And initially they really can be considered as improvements.

However, if you listen to written by you above principles, you'll see that they instead of convenience can sound just like limitations. In other words you say:

  1. There are only global (per module) variables. Your code will be issued with a warning in case of shadowing.
  2. CoffeeScript isn't good enough for writing more-less complex structures in one file. Even if your file will contain 3-5 procedures, they better should be small (and it's really better by the way) or else you risk to mess with variables. But in reality there can be really complex procedures which needs severl/many local variables.
  3. CoffeeScript is basically for programming for only one programmer in the project. In case you have several other programmers, they all should first find out which variable names are already borrowed.

Again -- yes, I agree that the basic wish to make the variable definition syntactically elegant (i.e. without any keyword) is a good wish. However, since JS/Coffee has model of chained environment frames there should be the way to distinguish free variables (that is variables which are not in the own frame) from the local variables.

Currently it's not possible. I still don't understand why in arguing you always mention only one programmer which works with the code. Why do you avoid another programmers which may support the code working in the same project.

And by looking on the following code:

foo = ->
  x = 10
  y = 20

it's not possible to say anything about x and y identifiers -- i.e. whether they are local or not. The programmer should return back and scan all the source first (okey, to use automatic search).

So I also want to make it more convenient and like the initial idea and principles, but at the same time I see the described above issue which breaks the principle of abstraction (though, the ideology "one name per file" already breaks it and seems expressly -- excluding nested scope concept).

I can assume that I can program in the current Coffee's implementation (regarding var definitions/mutations), but at the same time I want to patch the holes I see.

Dmitry.

Contributor

StanAngeloff commented Feb 10, 2011

I am sorry, I just find this amusing:

CoffeeScript isn't good enough for writing more-less complex structures in one file

Seriously?

it's not possible to say anything about x and y identifiers -- i.e. whether they are local or not.

I can achieve this using comments, without introducing any new keywords to the language:

# Foo does boo.
#
# @globals x, y
# @see my_app.coffee
foo = ->
  x = 10
  y = 20

I can assume that I can program in the current Coffee's implementation (regarding var definitions/mutations), but at the same time I want to patch the holes I see.

Great! You should definitely fork the project, add your patch and then send through a pull request. We can then actually have an implementation to look at and maybe, just maybe, we have a change or heart.

@StanAngeloff

Seriously?

I though it should went without saying that I rephrased from the other viewpoint your words. I don't want to turn the discussion into the demagogy. Moreover, I think this discussion becomes already noisy and less technical. But on your "seriously" -- I have a recent project on Coffee and manage it normally. It's not about my own inconvenience, it's about analyzing the design of the language.

I can achieve this using comments

You mean "I created a problem and then search the way how to fix this problem (using comments or something)".

I.e. just right after you start to find justifications of something, you already accept that there is an issue. Meanwhile in a well-design system there should be no such issues at all -- and no need to find justifications for anything.

You should definitely fork the project, add your patch

Yeah, maybe, will see.

and maybe, just maybe

Oh, it's a honor, I appreciated, thanks ;)

P.S.: repeat, I can program in current Coffee's way with variable definitions/assignments, I can find the way how to manage it efficiently (the way with comments also good). But what I do -- is analyze the design and mention the issues I see. It's not about me and my convenience, but the wish to avoid possible problems.

Dmitry.

Owner

jashkenas commented Feb 10, 2011

Despite this issue's tendency to break out in flames -- strict lexical scoping is very much a core principle of CoffeeScript, and it's a great issue worth discussing.

Dmitry raises a couple of specific criticisms: First, that the lack of shadowed variables "breaks the principle of abstraction", because 3rd party code can't be blindly copied-and-pasted into the middle of your source. Second, that CoffeeScript is inconsistent, because function parameters do give you a way to shadow variables. He then proposes a change: tagging either local variable with a var keyword, or closed-over variables with an outer keyword, to distinguish between the two.

I'd like to persuade y'all that strict lexical scope is a defining feature of CoffeeScript -- a massive improvement over the manual var-tagging of variables, and similar in spirit to the notion of structured programming. ... Think of it as "structured variable naming".

We all know that dynamic scope is bad, compared to lexical scope, because it makes it difficult to reason about the value of your variables. With dynamic scope, you can't determine the value of a variable by reading the surrounding source code, because the value depends entirely on the environment at the time the function is called. If variable shadowing is allowed and encouraged, you can't determine the value of a variable without tracking backwards in the source to the closest var variable, because the exact same identifier for a local variable can have completely different values in adjacent scopes. In all cases, when you want to shadow a variable, you can accomplish the same thing by simply choosing a more appropriate name. It's much easier to reason about your code if a local variable name has a single value within the entire lexical scope, and shadowing is forbidden.

So it's a very deliberate choice for CoffeeScript to kill two birds with one stone -- simplifying the language by removing the "var" concept, and forbidding shadowed variables as the natural consequence.

This brings us to Dmitry's second point: It's still possible to shadow with parameter names, because of the nature of JS functions. I think that Trevor has the right idea here, we should be more strict about shadowing instead of less. It would be great to entertain tickets that either make parameter shadowing a syntax error, or a compile time warning. If we ever go down the road of having a coffee --warn, it should be one of the first rules.

Finally, the arguments about strict lexical scope making CoffeeScript a one-programmer-per-project language are total baloney, in my opinion. Accidentally clobbering an outer variable in a nested function is certainly possible ... but accidentally shadowing an outer variable is just as likely, and can also break your code. If I try to use a top-level define'd variable, in a nested function, but you've shadowed it, I'm hosed. And having strict lexical scope makes it much easier for me to determine what exactly has gone wrong, as opposed to "hunt for the var".

Strict lexical scope isn't going to change in CoffeeScript proper, but I encourage you add outer to your own dialect, or take a look at Coco, which includes two different kinds of variable assignment.

Re-Closing the ticket.

odf commented Feb 10, 2011

In all cases, when you want to shadow a variable, you can accomplish the same thing by simply choosing a more appropriate name. It's much easier to reason about your code if a local variable name has a single value within the entire lexical scope, and shadowing is forbidden.

But that's just the thing. People make errors all the time, and at present, there's no way for the compiler to tell whether someone accidentally re-used a name from the including scope. I support the idea of issuing compile time warnings on parameter shadowing, but I think it would be even more useful if the programmer could indicate that they would like to use a name locally and assume that it's not taken, so that the compiler could issue a warning if they're wrong.

The do construct could be such a way, and I for one would be perfectly happy if it were the only one. I think outer is a very bad idea, because it would make writing closures extremely clumsy, and the proposed let or := would be just as bad as sprinkling the code with var statements. I like how do confines the new bindings to a well-defined place, but the problem at the moment is that if a name already exists in the including scope, then the compiler silently assigns to that outer variable.

I'll continue this on #960.

Owner

jashkenas commented Feb 10, 2011

odf: Sure, in theory. In practice, well structured JavaScript code doesn't litter the top-level scope with lots of global variables, and keeps function scope shallow. This problem literally never comes up. And in the rare cases it does -- it's the same as when you try to reuse a variable name inside of a deep if/else statement -- you think "oh, I need to use a different name for this" ... and do just that.

Collaborator

satyr commented Feb 10, 2011

Join the club--it's much easier to fork it than convince its creator. ;)

Owner

jashkenas commented Feb 10, 2011

Yep -- and that's great. The reason why the entire source code is annotated is to make it easy for folks to modify the language to suit their fancy.

@jashkenas
But if anyone does coffee script work with let or local/outer or even := you'll accept the pull request?
Doesn't look like an issue however would be nice coffee script accept something like that.

Owner

jashkenas commented Feb 10, 2011

Absolutely not -- what I was trying to make clear above is that CoffeeScript's strict lexical scope is an important feature ... having to distinguish your variables with let/local/outer/var/:= would be a nasty step backwards.

odf commented Feb 10, 2011

jashkenas: I agree with you, but as others have pointed out, even though it's a rare problem, it could lead to really hairy bugs, especially when cutting and pasting code around as Dmitry mentioned. I don't know if it would justify introducing a new language construct, but since we already have do, why not try to make it do the right thing? (No pun intended.)

satyr: Agreed! I haven't gotten around to it yet, but I'll definitely fork and fiddle. :)

@odf
Looks like jashkenas is trying to show us is the all the alternatives we suggest or show up on this discussion could break the way lexical scope is made on CoffeeScript.
To avoid the problems we talk about it in this discussion we could using some testing and letting the global scope clear.
Right, jashkenas?
Any other idea to solve this problem without changing the way CoffeeScript is?

Owner

jashkenas commented Feb 10, 2011

cairesvs: That's not at all what I'm saying.

I'm saying that there's not a "problem" here -- there's a major CoffeeScript feature. Certainly, you have to be more aware of your variable names than you would otherwise ... but the bonus side is that with better naming, you can entirely avoid the concept of var -- and at the same time, your code gains the quality of lexical consistency: variable means variable ... throughout it's entire lexical scope.

There's no problem to be solved, here.

jashkenas
I didn't express myself well, sorry. I understand there is no issue. Please read my older answers.
Seems to me we talk about improvement not to solve a bug.

odf commented Feb 11, 2011

cairesvs: The do construct does not change the way the language works. It just is - or should be, in my opinion - syntactic sugar for a common javascript idiom. Here's what I mean:
a = 1; b = 2

((a, b) ->
  console.log "inner: a = #{a}, b = #{b}"
)('a', 'b')

console.log "outer: a = #{a}, b = #{b}"

prints
inner: a = a, b = b
outer: a = 1, b = 2
This works fine, and with the proposed compiler check for parameter shadowing, we'd get a warning or error since the inner a and b shadow the outer ones. It's just clumsy and hard to read, so it would be nice if instead of the three inner lines, we could write
do (a = 'a', b = 'b') ->
console.log "inner: a = #{a}, b = #{b}"
and get exactly the same javascript and the same compiler checks. That's essentially what's been proposed in #960 (if I read it correctly), and as I've said before, I think it would be completely sufficient to solve any potential (or imagined, as some may think) problems.

OK, thanks everyone for the discussion, it cleared some things in the reasons of accepted initially design. Some notes below.

The issues I mentioned still stands. One of which (not mentioned here): in case if a name appears before the function - there is "one scope", and there are several scopes of the same name if the name appears after the function:

foo = -> x = 10
x = 20

foo()
console.log x # 20, local x was changed

bar = -> x = 10

bar()
console.log x # 10 # global x is changed with the same code function

Although, it can also be argued as "complete and strict lexical scope" (first x definition appears inside the foo, second -- in the global, and third -- is assignment to global).

However, if there is only a reference to x from foo then it will be resolved in the global frame (this is btw, the difference from Ruby's closure which statically captures only existing at the moment bindings and produces an error in case of calling foo even if x will be defined in the global frame later; it's of course shouldn't be an issue for Coffee since it uses dynamic resolution of bindings anyway -- so it's just a small mention)

If I try to use a top-level define'd variable, in a nested function, but you've shadowed it, I'm hosed.

You mean, there is a global foo, then I shadow it in own function, and then you have another third, nested in my, function and think that you work with the global foo? That is:

foo = -> console.log 'foo'

myFunciton = ->
  
  let foo = 10 # I shadow it

  /* some (long?) code */

  yourFunciton = ->
    foo() # you try to call global `foo` and fail

Yes, it seems fair note, since you should analyze the code of myFunction and check whether some variables are shadowed -- i.e. the same as the copy-pasted block of code should be analyzed first on presence of used variables and checking whether some of them is already borrowed.

But the difference is -- the code of inner function is much less to check and if you write another inner function you should know the code of the surrounding one -- and this is not the same as to know the whole code of the file. So the issue with "imported" (copy-pasted) source stands and isn't baloney -- the user will check all the variables of the imported functions, and he will check in all outer scopes all presented (borrowed) names -- of both -- vars and functions, and he's forced to do this.

So, OK, my mission here was to mention the issues I see. Though, repeat, at the same time I also like the idea to define vars without any keyword.

P.S.: notice though, that even Ruby since 1.9 introduced the local scope with shadowing for the block arguments:

I.e. old semantics of 1.8 (which was considered as ugly design):

x = 10  
p x # 10
  
[1, 2, 3].each{|x|}  

p x # 3 ? Strict lexical scope?

And is fixed in 1.9 (though it's only for parameters, assignment to outer vars modifies them):

x = 10  
y = 20
  
[1, 2, 3].each{|x| y = x; z = x }  

p x, y, z # 10, 3, error

This seems corresponds to the current Coffee's way.

Though, consider also (about what I also mentioned above) -- methods in Ruby aren't closures and doesn't capture outer local vars, so that imported method can easily be copy-pasted:

# my code

a = 10 # global "local" var

def foo
  a = 20
end

# imported (copy pasted)

def bar
  a = 30
end

foo
p a # still 10

bar
p a # still 10

OK, let's leave it as is then. As I said, personally I can program in the current Coffee's way. Though, probably another fork will be good, yeah ;)

Dmitry.

Collaborator

satyr commented Feb 11, 2011

And is fixed in 1.9 (though it's only for parameters, assignment to outer vars modifies them)

Note that Ruby 1.9 also got a syntax for pure shadowing:
v = 1
tap{|;v| v = 2 }
p v # 1

@satyr

Note that Ruby 1.9 also got a syntax for pure shadowing:

Ha, I didn't know about it (since do not practice Ruby much today, and especially 1.9., though, I know its initial design). Thanks for the example -- which again shows that even in Ruby (from which the design of Coffee was borrowed) considered such an ability as needed ("needed" means not as "shadowing is good", "needed" means the ability to write this code by another programmer -- to keep the abstraction principle).

P.S.: Just another similar example to make it even clear:

x = y = 0            # local variables
1.upto(4) do |x;y|   # x and y are local to block
                     # x and y "shadow" the outer variables
  y = x + 1          # Use y as a scratch variable
  puts y*y           # Prints 4, 9, 16, 25
end
[x,y]                # => [0,0]: block does not alter these

Dmitry.

akva commented Apr 11, 2011

Thanks for the link @satyr. Appealing to authority is the last resort :)

Contributor

joliss commented Dec 23, 2011

(I realize the discussion is old and closed, but:)

Most of the time you access outer variables (i.e. from surrounding scopes), you only read them, don't you?

Python does something very slick to stop me from clobbering outer variables: Assignment implicitly makes a variable local. For instance:

def a():
    x # accesses outer/global x

def b():
    x = 42 # creates local variable x -- shadows outer x
    x # accesses local x

Localness is determined by static analysis for the entire scope:

def c():
    x # => UnboundLocalError: local variable 'x' referenced before assignment
    x = 42

In those rare cases where you want to overwrite the value of an outer variable (instead of shadowing), Python has the nonlocal and global keywords (outer has been proposed for CoffeeScript).

IMO this gives you the best of both worlds:

  • No accidental clobbering.
  • No declaration keywords sprinkled all over your code.
  • You can still write to outer scopes if you need to.

Thoughts?

Owner

jashkenas commented Dec 23, 2011

It's very tempting. Let's look at a reduced example of code that uses this style:

watch = (source, base) ->

  prevStats = null
  compileTimeout = null

  compile = ->
    clearTimeout compileTimeout
    nonlocal compileTimeout = wait 25, ->
      fs.stat source, (err, stats) ->
        return if prevStats?.size is stats.size
        nonlocal prevStats = stats
        compileScript()

I think that it's quite hard to make the case that the above is more readable or "better" than the original:

watch = (source, base) ->

  prevStats = null
  compileTimeout = null

  compile = ->
    clearTimeout compileTimeout
    compileTimeout = wait 25, ->
      fs.stat source, (err, stats) ->
        return if prevStats?.size is stats.size
        prevStats = stats
        compileScript()

... especially when the former breaks symmetry of access. It's just compileTimeout at first, and nonlocal compileTimeout in the line immediately below.

When prevStats and compileTimeout always refer to the same thing, this chunk of code is much easier to understand -- your suggestion is still introducing shadowing-by-default, when shadowing is what we're trying to avoid as much as possible.

jaekwon commented Dec 26, 2011

No changes to CoffeeScript is needed for the following convention for declaring local variables:

local = (fn) -> fn()

foo = 'a global!'

local (foo, bar) ->
  foo = 'a local!'
  console.log foo # a local!

console.log foo # a global!

CoffeeScript could be updated to define this local function by default.

Here's a contrived example...

{log, sin, cos, tan, PI} = Math

myCos = (theta) ->
  local (cos) ->
    cos = sin(PI/2 - theta)

console.log myCos(PI/3) # 0.5
console.log cos         # [Function: cos]

Also...

watch = (source, base) ->
  local (prevStats=null, compileTimeout=null) ->
    compile = ->
      clearTimeout compileTimeout
      compileTimeout = wait 25, ->
        fs.stat source, (err, stats) ->
          return if prevStats?.size is stats.size
          prevStats = stats
          compileScript()

The only gotcha is that you may accidentally call local incorrectly (though this can be checked during compile-time):

#correct: a space after `local`
local (foo) ->
    foo = 'bar'

#incorrect: may run if `foo` was declared to be a function in the outer scope.
local(foo) ->
    foo = 'bar'

Finally, you may not like the extra indentation, so you might be tempted to do...

watch = (source, base) -> local (prevStats, compileTimeout) ->
  compile = ->
    clearTimeout compileTimeout
    compileTimeout = wait 25, ->
      fs.stat source, (err, stats) ->
        return if prevStats?.size is stats.size
        prevStats = stats
        compileScript()

so maybe CoffeeScript could allow the following shorthand...

watch = (source, base ; prevStats, compileTimeout) ->
  compile = ->
    clearTimeout compileTimeout
    compileTimeout = wait 25, ->
      fs.stat source, (err, stats) ->
        return if prevStats?.size is stats.size
        prevStats = stats
        compileScript()
Contributor

aseemk commented May 21, 2012

Funny -- I ran into this problem when reading CoffeeScript's own source code.

Global variables defined here:
https://github.com/jashkenas/coffee-script/blob/1.3.3/src/command.coffee#L51-L57

Which makes it hard to tell in a given function whether an assignment is local or global.

It'd be great if assignments were always local, and there were an outer or global keyword (like Python) to signal that you want an assignment to affect the outer or global scope.

flynx commented Sep 16, 2012

there is a trivial way around this issue, it needs no change to the language, works just like Common Lisp's let and even doesn't look bad ;)

foo = 5

func = -> ((moo = 1, foo, boo = 5) ->
    foo = moo + boo
)()

alert func() + ' ' + foo

the only thing pissing from the language to make this really beautiful is Smalltalk-style code blocks :)

Collaborator

michaelficarra commented Sep 16, 2012

@flynx:

foo = 5

func = -> do (moo = 1, foo, boo = 5) ->
    foo = moo + boo

alert func() + ' ' + foo

Hi,

Cant we write var localvariable between `` before the assignment?

I think its better solution than (-> )() or do () -> etc.

trusktr commented Jan 6, 2016

Just go back to JavaScript: ES2015 modules, structuring/destructuring, arrow functions, classes... and let for your scoping needs! 👍 🎉

This issue was closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment