Elixir 1.7.* syntax backwards incompatability with 1.6* and earlier

[OvermindDL1]

Precheck

Initially reported at: https://elixirforum.com/t/elixir-1-7-4-backwards-incompatibility/17917/2

Environment

• Elixir & Erlang/OTP versions (elixir --version):

╰─➤  elixir --version               
Erlang/OTP 21 [erts-10.1.1] [source] [64-bit] [smp:6:6] [ds:6:6:10] [async-threads:1] [hipe]

Elixir 1.7.4 (compiled with Erlang/OTP 21)


╰─➤  elixir --version                                                                                                             1 ↵
Erlang/OTP 21 [erts-10.1.1] [source] [64-bit] [smp:6:6] [ds:6:6:10] [async-threads:1] [hipe]

Elixir 1.7.0 (compiled with Erlang/OTP 19)


╰─➤  elixir --version        
Erlang/OTP 21 [erts-10.1.1] [source] [64-bit] [smp:6:6] [ds:6:6:10] [async-threads:1] [hipe]

Elixir 1.6.6 (compiled with OTP 19)

• Operating system: Linux 4.4.0-134

Current behavior

Minimal code for iex:

quote do
  a <- b
  c -> d
end

Complete session in 1.7.4 (same in 1.7.0):

╰─➤  iex   
Erlang/OTP 21 [erts-10.1.1] [source] [64-bit] [smp:6:6] [ds:6:6:10] [async-threads:1] [hipe]

Interactive Elixir (1.7.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> quote do
...(1)>   a <- b
...(1)>   c -> d
...(1)> end
** (SyntaxError) iex:3: unexpected operator ->. If you want to define multiple clauses, the first expression must use ->. Syntax error before: '->'

Expected behavior

Complete session in 1.6.6:

╰─➤  iex
Erlang/OTP 21 [erts-10.1.1] [source] [64-bit] [smp:6:6] [ds:6:6:10] [async-threads:1] [hipe]

Interactive Elixir (1.6.6) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> quote do
...(1)>   a <- b
...(1)>   c -> d
...(1)> end
{:__block__, [],
 [
   {:<-, [], [{:a, [], Elixir}, {:b, [], Elixir}]},
   {:->, [], [[{:c, [], Elixir}], {:d, [], Elixir}]}
 ]}

Thus this is a backwards incompatibility without an Elixir 2.* version bump and it has as such broken the library at https://github.com/OvermindDL1/ex_core.git in a backwards incompatible way (and in a way in which it cannot emulate again without a syntax overhaul).

Backwards incompatible syntax changes should necessitate a major version bump, thus Elixir 2.*

[josevalim]

It is not a breaking change because the previous behaviour was never expected nor specified. do/end blocks should either have expressions OR clauses but never both. I will be glad to document this behaviour more clearly to avoid such confusion in the future but, in this case, I am afraid you were relying on accidental behaviour.

[OvermindDL1]

Well that sucks, I'm not sure how else I can make that so easily succinct and readable then... Might have to use a verbose word or change to a parsed string horror... >.>

[OvermindDL1]

Isn't it still backwards incompatible though as this was entirely useable syntax within macor's? Plus there is no syntax spec for Elixir as it is (which would have prevented such a mis-parsing to exist in the first place) thus whatever the compiler accepts into a macro is by all rights valid since we have no defined syntax spec?

[josevalim]

I don't think having a spec would have prevented the bug, because the behaviour was always clear to me when implementing this feature set, but the bug was still unnoticed for large periods of time. Maybe it would have been caught earlier though.

That said, I definitely wouldn't want all parser bugs to be considered features. Over the years we have improved how we spec our syntax, including the document changed above, so I would continue treading this path.

My suggestion here would be to replace those operators by any of our custom operators.

[OvermindDL1]

My suggestion here would be to replace those operators by any of our custom operators.

There are cases where that would fail as a direct replacement, necessitating parenthesis to disambiguate. -> was really nice with how 'quick' it was in the precedence.