Skip to content

7ML7W Elixir Day 2 Controlling Mutations

Jump to bottom Edit New page
Richard Cooper edited this page Jun 7, 2018 · 7 revisions

The Meeting

Intros are made. Bread is dipped. Pringles are stacked. Drinks are drunk.

Time to get into Elixir Day 2 and learn all about macros.

@adzz is our resident Elixir expert (in that unlike the rest of us he does Elixir at work) so he volunteers to lead us through the material both physically and spiritually.

The chapter is mostly about how Elixir is a bit like LISP, and so it leads us towards writing Macros, which is a strange choice as @adzz tells us that the community preference is not to use macros unless you really need them. We wonder if, like the Elm chapter, the material in the book was written before the community had properly coalesced on this opinion.

We explore mix, as we're mostly rubyists we compare it to a combination of bundler, gem and rake all in one.

@h-lame: do all modern langs have a mix?

All: yes!! (mostly)

We agree that this seems good.

Go doesn't have one yet, but they're getting close to coming up with one. What it does have is gofmt. We noted that Elixir seems to complain about whitespace during compilation time so it has a gofmt baked in. @adzz shows us that there is a command in mix to format it which will fix compilation errors that might occur, but it will also fix formatting that wouldn't cause compilation errors. It's a mix of preferential vs. compilation-breaking rules.

@adzz: It's not community led like rubocop, but you can raise issues to change it. Our team is happy with it.

@marksweston: I found Elm's style weird, it seems it was optimised to minimise the git diff, like putting commas first on newlines. It wasn't optimised for human readability!

All: murmurs of agreement that this seemed bad.

@charlieegan3: In ruby you ship gems so you can run their rake tasks in production and in rust you do similar by shipping crates. Is that the same with elixir and mix?

@adzz: I'm not sure!

He then shows us a library of his called maybex so we can explore the mix stuff that a library would provide. We explore it for a bit but are sidetracked by his chrome extension for github, octotree, which is very shiny.

@h-lame: What's the difference between .ex files and .exs files?

@adzz: I don't know really. I think it's something like ex files are library code and exs files are scripts, but both contain the same kind of code.

@dkandalov: I think exs are compiled on the fly, but ex are compiled once?

@adzz: Yes, maybe. One thing I know is that you pretty much only use exs for test files, and ex for everything else.

The book next introduces us to structs, as the grown up sibling of the maps we saw in Day 1. Apparently using structs is very idiomatic in Elixir, for example, in Phoenix, Elixir's Rails-a-like, the models which come from a library called Ecto are based on structs.

We spend some time exploring the differences between structs and maps in the repl. Mostly we focus on how a map can have any keys, but the struct only has the defined keys.

We wonder why you'd use structs instead of maps and @adzz explains that structs are more robust and allow for some type checking. Using a struct lets us do more robust pattern matching and type-checking, and to automatically destructure the args in the method signature. For example:

def thing({a: a, b: b}) do
end

def thing(%Test{a: a, b: b}) do
end

def thing(arg = %Test{}) do
end

def thing(a, b) do
end

We wonder about defining structs where some keys have default values, but others don't as the book and in the repl so far we've only ever seen ones that do have default values. Turns out, the arguments to defstruct can be a list, as well as a map. So we can do the following to get a struct with only one key that has a default value:

defmodule Test do
  defstruct [:a, :b, c: ""]
end

@h-lame: So we have %{} for defining maps and %TypeName{} for defining structs. Does that mean they are the same?

@adzz: Yeah, basically, but they also have some metadata.

@h-lame: Is it like the metadata in lua on tables? can we mess with stuff?

@adzz: I think so, I can't remember what it is. {GOOGLING HAPPENS}

    %Test{}.__struct__

It seems to just contain the name of the type, but @adzz seems to remember that Ecto models have more in their metadata, so maybe you they are the same as Lua's table metadata. We're not sure, so we move on.

The book suggests structs are safer than maps, because if we have:

defmodule Foo do
  defstruct bar: 1
end

foo1 = %Foo{}

foo2 = %{bar: 1}

Then try to run %{ foo1 | baz: 2 } it will fail with a key baz not fond because baz isn't a key in the Foo struct. Whereas the book implies that doing the same with the map, %{ foo2 | baz: 2 } would work. Except when we try this in the repl, it also raises a key error with the map version. We reason that this might be a change since the book was written. @adzz reminds us that the pipe syntax is a short-hand sugar and there are other ways to update a map that don't fail. We accept this.

@adzz: A cool thing is that all functions live in modules.

@adzz points out that this can helps with the transition from other languages, particularly ruby, as you can kind of think of elixir modules as a bit like classes in other languages. They're not not, but it helps.

@marksweston: If all functions live in modules, where do things like put_in come from? Is there a global namespace?

@adzz: Yes. It comes from Kernel, which is similar to the default receiver for methods in ruby. It's implicit.

@dkandalov: Can you only define a module once?

@adzz: No. It might warn though if you redefine it, and you can break things if you redefine stuff.

We demo this in the REPL, but I didn't note it down, sorry!

@dkandalov: So there's a global scope for names, even if redefined?

All: oh :(

Now we're looking at the state machine in the book. The log function in particular trips us up because it seems weird to us that a function called log wouldn't really log something, but instead return a new video instance built by adding something to the log attribute of the video instance passed in.

We wonder if the function had a name like add_log_to it'd feel clearer to us. We also discuss that we're still thinking like OO programmers and remind ourselves that we can't write functions on a video instance, because they're just data containers. All functions live in a namespace and have to be given the data they're going to act on. Everything is immutable too, so calling even if we could call a function on an instance, it wouldn't be able to mutate itself so we'd still have to track the return values.

We hark back to __struct__ and Lua and wonder if we could do something special in there to let us call methods on the struct directly. We quickly decide we probably wouldn't want to though!

This brings us back to the differences between modules in elixir and classes in ruby. To build some simple geometry app in ruby we'd have multiple classes with their own implementation of an area function. Then we'd just call area on whatever instance we wanted to find out the area of, and it'd call the right implementation.

class Shape
  def area
    raise NotImplementedError
  end;
end

class Circle < Shape
  attr_reader :radius
  def area
    Math::PI * radius * radius
  end
end

class Square < Shape
  attr_reader :side
  def area
    side * side
  end
end

In Elixir though we'd use pattern matching or case statements to do the same thing:

defmodule Square
  defstruct [:side]
end

defmodule Circle
  defstruct [:radius]
end

defmodule Shape do
  def area(shape) do
    case shape do
      %Square{side: s} -> s * s
      %Circle{radius: r} -> :math.pi * r
    end
  end
end

We reason that these are both fairly elegant.

@charlieegan3: but it's hard in Elixir to add a new shape, like a Hexagon for example, because you'd have to open up and edit the area function? In ruby you'd just write a new class.

@adzz: Yes, but the flip-side of that is that it's hard to add perimiter to all shapes in Ruby because you have to open all the classes and add that method, but in Elixir you can just add a new permiter function and pattern match all the cases in one place.

Also protocols might be useful here (but we don't talk about those because it'd probably be spoilers for day 3).

We move on to more of the state machine in the book. In particular the state_machine function in VideoStore.Concrete.

def state_machine do # (2)
  [ available: [
      rent:   [ to: :rented,    calls: [&VideoStore.renting/1]   ]],
    rented: [
      return: [ to: :available, calls: [&VideoStore.returning/1] ],
      lose:   [ to: :lost,      calls: [&VideoStore.losing/1]    ]],
    lost: [] ]
end

@adzz: How'd we feel about this syntax? &VideoStore.renting/1 The name of the function is the namespace, function name, and the arity.

@h-lame: I find it weird, but understandable. Can you not have two methods with the same number of args? If you can then how do to distinguish between them using that arity syntax?

@adzz: well, maybe, yes, I think so? Let's find out:

We crack open the REPL once again and see what we can work out.

defmodule Test do
  def renting(thing) do
  end

  def renting(thing, other) do
  end
end

These are 2 different functions and it's obvious that &Test.renting/1 refers to the first version of renting and &Test.renting/2 is a reference to the last one. But...

defmodule Test do
  def renting(%Test{}) do
  end

  def renting(%Thing{}) do
  end

  def renting(thing) do
  end
end

These are all &Test.renting/1 and the correct one will be called depending on the type of the argument provided. We note that order matters when defining these functions, and if we'd put the untyped variant first, it'd be the one that is invoked even if we supplied a Test or Thing instance as the argument.

We wonder if this kind of multiple definitions code is actually sugar for writing a case version like:

defmodule Test do
  def renting(thing) do
    case thing do
      %Test{} -> # test variant
      %Thing{} -> # thing variant
      _ -> # otherwise
    end
  end
end

We're not sure, but it seems like a possibility which would explain the order as order also matters in the case statement.

We look at the state machine behaviour from the book and note how confusing it seems. In particular it takes us some time to parse out exactly what's going on in StateMachine.Behaviour. fire/3 does some stuff to call fire/2 which calls down to activate/2. We take a while to tease out exactly what it's doing to:

def activate(context, event) do
  Enum.reduce(event[:calls] || [], context, &(&1.(&2)))
end

In particular we call out &(&1.(&2)) as particularly impenetrable, but @adzz reassures us that it's not idiomatic. This kind of thing can be written less briefly and let us use named variables, not numbered ones, which would clear things up. We wonder if the book chose this syntax just to show how concise the language can be.

@richardcooper is calls always length one? This is just chaining all the possible functions in calls together. There's only one call in the code we've seen so far, so it'd be clearer if it was call and we only had to call a single function, without needing the reduce.

All: ahhhhh

The book and @adzz confirm that this kind of thing is something of a standard pattern in elixir code. Chaining methods and recursively calling them is confusing when written like this, but it's an idiomatic pattern.

We briefly look at ecto's changesets for validation to see how a similar pattern looks to an end-user. @adzz confirms that it's very powerful and yet easy to work with.

The VideoStore.Concrete implementation brings in StateMachine.Behaviour with an import and this leads us to explore the various different ways that we can use functions from other modules in Elixir.

  1. use the function by it's fully qualified name. e.g. StateMachine.Behaviour.fire instead of importing it so we can call fire directly.
  2. include it with one of three options:
    1. require this pulls all the code from another module into scope for this module so it can be called directly. e.g. as we've seen, we can call fire directly, not StateMachine.Behaviour.fire.
    2. import which does the same as require, but optionally lets us specify which functions we want to include, rather than the whole module. e.g. we could import only activate from StateMachine.Behaviour, and leave fire behind.
    3. alias this is a halfway house, it lets us shorten a long fully qualified module name to a single name, by default just the last bit. e.g. we could alias StateMachine.Behaviour and then we can call fire via Behaviour.fire instead of StateMachine.Behaviour.fire

In general though, idiomatic elixir seems to be to only import when we have to (e.g. to use a macro) as fully qualified functions are clearer.

The book gets into testing now so we talk about the ecosystem for elixir. The language defaults draw a lot from rails (unsurprisingly as the creator, José Valim was a rails core member). This means ExUnit is inspired by test_unit and minitest rather than rspec. It does have describe for nesting some tests, but it only allows one level of nesting.

We seem split on this being a good or bad restriction.

We ask if there is an rspec equivalent, exspec perhaps, but @adzz is unsure. He says that exunit is what everyone uses though, and it's pretty good so he's not felt the need to look further.

He does point out that it has strong opinions on mocks, which come from José. Specifically, that you should mock nouns not verbs. So mocks are objects, and you don't stub the function calls.

We don't explore this any more.

The book starts explaining macros and how we do metaprogramming in elixir.

@h-lame: In ruby, metaprogramming is a bit like macros in that you do have two distinct "times". There's the code that runs as you load the class, like a has_many on an activerecord model. This defines a bunch of new methods on our class. We can think of this as class evaluation time, which is similar to what the book calls macro expansion time. And then there's the code that runs when you actually run your program, the standard runtime. Some of the code in the runtime was put there at class evaluation time, like the methods added by has_many.

Of course, it's also totally different! In ruby we don't have quote to say here's some code, but treat it as data for now. Our best option for doing something like that is "here's a string, do an eval on it" and pulling a big sadface.

We're not entirely clear on what quote really does, so we start exploring it in the repl:

quote do 1 + 2 end
Macro.to_string(quote do 1 + 2 end)

The first line returns the AST of 1 + 2, and Macro.to_string can turn the AST back into a string of elixir code. But you can also write the AST directly you don't need to use quote to get it from some "real" code:

Macro.to_string({:+, [context: Elixir, import: Kernel], [1, 2]})

We talk for a bit about what unquote does. @adzz suggests we think of it a bit like string interpolation, but for passing from macro time to runtime. He demonstrates that quote and unquote aren't reverses of each other: e.g. that unquote(quote do 1 + 2 end) won't return 1 + 2 as code. For a start you can only use unquote when you're inside a quote so it's not even valid elixir to even try this.

@elenatanasoiu: so why call it unquote? this seems like asking for confusion.

All: ugh, yeah :(

We play around some more and @adzz shows us that what we unquote has to exist outside the quote. e.g:

quote do
  unquote(a)
end

breaks, but:

a="z"
quote do
  unquote(a)
end

doesn't. A light bulb goes off.

@h-lame: oh, so in a quote context, we use unquote to reach back outside of the quote to get something

All: aaaah!

Despite what we said about what you try to unquote, the code that you quote doesn't need to exist outside the quote. So in the following:

quote do
  a + b
end

a and b don't need to exist in the macro, they just need to exist at runtime when you actually call the quoted code.

@adzz points out that in elixir, macros are hygienic by default.

defmacro thing do
  a = "z"
  quote do
  end
end

This means that in the above snippet, a only exists during the macro and is not available to the quoted code unless you unquote it to pull it in. There are ways of letting a leak outside the macro without unquote, but it's painful and you shouldn't. This is in contrast to other macro-ish languages where the macros aren't hygienic by default.

@adzz: I think it's a modern lang thing, they've observed the problems in lisp et al and just decided not to do it to avoid a class of problems.

All: sage nodding

We're at the end of the first macro in the book, but we're out of time so decide not to plough on with the rest of the book and start to wrap up.

@adzz: Like metaprogramming, think about if you should use a macro before you think about if you could, as often a normal function will do just fine

@adzz then offers to show us an example of a macro that his team uses in production, one of only a few. It's what they call the maybepipe. It is used chain multiple functions together, but halt the chain and return an error if that is the result of any of the functions. The code is a bit hairy for us elixir-newbs, but we can see the benefit of collecting all your error handling in one place. It also exposes us to the idiom of returning a tuple with {:ok, result} for success or {:error, error} for failure. It's not baked in to elixir, but lots of libraries do something like this, which is why the maybepipe works so well for @adzzs team.

@charlieegan3: Rust does this, there's a talk called "Railway Oriented Programming" by Scott Wlaschin that's worth watching.

@adzz: The whole thing is a bit monady.

Done with maybepipe we talk about macros some more.

@richardcooper: in defmacro how does it know how the bits that are the name work? We've seen macros with names that look like method calls, defmacro foo(bar) do, but maybepipe was an infix operator like defmacro foo ~> bar do. How does that work? Can you just put anything in?

@adzz: No, they have to look like a function call, or be one of limited number of infix operators. It's not a free for all unfortunately.

@tuzz: It's interesting that you can't tell what's a function or a macro because they look the same.

@richardcooper: Maybe it doesn't matter?

@adzz: Not much, it really only matters for how you require it. Often if it is a macro there's a convention for doing use instead of require/import as use can do some stuff.

We reflect that the first macro we saw in the book used use, but also introduced the __using__ function to let us hook into when use is called for our macro and let us do stuff.

@h-lame: I thought using was a weird introduction for macros. It'd be like the first introduction to modules in ruby also talking about Module#included or Module#append_features. It's powerful, but it's pretty deep.

@adzz: Yeah, but you need to define __using__ or you can't call use. One thing you see people do in __using__ often is to locally redefine something to change behaviour only for that macro. Sort of like ruby refinements.

We agree that the macro stuff seems weird and advanced for Day 2. @adzz says that he doesn't often use macros in his day-to-day elixir work, and the community does suggest restraint with them.

@richardcooper: To be fair, the book is trying to show us what's different about each language, so it will show us the weird stuff that might get language nerds excited.

@charlieegan3: Are their langs where you do loads of macros?

All: shrug

@adzz: I've heard people say macros are hard to debug in clojure, but I've not seen that much pain in elixir.

@marksweston: Remember that there are often two audiences for language features. As a day-to-day web developer writing applications it's unlikely you'd need to, whereas if you're a library author, or writing a web development framework, it's a different story and you might use them a lot.

@adzz: Oh, yeah, phoenix probably does a lot of stuff with macros.

Some final thoughts

@adzz: Macros really make you think about code being compiled then run, even if it's at runtime, but there's still two steps to think about.

@charlieegan3: So this isn't in book, but I think it's interesting - if there are these two steps, what do you give your production environment when you deploy an app? Is it the complete source tree or some compiled thing?

@adzz: Well it runs on the erlang BEAM which is powerful, but ...

your note-take wasn't following this well, sorry

@adzz: amazing tooling!

@marksweston: So it's always something running on the BEAM, we're not dealing with anything lower level than that?

@charlieegan3: If there was an exploit where someone could get filesystem access to your machine, would they be able to read your elixir source or just some binary or intermediate blob?

@adzz: ah, right, no, not sure

We agree this is an exercise left for the reader.

@charlieegan3: I was surprised that you had to write maybepipe, is there no maybe style thing in here already? It seems like an odd omission.

@adzz explains about the with statement which does allow us to do pattern matching on the return value of a function and decide what to do next. It also allows us to chain things.

with {:ok, foo} <- bar(arg),
     {:ok, baz} <- quz(foo),
    do: success(baz)
else
  {:error, badthing} -> IO.inpsect(badthing)
end

The difference is that with is general purpose, whereas @adzzs maybepipe was very specific about matching {:error, error} style things. Often that is what with is used for (as in our example above), but it doesn't have to be. You often see it inverted and used like a guard clause.

Wrap up

Many: It's been great having someone who knows the language quite a bit to drive and prod us in the right direction.

@tuzz: It's been a good format to jump between repl, book, examples, etc.

@dkandalov: Exploring tangents that were not really covered in the book was good too. We shouldn't be stuck on just the book as what to explore.

@h-lame: We didn't cover everything in the chapter tonight, but I got lots out of what we did cover. Not charging ahead is good.

@tuzz: I'll organise next time!

We leave, laden down with excess bread.

~fin~

Thanks

Thanks to @elenatanasoiu and @h-lame from Unboxed for hosting and providing beverages, to @h-lame, @charlieegan3 and @dkandalov for bread, dips and snacks and to @adzz for shepherding as our resident elixir expert.

Add a custom footer
Clone this wiki locally