Protocols consolidation for releases #950

Closed
josevalim opened this Issue Apr 13, 2013 · 7 comments

Comments

Projects
None yet
3 participants
@josevalim
Member

josevalim commented Apr 13, 2013

Protocols main goal is to allow code extensibility. Athough we compile Elixir's built-in clauses to fast protocol dispatches, dispatching a protocol for a record goes through a slower process:

  1. The record name is converted to a list and it is verified it starts with Elixir-
  2. The generate the target implementation module by concatenating the Protocol name with the Record name
  3. We attempt to dispatch to the implementation module
  4. If the dispatch fails, we go to the fallback branch, which may be to invoke the Tuple implementation (after all, all records are Tuple), the Any implementation or raise an exception

This lookup is essential for the extension mechanism. However, in some cases, it may inflict a too big performance overhead. Based on this concern, we have received a couple proposals to improve this situation. One of those are outlined in #802, which allows us to inline implementations into the protocol. The problem with the proposal above is that it gives preference to the types known upfront, sacrificying the extensibility mechanism protocols are meant to bring in the first place!

This proposal intents to solve this problem without introducing side effects.

The release process

Erlang applications and projects are familar with the term "releases". When you bundle a release, your project is compiled and ready to be used in production. You can read more about releases on OTP's website.

In other words, when building a release, the software knows all protocols available and all of their implementations, meaning we can generate a protocol with a fast dispatch logic to all available implementations. This ensures protocol will be fast for both positives (a target exists) and negative cases (the target does not exist).

In terms of a code, the Protocol API would look like:

Protocol.consolidate(MyProtocol)

This code will look at all load paths, gather all implementations available and recompile the protocol using the fast dispatch clauses. If known up-front, the list of implementations could be given as argument:

Protocol.consolidate(MyProtocol, [Binary, List, SomeRecord])

This approach has two caveats:

  1. If a protocol is consolidated, code loading a new implementation for a new record to production servers won't work if a new protocol consolidation is not available
  2. Some people may want to define records and implementations at runtime, although very unlikely, they will probably want a way to disable this feature

Although Elixir doesn't ship with its own release tool yet, the available solutions would be rewriten to rely use this mechanism. By the time Elixir has its own release tool (planned for 0.9), we can integrate this behaviour as part of the release process.

@josevalim josevalim referenced this issue Apr 13, 2013

Closed

Inline protocol implementations #802

4 of 4 tasks complete
@ericmj

This comment has been minimized.

Show comment
Hide comment
@ericmj

ericmj Apr 13, 2013

Member

I really like this idea. I'm not really sure how this would work, but could caveat 1 be solved by an appup tool?

Member

ericmj commented Apr 13, 2013

I really like this idea. I'm not really sure how this would work, but could caveat 1 be solved by an appup tool?

@josevalim

This comment has been minimized.

Show comment
Hide comment
@josevalim

josevalim Apr 13, 2013

Member

@ericmj Both caveats could be solved somehow, that's why I called them caveats and not something like drawback. :) And for the first case, you are right, appups would catch it. :)

Member

josevalim commented Apr 13, 2013

@ericmj Both caveats could be solved somehow, that's why I called them caveats and not something like drawback. :) And for the first case, you are right, appups would catch it. :)

@meh

This comment has been minimized.

Show comment
Hide comment
@meh

meh Apr 18, 2013

Contributor

Can we have a helper to check if a record implements a protocol?

Contributor

meh commented Apr 18, 2013

Can we have a helper to check if a record implements a protocol?

@meh

This comment has been minimized.

Show comment
Hide comment
@meh

meh May 4, 2013

Contributor

Can we have additional non-protocol definitions in the defprotocol?

It could be helpful to have helpers or other things which are supposed to work with protocol implementing records but aren't supposed to end up in protocol implementations.

Contributor

meh commented May 4, 2013

Can we have additional non-protocol definitions in the defprotocol?

It could be helpful to have helpers or other things which are supposed to work with protocol implementing records but aren't supposed to end up in protocol implementations.

@josevalim

This comment has been minimized.

Show comment
Hide comment
@josevalim

josevalim May 4, 2013

Member

We can't because that would make consolidation impossible. We can only consolidate because Elixir has control of all the code inside the protocol, so we can recompile it as many times we want.

Member

josevalim commented May 4, 2013

We can't because that would make consolidation impossible. We can only consolidate because Elixir has control of all the code inside the protocol, so we can recompile it as many times we want.

@meh

This comment has been minimized.

Show comment
Hide comment
@meh

meh May 4, 2013

Contributor

Oh 🐼

Oh well.

Contributor

meh commented May 4, 2013

Oh 🐼

Oh well.

@josevalim

This comment has been minimized.

Show comment
Hide comment
@josevalim

josevalim Jun 15, 2013

Member

Note: this is also an opportunity to clean up @only and @except attributes.

Member

josevalim commented Jun 15, 2013

Note: this is also an opportunity to clean up @only and @except attributes.

@josevalim josevalim closed this May 26, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment