Adds Kernel.defguard/1 macro to build guard-safe macros.

[christhekeele]

Closes #2469.

Public API

(Checks denote that documentation has been updated where necessary to the satisfaction of involved reviewers. Uncheck if they need work.)

Public functions modified

• Macro.validate/1 is now implemented in terms of Macro.validate/2
• Macro.expand(_once)/2 is now implemented in terms of Macro.expand(_once)/3

Public functions added

• Macro.validate/2 allows you to verify an AST tree against an additional custom validator function
• Macro.expand(_once)/3 allows you to provide option rewrite: true to perform the expansions in elixir_rewrite.erl
• Macro.validate_guard/1 allows you to check if an AST tree is guard-safe
• Macro.validate_guard/2 does so after a rewrite-expansion over the provided Macro.Env.t
• Macro.guard/2 prepares an AST tree to be used in a guard by handling the provided list of variable names differently inside and outside of guards

Public macros added

• Kernel.defguard/1 creates a guard macro that behaves correctly inside and outside of guards, provided the macro implementation is valid in guards

Public macros modified

• Record.is_record/{1,2} now uses defguard

How it works

The heart of this implementation is that it makes few hard-coded assumptions about what is allowed in a guard.

The criteria used to validate a guard is that:

• it can be rewritten into an expression
  • composed exclusively of calls to sequences allowed in guards
  • or special forms
    • except for special forms known to assert_no_guard_scope in :elixir_expand
• the arguments to those calls must be simple terms
• the expression must be one line long

The only hardcoded values in the algorithm are the list of special forms that assert_no_guard_scope. The rest uses erlang introspection and Elixir compiler helpers to inform the validation.

This is only a rough algorithm. It's still possible to create an invalid guard withdefguard that passes this test and is syntactically correct, but semantically invalid. For example, odd pins, using variables in map keys, other funny business. The errors in those guards won't be discovered until the macro that defguard generates is invoked, and the AST proves to be subtly invalid.

To get any further into those woods, we would have either have access to more metaprogramming predicates like those found in :erl_internal and :erl_syntax, or start emulating the low-level compiler translations of things like binary literals that expand into forms that are only valid erlang, but not valid Elixir, AST.

This is the best algorithm I could come up with that is

• simple (single extra expansion step, single validation function)
• accurate (no false negatives)
• helpful (has as few as possible false positives)
• future-proof (not too much hardcoded into the implementation)

Here are the semi-private erlang introspection and Elixir compiler calls I make:

In Kernel.defguard/1

• :elixir_utils.extract_guards/1 to pull out the guard definition in the macro

In Macro.validate_guard/1

• :erl_internal.guard_bif/2 to check for guard bifs
• :elixir_utils.guard_op/2 to more concisely make other erlang guard checks

In Macro.expand(_once)/3

• :elixir_rewrite.inline/3 to perform simple compiler rewrites of the AST in advance to check for validity
• :elixir_rewrite.rewrite/5 to perform more complicated ones that involve changing arities or changing AST forms

Tests

(Please uncheck any items that need better/more/any tests, or check off unchecked things you determine are sufficiently covered.)

For changed functions

• Macro.validate/1 tests still all pass
• Macro.expand(_once)/2 tests still all pass

For added functions

• Macro.validate/2 is indirectly tested by the above and below
• Macro.expand(_once)/3 is indirectly tested by the above and below
• Macro.validate_guard/1 is indirectly tested by the below
• Macro.validate_guard/2 explores 45 forms in guards, exercising Macro.validate/2 and Macro.expand(_once)/3 in the process.
• Macro.guard/2 is not really tested beyond usage

For added macros

• Kernel.defguard/1 is not really tested beyond usage

For changed macros

• Record.is_record/{1,2} tests still all pass

Errata: I have a rather lengthy explanation of how I reached this implementation here.

[christhekeele]

It occurred to me to try to raise the exact same error invalids guards do, but I don't really see any other calls to :elixir_error.form_error/4 from Elixir so I decided against it.

[christhekeele]

All the other guards that Elixir offers can be implemented with normal one liner macros, since they are relatively simple and don't need special unquoting, so only Record.is_record/{1,2} benefits from defguard.

[josevalim]

Thank you @christhekeele. After looking at the implementation, it seems it would be best if we could use Elixir's existing :elixir_expand pass instead of reimplementing everything in Elixir land?

[OvermindDL1]

Any thought of making defguard be a special syntax instead of a macro? Would open a lot more possibilities.

[ericmj]

@OvermindDL1 What do you mean by special syntax? If you are referring to making it support things like is_struct then that's a separate feature and I think @josevalim covered that in #2469 (comment).

[josevalim]

Plus defguard is not about allowing new constructs but about allowing the constructs available today. Supporting new constructs is a whole other discussion that should happen separately.

[OvermindDL1]

Ah I meant that it is worth adding a a 'defguard' if it brings something more powerful than just making a macro of a guard. For example is_record is already a macro, it reduces the code a bit but does not give any more ability.

[ericmj]

Ah I meant that it is worth adding a a 'defguard' if it brings something more powerful than just making a macro of a guard.

That's a fair opinion but not one that the core team shares. For me the main value of defguard is to allow developers to create functions that will work safely in guards, many developers do not fully know everything that is or is not allowed in guard expressions, but defguard will ensure that it works or it will raise.

[christhekeele]

@josevalim I think I shied away from : elixir_expand initially because I hoped that Macro.expand could do more for me and I wasn't sure of the scope of : elixir_expand.expand/2. Having gotten much more comfortable with reading src, though, I can see that it makes for a painfully cleaner solution that doesn't involve touching existing functions. :)

Working on that now.

[josevalim]

@christhekeele let me clarify one thing before you move on to do more work though. We still have not decided if we are going to merge this feature or not. :) It seems you are having fun though and we will be glad to review it regardless.

[christhekeele]

Yep, it's loads of fun and I have no expectations. :)