RFC: type section macros

[alehander92]

The new forLoop macros seem to be a good fit, and I feel we can do something
similar for type sections. Currently one can't really generate code in an existing section with a macro which can be limiting:
even if you can work around it, it makes the macro invocation weird

e.g.

type
  A = B

..
variant E

type
  C .. E

I propose a mechanism where one can invoke a macro directly in a type section.

I find it useful to have a way for the macro to return code for the type section and code for the code section after the type section(e.g. for $ / == definitions)

one possible api

macro variant(e: TypeStatement): untyped = # we don't need the whole type section, I just want to express this is a typeSection macro
  # somehow return a tuple of NimNode of typeSection and codeSection or if not possible
  # can we have (untyped, untyped) 

and invoke it

type 
  A = B
  ..
  E = variant: # maybe we can skip the colon
    ..

Now, I can imagine more usecases, e.g. interface / protocol

The initial motivation I had was that I wanted to propose variant keyword syntax for types, but I know extending the core language is harder to justify(and having a good extension mechanism is probably more "Nim")

[mratsim]

Note that today you can attach pragma to fields, use macro on types within the type section or use the following syntax:

import macros, strformat, strutils

macro fill_enum_holes*(body: untyped): untyped =
  ## Fill the holes of an enum
  ## For example
  ## fill_enum_holes:
  ##     type Foo = enum
  ##       A = 0x00,
  ##       B = 0x10

  body[0].expectKind(nnkTypeSection)
  body[0][0][2].expectKind(nnkEnumTy)

  let opcodes = body[0][0][2]

  # We will iterate over all the opcodes
  # check if the i-th value is declared, if not add a no-op
  # and accumulate that in a "dense opcodes" declaration

  var
    opcode = 0
    holes_idx = 1
    dense_opcs = nnkEnumTy.newTree()
  dense_opcs.add newEmptyNode()

  # Iterate on the enum with holes
  while holes_idx < opcodes.len:
    let curr_ident = opcodes[holes_idx]

    if curr_ident.kind in {nnkIdent, nnkEmpty} or
      (curr_ident.kind == nnkEnumFieldDef and
      curr_ident[1].intVal == opcode):

      dense_opcs.add curr_ident
      inc holes_idx
    else:
      dense_opcs.add newIdentNode(&"Nop0x{opcode.toHex(2)}")

    inc opcode

  result = body
  result[0][0][2] = dense_opcs

fill_enum_holes:
  type Foo = enum
    A = 0x00,
    B = 0x10

macro bar(x: static int): untyped =
  result = getType(int64)

type Bar = object
  field0: bar(10)

{.pragma: baz.}

type Baz = object
  field0{.baz.}: int

A bit linked to nim-lang/Nim#6696 regarding pragmas/macro in type sections

[alehander92]

Nice stuff, thanks for the examples, I personally wasn't aware of the last two.

However, they don't solve the original usecase: ability to generate types in existing type sections with macros e.g.

type
  A ..
  
  Foo = fill_enum_holes..

[timotheecour]

IMO nim-lang/Nim#13830 is a strictly better proposal and fits the language more naturally, allowing these:

template skipIfJs(body): untyped =
  when not defined(js): body

macro allPublicFields(body): untyped = ... # adds `*` to each field

type
  Foo1 = object
    seq[Foo3] # don't break type section!
  Foo2 {.skipIfJs.} = object
  Foo2b {.skipIfJs.} = enum x1, x2 # works with existing syntax including enum etc
  Foo3 {.allPublicFields.} = object
    a, b: int
  Foo3 = ref Foo1

it's also more flexible: it allows generating 0, 1 or more type definitions from macro invocation, that gets pasted in the parent type section.

Example of a concrete application where this could be used: enummaps (refs timotheecour/fusion#1)

[metagn]

IMO it does not really matter what is done here anymore, for a couple reasons:

• Templates are pretty much always not powerful enough for it to matter whether or not you use nnkTypeDef or a unary type section
• Type sections are now accepted from macros which should work with recursion (this is not tested)
• The only problem _ + nnkStmtListType (beyond its scope semantics maybe not being documented) has is dealing with recursion, but it deals with it better than breaking the type section in the middle as proposed in macro pragmas in type section should apply to TypeSection, not TypeDef Nim#13830. Given how the type section sem is set up, this is the simplest way for it to be implemented anyway. With lazy symbol semantic checking #416, both solutions would work equally well. Breaking the type section would make it easier for macros to read the sem'd output, however they should handle the case that already is possible with nnkStmtListType anyway + the type section sem would have to be rewritten (again, doable with lazy symbol semantic checking #416).
• Accept object type node from macros Nim#19179 + Allow full commands and blocks in type sections Nim#19181 mean the variant: syntax is possible now but you can't use syntax exclusive to object/tuple/enum declarations inside them, and it probably doesn't work with recursion (solved with lazy symbol semantic checking #416 again). You can mitigate this by piggybacking off of var/let syntax. Or you can just roll your own syntax which you would do for ADTs etc.

The root of the discourse here is the only thing type sections do differently than var/let/const sections, which is recursion. And we seem to have a solution for this.

Edit: For the record I am not advocating for any specific solution in general. Either we don't have lazy symbols and all solutions have the same issues with recursion, or we do have lazy symbols and all of them have no issues. The choice between the solutions in either case is purely ergonomic, but currently it makes no sense to change anything IMO.

[Araq]

@metagn is correct.