Unable to keep global state across macro calls #903

Closed
gradha opened this Issue Feb 11, 2014 · 2 comments

Comments

Projects
None yet
4 participants
@gradha
Contributor

gradha commented Feb 11, 2014

Following the discussion at http://forum.nimrod-lang.org/t/97 I tried to use some string tables in my macros for state keeping, but failed:

import macros, strtabs

var x {.compileTime.}: PStringTable

macro addStuff(stuff, body: expr): stmt {.immediate.} =
  result = newNimNode(nnkStmtList)

  if x.isNil:
    x = newStringTable(modeStyleInsensitive)
  x[$stuff] = ""

macro dump(): stmt =
  result = newNimNode(nnkStmtList)
  for y in x.keys: echo "Got ", y

addStuff("Hey"): echo "Hey"
addStuff("Hi"): echo "Hi"
dump()

Attempting to compile that crashes:

Traceback (most recent call last)
nimrod.nim(87)           nimrod
nimrod.nim(55)           handleCmdLine
main.nim(308)            mainCommand
main.nim(73)             commandCompileToC
modules.nim(193)         compileProject
modules.nim(151)         compileModule
passes.nim(192)          processModule
passes.nim(136)          processTopLevelStmt
sem.nim(379)             myProcess
sem.nim(353)             semStmtAndGenerateGenerics
semstmts.nim(1290)       semStmt
semexprs.nim(819)        semExprNoType
semexprs.nim(1908)       semExpr
sem.nim(287)             semMacroExpr
vm.nim(1174)             evalMacroCall
vmgen.nim(1485)          genProc
vmgen.nim(210)           gen
vmgen.nim(1326)          gen
vmgen.nim(210)           gen
vmgen.nim(1326)          gen
vmgen.nim(210)           gen
vmgen.nim(1308)          gen
vmgen.nim(300)           genIf
vmgen.nim(205)           gen
vmgen.nim(1286)          gen
vmgen.nim(723)           genMagic
vmgen.nim(504)           genUnaryABC
vmgen.nim(215)           genx
vmgen.nim(1260)          gen
vmgen.nim(1045)          genRdVar
vmgen.nim(1032)          genGlobalInit
vmgen.nim(215)           genx
vmgen.nim(1255)          gen
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

I tried to rework the example a little bit:

import macros, strtabs

macro addStuff(stuff, body: expr): stmt {.immediate.} =
  result = newNimNode(nnkStmtList)

  var x {.compileTime, global.}: PStringTable
  if x.isNil:
    echo "Initializing buffer"
    x = newStringTable(modeStyleInsensitive)
  x[$stuff] = "a"
  for y in x.keys:
    echo "Got ", y
  echo "Did addstuff ", $stuff

addStuff("Hey"): echo "Hey"
addStuff("Hi"): echo "Hi"

But this has two problems:

  1. I can't access the internal x from different macros, so I would have to create a dispatcher macro sharing the global state.
  2. The compilation of that code produces the following output:
Initializing buffer
Did addstuff Hey
Did addstuff Hi

That output is missing the iteration through x's keys, which should render first a single Got Hey then that plus the second string. What am I doing wrong there? I tried then to use a plain table:

import macros, tables

var x {.compileTime.}: TTable[string, string] = initTable[string, string](4)

macro addStuff(stuff, body: expr): stmt {.immediate.} =
  x[$stuff] = ""
  echo repr(x)
  result = newNimNode(nnkStmtList)

macro dump(): stmt =
  for y in x.keys: echo "Got ", y
  result = newNimNode(nnkStmtList)

when isMainModule:
  addStuff("Hey"): echo "Hey"
  addStuff("Hi"): echo "Hi"
  dump()
  echo "Did run"

This time the compiler doesn't crash, but it seems nothing gets added to x and fails during the second call to addStuff:

([(seEmpty, nil, nil), (seEmpty, nil, nil), (seEmpty, nil, nil), 
  (seEmpty, nil, nil)], true)
stack trace: (most recent call last)
ma3.nim(6)               addStuff
lib/pure/collections/tables.nim(122) []=
lib/pure/collections/tables.nim(118) enlarge
lib/pure/collections/tables.nim(118, 13) Error: index out of bounds

The same example using runtime procs works fine.

@gradha

This comment has been minimized.

Show comment
Hide comment
@gradha

gradha Feb 11, 2014

Contributor

Ah, the issue with the tables module seems a dupe of #404. Using the default size of 64 works but the code doesn't seem to work very well. After changing it to this:

import macros, tables, sequtils

var x {.compileTime.}: TTable[string, string] = initTable[string, string](64)

macro addStuff(stuff, body: expr): stmt {.immediate.} =
  x[$stuff] = ""
  result = newNimNode(nnkStmtList)
  echo "Added ", $stuff

macro dump(): stmt =
  echo x.hasKey("Hey")
  echo x.hasKey("noHey")
  for y in x.keys(): echo "Got ", y
  result = newNimNode(nnkStmtList)
  echo "Stored ", x.len

when isMainModule:
  addStuff("Hey"): echo "Hey"
  addStuff("Hi"): echo "Hi"
  dump()
  echo "Did run"

Compilation of that shows:

Added Hey
Added Hi
false
false
Stored 2

So the keys iterator doesn't seem to work, and hasKey is not finding the values?

Contributor

gradha commented Feb 11, 2014

Ah, the issue with the tables module seems a dupe of #404. Using the default size of 64 works but the code doesn't seem to work very well. After changing it to this:

import macros, tables, sequtils

var x {.compileTime.}: TTable[string, string] = initTable[string, string](64)

macro addStuff(stuff, body: expr): stmt {.immediate.} =
  x[$stuff] = ""
  result = newNimNode(nnkStmtList)
  echo "Added ", $stuff

macro dump(): stmt =
  echo x.hasKey("Hey")
  echo x.hasKey("noHey")
  for y in x.keys(): echo "Got ", y
  result = newNimNode(nnkStmtList)
  echo "Stored ", x.len

when isMainModule:
  addStuff("Hey"): echo "Hey"
  addStuff("Hi"): echo "Hi"
  dump()
  echo "Did run"

Compilation of that shows:

Added Hey
Added Hi
false
false
Stored 2

So the keys iterator doesn't seem to work, and hasKey is not finding the values?

@bogen

This comment has been minimized.

Show comment
Hide comment
@bogen

bogen Aug 6, 2014

This works for me using a string sequence.
I'm using compile time variable to collect procedure names that will be added to a proc table.

import macros
import tables
import sequtils
import strutils

var huhx {.compileTime.} : seq [string]

macro huh (name: expr, body: stmt) : stmt =
  var sname = $name
  echo sname
  huhx = huhx & @[$name]
  return newProc (name = name, body = body)

macro huht () : expr =
  return parseStmt (
    "{" &
    join (huhx.mapIt (string, "\"" &
      it & "\": " &
      it & ",")) &
    "}.toTable")

huh boo1:
  echo "boo1"

huh boo2:
  echo "boo2"
  boo1 ()

huh boo3:
  echo "boo3"
  boo2 ()

boo3 ()

let ft = huht

ft ["boo3"]()
ft ["boo2"]()
ft ["boo1"]()

The above works both with the 0.9.5 release:
Nimrod Compiler Version 0.9.5 (2014-05-02) [Linux: amd64]
and today's git:
Nimrod Compiler Version 0.9.5 (2014-08-05) [Linux: amd64]

bogen commented Aug 6, 2014

This works for me using a string sequence.
I'm using compile time variable to collect procedure names that will be added to a proc table.

import macros
import tables
import sequtils
import strutils

var huhx {.compileTime.} : seq [string]

macro huh (name: expr, body: stmt) : stmt =
  var sname = $name
  echo sname
  huhx = huhx & @[$name]
  return newProc (name = name, body = body)

macro huht () : expr =
  return parseStmt (
    "{" &
    join (huhx.mapIt (string, "\"" &
      it & "\": " &
      it & ",")) &
    "}.toTable")

huh boo1:
  echo "boo1"

huh boo2:
  echo "boo2"
  boo1 ()

huh boo3:
  echo "boo3"
  boo2 ()

boo3 ()

let ft = huht

ft ["boo3"]()
ft ["boo2"]()
ft ["boo1"]()

The above works both with the 0.9.5 release:
Nimrod Compiler Version 0.9.5 (2014-05-02) [Linux: amd64]
and today's git:
Nimrod Compiler Version 0.9.5 (2014-08-05) [Linux: amd64]

@dom96 dom96 modified the milestone: 0.9.6 Aug 14, 2014

Araq added a commit that referenced this issue Sep 3, 2014

@Araq Araq closed this in dafa8cc Sep 10, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment