I don't get why this goes out of scope

[tigersoul925]

sharedwatchers.lua:

local testtimer = hs.timer.new(3,function()
  print("hello special timer")
end):start()

If requiring this module early in init.lua, the timer variable immediately goes out of scope and never triggers.
If requiring far down in init.lua, the timer variable stays in scope and keeps working. And I just don't get it as
there are NO other calls or anything going on relating to sharedwatchers within init.lua (or any other module).
If I make testtimer global, it doesn't matter where I put the require in init.lua.

Why is there a difference? Why would a local module-level variable ever go out of scope? I don't know if it's just me, but the scope of variables and GC in Lua is a chore to wrap your head around.

init.lua:
sharedWatchers = require("sharedwatchers")

[asmagill]

In a seeming paradox, the busier your lua code is (i.e. how much there is to evaluate), the quicker garbage collection occurs. This is why putting it later in your init.lua seems to work -- if you do more in the console or have a very active setup with lots of timers or callbacks, eventually it will be collected.

I discuss local variables and scope in this wiki article I wrote a while back... I need to go through it again to clean it up and add some more examples, before adding it to the official Wiki, but hopefully even in its current state, it will help your understanding: https://github.com/asmagill/hammerspoon/wiki/Variable-Scope-and-Garbage-Collection

[tigersoul925]

I think I've read that before (great document!) but I was under the impression that local variables in the MAIN section of a module would not go out of scope. Apparently they still do? I guess this means that variables in a module HAS to be made global one way or the other? Even if it can be by being referenced by a global variable, but just local blablabla = "hello" and not referencing that to anything just won't work then?

It's so weird because I've been doing just that in many places and this is the first time I've found it NOT working. It has worked for days and days, weeks even in other places.

What a mess 😂

Ok, sorry for digging, but I just don't get it as it worked for so long in other places. I took the same thing that I quoted above that will not work because I put the require at the top of init.lua, and this assumingly gets the testtimer GC'd quickly because of all the action that follows the require call in init.lua

I added this to sharedwatchers.lua:

function test()
   print(testTimer)
   testTimer:start()
   print("started it")
end

I reload, wait 3 seconds... nothing... the timer is apparently GC'd and dead. But... calling test() in the console shows:
hs.timer: not running (0x600002ac90f8)
and the timer happily starts, rather than tossing me a nil object error. This doesn't look like GC to me. I'm still puzzled ;)

EDIT: seems it's the referencing of testTimer in a GLOBAL variable that does it. Making the function local produces the same result: GC. So referencing a local variable in a global function is apparently another way to keep it alive. My guess here is that I've just happened to do this in the places it works.

[Rhys-T]

Yeah, referencing a local variable from a function makes that variable - and thus, its value - stay around as long as the function does, because it becomes one of the function's 'upvalues' (Lua reference, asmagill's document). This works regardless of why exactly the function itself is staying around.

I've run into issues as a result of this sort of thing often enough that I finally added a garbage collection pass to the very end of my init.lua, to make sure that if something is going to break, it does so immediately, while I'm still thinking about the code, rather than whenever garbage collection happens to trigger naturally. (Of course, it won't help for things created after init.lua is done - e.g. if a hotkey starts a timer and doesn't save it somewhere.)

---------- at the bottom of ~/.hammerspoon/init.lua: ----------

-- If I'm forgetting to save any watchers, timers, etc. into variables,
-- make sure they break as soon as possible.
-- I don't know if wrapping this in `doAfter` is really necessary or not -
-- I'm trying to make sure this runs after _everything_ the config does.
gcTimer = hs.timer.doAfter(0, function()
	gcTimer = nil
	collectgarbage()
	collectgarbage() -- second pass in case any objects were resurrected for finalizers the first time
end)

(Interesting side note: normally Lua's GC is smart enough to detect reference cycles and collect them anyway… but a timer seems to stay alive even if the only reference to it is in an upvalue on that same timer's callback. This happens for rather technical reasons¹, and doesn't seem to be documented, so you might not want to rely on it.)

Footnotes

1. As far as I can tell, what's happening is that C/Objective-C code - like, say, an NSTimer - has no way to directly hold a reference to a Lua value - like a callback function. So the callback gets saved in a global² Lua table, and the NSTimer just keeps a numeric index into that table. And since that table is global, Lua thinks there's a reference from outside the cycle. ↩

2. If you want to get even more technical, it's actually not stored with the normal globals, but in a special hidden table called the registry. Still basically global for GC purposes, though. ↩

[tigersoul925]

That's actually a great idea! The worst is having the realisation that you did a boo boo 30 minutes in and thus having no idea where the heck you did something wrong. It's so easy to do it, even when knowing exactly what you're going to do.

I think I get it now. Crazy how complicated this is. Thanks!