asychronous calls

[deb75]

Hello,

I would like to make some non blocking calls to lua functions. Up to now, the only way I figured it out to do is :

awful.spawn.easy_async("echo \"mymodule.myfunction()\" | awesome-client",
                                         function(stdout) ... end)

But this requires mymodule to be declared globally in the rc.lua.

How can I perform a call to mymodule.myfunction() without making it block awesome ?

Regards

awesome v4.2 (Human after all)
• Compiled against Lua 5.3.3 (running with Lua 5.3)
• D-Bus support: ✔
• execinfo support: ✔
• xcb-randr version: 1.6
• LGI version: 0.9.2

[psychon]

Up to now, the only way I figured it out to do is:

That's not non-blocking, I think. That's just "run this code later" and is roughly equivalent to gears.timer.delayed_call(mymodule.myfunction). (Run the function at the end of this main loop iteration)

What exactly do you mean with non-blocking? What is it that mymodule.myfunction is doing that blocks?

[deb75]

The mymodule.myfunction is run at startup and creates or updates a database stored in ram memory. This can take a few seconds. As far as I understood, this will block awesome from doing anything else until the function exits.

I would like to prevent this, as if the function was run in the background by awesome. I do not know if this is feasible.

[psychon]

Lua is single-threaded, so you cannot do something in a background thread. Anything that is run blocks awesome.

However, with GIO it is possible to do asynchronous file I/O, i.e. the code does "I want to read this file" and instead of blocking and waiting for the file contents, other stuff can run and the code continues when the file contents were read. I was asking for what exactly mymodule.myfunction does to see if this might help you.

Does this code already exist? Is it public somewhere and I could take a look?

[actionless]

see those for example:

https://github.com/awesomeWM/awesome/blob/master/lib/menubar/utils.lua#L364
https://github.com/awesomeWM/awesome/blob/master/lib/menubar/menu_gen.lua#L89

[deb75]

Hi, here is the code :

#!/usr/bin/env lua5.3

local lfs = require "lfs"
local PATH = require "path"

local exclude = {
   path = {".cvs", ".git"},
   suffix = {"aux", "pyc", "o", "out", "blg", "bbl", "toc", "elc", "tns"}
}

local function get_basename_suffix_from_file(file)
   if not string.match(file, ".*%.[^%.]+$") then
      return file, ""
   else
      return string.match(file, "(.*)%.([^%.]+)$")
   end
end


local function get_suffix_from_database(database)
   local suffix = {}
   for k, _ in pairs(database) do
      table.insert(suffix, k)
   end
   return suffix
end


local function is_suffix_excluded(path, exclude_list)
   for _, v in ipairs(exclude_list) do
      if string.match(path, "%." .. v .. "$") then
         return true
      end
   end

   return false
end


local function is_path_excluded(path, exclude_list)
   for _, v in ipairs(exclude_list) do
      if string.match(path, "/" .. v .. "$") then
         return true
      end
   end

   return false
end


local function insert(directory, basename, database, i)
   i = i or 1

   if i > #basename then
      if database[0] ~= nil then
         if database[0][directory] == nil then
            table.insert(database[0], directory)
         end
      else
         database[0] = {directory}
      end

      return database
   else
      local c = string.sub(basename, i, i)
      if database[c] == nil then
         database[c] = insert(directory, basename, {}, i + 1)
      else
         database[c] = insert(directory, basename, database[c], i + 1)
      end

      return database
   end
end


local function remove(directory, basename, database, i)
   i = i or 1

   if i > #basename then
      if database[0] ~= nil then
         for j = 0, #database[0]
         do
            if database[0][j] == directory then
               table.remove(database[0], j)
               break
            end
         end
      end
   else
      local c = string.sub(basename, i, i)
      if database[c] ~= nil then
         remove(directory, basename, database[c], i + 1)
      end
   end
end


local function insert_path(path, database)
   local directory, file = string.match(path, "(.*)/([^/]+)$")
   local basename, suffix = get_basename_suffix_from_file(file)

   database[suffix] = insert(directory, basename, database[suffix] == nil and {} or database[suffix])

   return database
end


local function remove_path(path, database)

   local basename, suffix = get_basename_suffix_from_file(file)

   if database[suffix] ~= nil then
      remove(directory, basename, database[suffix])
   end
end


local function fill(root, database, args)
   database = database or {}
   local count = 0

   for relpath in lfs.dir(root)
   do
      if not string.match(relpath, "^%.%.?$") then
         local abspath = root .. "/" .. relpath

         local target = abspath
         while target do
            if lfs.symlinkattributes(target) == nil then
               abspath = nil
               target = nil
            else
               if lfs.symlinkattributes(target).mode == "link" then
                  target = lfs.symlinkattributes(target).target
                  if not string.find(target, "^/") then
                     target = root .. "/" .. target
                  end
               else
                  target = nil
               end
            end
         end

         if abspath then
            if lfs.attributes(abspath) ~= nil then
               if lfs.attributes(abspath).mode == "file" then
                  if not is_suffix_excluded(abspath, args.exclude_suffix) then
                     database = insert_path(abspath, database)
                     count = count + 1
                  end
               elseif lfs.attributes(abspath).mode == "directory" then
                  if not is_path_excluded(abspath, args.exclude_path) then
                     database, countup = fill(abspath, database, args)
                     count = count + countup
                  end
               end
            end
         end
      end
   end

   return database, count
end


local function build(roots, args)
   roots = type(roots) == "string" and {roots} or roots
   args = args or {}
   args.exclude_path = args.exclude_path and (#args.exclude_path > 0 and args.exclude_path or exclude.path) or exclude.path
   args.exclude_suffix = args.exclude_suffix and (#args.exclude_suffix > 0 and args.exclude_suffix or exclude.suffix) or exclude.suffix

   local count = 0
   local database = {}

   for _, root in ipairs(roots)
   do
      database, countup = fill(root, database, args)
      count = count + countup
   end

   return database, count
end


local function query(basename, database, i)
   i = i or 1

   if i > #basename then
      return database[0]
   else
      local c = string.sub(basename, i, i)

      if database[c] ~= nil then
         return query(basename, database[c], i + 1)
      else
         return nil
      end
   end
end


local function query_file(database, file)
   local basename, suffix = get_basename_suffix_from_file(file)
   return database[suffix] and query(basename, database[suffix]) or nil
end


local function walk(database, expr, key)
   expr = expr or ".*"
   key = key or ""

   local res = {}

   if database[0] ~= nil and string.match(key, expr) then
      for _, v in ipairs(database[0])
      do
         table.insert(res, v .. "/" .. key)
      end
   end

   for c, d in pairs(database)
   do
      if not tonumber(c) then
         for _, v in ipairs(walk(d, expr, key .. c))
         do
            table.insert(res, v)
         end
      end
   end

   return res
end


local function filter(database, expr, suffix)
   local res = {}

   for _, suf in ipairs(suffix or get_suffix_from_database(database))
   do
      for _, v in ipairs(walk(database[suf], expr))
      do
         table.insert(res, v .. (#suf > 0 and "." .. suf or ""))
      end
   end

   return res
end


return {
   build = build,
   filter = filter,
   insert_path = insert_path,
   remove_path = remove_path,
   query_file = query_file,
   exclude = exclude,
   get_suffix = get_suffix_from_database
}

The module db can be used as follows :

db = require "db"

-- build the database
mydb = db.build("/some/path")

-- This returns all directories within "/some/path" where the file "note.org" is found
ls = db.query_file(mydb, "note.org")

-- This returns the absolute path of all files containing "note" within "/some/path"
ls = db.filter(mydb, "note")

Depending on "/some/path" the call to db.build can take some time. Given what you said I do not think I can avoid it.

[actionless]

yeah, it's the same pattern as going through filetree of menuitems, so you could somehow modify those menu-gen functions to work with your files instead of .desktop files

[warptozero]

Using a lua serializer like serpent you should be able to do something like this:

local awful = require("awful")
local serpent = require("serpent")

local mydb

local code = [[
    local function code()
        local db = dofile("/path/to/db.lua")
        return db.build("/path")
    end
    local serpent = require("serpent")
    io.write(serpent.dump(code(), { sparse = false }))
]]

local function callback(stdout, stderr)
    local ok, data = serpent.load(stdout)
    if ok then
        mydb = data
    end
end

awful.spawn.easy_async_with_shell("echo '" .. code .. "' | lua", callback)

This runs code() is a separate lua process then serializes the return table to stdout and then loads it again into awesome. Not ideal, but should work ok for a slow process that returns a table without functions.

You can debug it by placing some naughty notifications on code, stdout, stderr, etc. And maybe you have to play a little with the serpent options.