Push the Button

Using LuaJIT + FFI + CoreBluetooth with an async design to automate a trivial task.

(Also published on Medium)

My office building has electronic window blinds. Push a button next to the light switches and they all go up or down. On most days, the afternoon sun hits just right to flood the office with a beautiful golden light that obliterates the contrast on my computer monitor. So someone gets up, walks over to the button, and lowers the blinds. My coworker bought a SwitchBot, which is a Bluetooth device with an app that will push a button when commanded, and affixed it to the wall of the office.

Now I won't need to get up if I either download the official app, or use one of the numerous open source projects. Instead, I used this frivolous problem as an excuse to delve into FFI bindings to Objective-C and CoreBluetooth from LuaJIT and to wrap Lua's native coroutines into something that looks like the async / await pattern from other languages.

I chose this path because I love writing code in Lua. The language is small and the simple syntax gets out of the way. When you stop thinking about classes and inheritance and instead embrace functions as first class values, you can stretch the language in really interesting ways. Besides being wildly fast, the LuaJIT implementation of Lua has a feature I view as an underappreciated superpower: the FFI interface. This allows you to directly call into native code, without having to compile a binding layer, without almost no performance penalty. FFI extends Lua's capabilities from what is supported in the limited standard library to anything that exists on your computer.

The result of this project is switchbot.lua. Run luajit switchbot.lua and the script will connect to a specific SwitchBot device and command it to push the button. Voilà! But a lot happens under the hood. Let's take a tour through how this all works.

Three Pieces

This project has a high-level control script with lower-level helper modules.

1. switchbot.lua: The main application logic that orchestrates the entire process.
2. objc.lua: A bridge to the Objective-C runtime, which is needed for controlling MacOS-specific features like CoreBluetooth.¹
3. async.lua: A lightweight library to handle asynchronous operations cleanly and flatten out callbacks into sequential code.²

Let's look at each one in detail.

switchbot.lua

This script contains the high-level logic. Its job is to:

1. Initialize the system's Bluetooth adapter.
2. Scan for the specific SwitchBot device.
3. Connect to it.
4. Discover the correct Bluetooth "service" and "characteristic" required to send a command.
5. Send the "press" command.
6. Disconnect cleanly.

The script defines the unique identifiers for the SwitchBot's services and the specific commands to send. The core of the script is the main function, which lays out the sequence of operations.

local main = a.sync(function()
    -- init and scan
    local delegate = makeDelegate()
    local central = a.wait(Ble:init(delegate))
    local peripheral = a.wait(Ble:scan(central))
    peripheral.delegate = delegate -- register for peripheral callbacks

    -- connect and get characteristic
    a.wait(Ble:connect(central, peripheral))
    local service = a.wait(Ble:discoverService(peripheral, CommandService))
    local characteristic = a.wait(Ble:discoverCharacteristic(peripheral, service, CommandCharacteristic))

    -- write command and wait
    a.wait(Ble:write(peripheral, characteristic, PressCommand))
    a.wait(Ble:sleep(delegate, 1.0)) -- wait for command to process

    -- disconnect and stop
    a.wait(Ble:disconnect(central, peripheral))
end)

Notice the a.sync and a.wait calls. This code looks synchronous, but each of those a.wait calls is a suspend point where the execution flow leaves the a.sync function and returns later to the same point when the task has been completed (e.g. the Bluetooth peripheral has been connected).

async.lua

Bluetooth communication is inherently asynchronous. You don't just call a function and get a result back immediately. Instead, you start an operation (like a scan) and then wait for the system to notify you with an event (like didDiscoverPeripheral). Handling this with traditional callbacks can lead to deeply nested, hard-to-read code often called "callback hell."

The async.lua module solves this by using Lua's native coroutines to create an async/await pattern, similar to what you might find in JavaScript or Python.

The module is built on the following concepts:

1. Coroutines: Lightweight, cooperatively scheduled threads. A coroutine can be paused (yielded) and resumed later with a value.
2. Futures: A "future" is a function that wraps a delayed computation. It has a uniform signature: function(cb) end. The future will not block when called, but instead call the provided callback (possibly with results as callback arguments) when the work is complete.
3. Tasks: The a.sync method wraps a function in a coroutine and returns an async task. This task is a future. Calling the future starts the coroutine.

This async work is created and managed using a few key methods:

1. a.sync(function(...) end)

This function creates a coroutine-based function that can be "paused" and "resumed." The returned async task is a function that matches the function signature for a future. This means it can be "awaited" like other futures. Any arguments passed to the base function are cached until the task future is called. Care is taken to handle nil values for the variadic list of function arguments and results.

The task future has an internal poll function, used as the future's callback function, that will recursively resume the coroutine until it is completed. Then the actual future callback function is called with any results. We are diligent to always return the invoked callback (even though it does not return anything) to finish the current stack frame and support proper tail calls. This allows for infinite recursion.

2. a.wait(future)

This function "pauses" an async function's execution while waiting for an asynchronous operation to complete. It's implementation is really simple, just coroutine.yield(future)! The yielded coroutine now waits for the async function's poll function to resume it with the results of a completed future. Since coroutine.yield returns the arguments supplied to coroutine.resume, we can do local val = a.wait(future).

3. a.wrap(function(..., cb) end)

This helper turns a traditional callback-based function into an "await-able" future. It caches any provided function arguments and returns a function that conforms to the future function signature (with just one callback argument). When called, the callback is passed to the original function. The only requirement is that the callback must be the last function argument. When this callback is called, the future completes.

In switchbot.lua, every CoreBluetooth operation, like Ble:scan or Ble:connect, is wrapped. When an async function calls a.wait(Ble:scan(central)), the coroutine yields. When the didDiscoverPeripheral delegate is eventually called by the system, it invokes a callback stored in an upvalue, resuming the coroutine, and returning the discovered peripheral inline.

4. a.run(future)

Kick off the execution of a future from non-async context. This method doesn't block, but instead depends on some other entity to eventually complete any pending futures by calling their completion callbacks.

This is a significant difference from the async / await implementations in other languages that's worth highlighting. Traditionally, an event loop is provided by the language. Async tasks are registered with the event loop. When the conditions are met for a particular task to run, the event loop "wakes" the task. It makes as much progress as possible, then yields back to the event loop. Because the event loop is also the work scheduler, it expects to be in control of execution. Any external libraries or frameworks need to integrate with the language's event loop and it cannot easily be swapped out.

If we consider the event loop with scheduler design from other languages a top down approach, then async.lua is a bottom up design. There is no integration between async.lua, futures, or even the Lua interpreter and the event loop. The only requirement is that futures get their completion callback called.

This allows for a bring-your-own-event-loop option, depending on the needs of the particular application. For example, switchbot.lua uses MacOS's native NSRunLoop. However, for a different application, no changes would needed to async.lua or high level tasks returned from a.sync to switch to an event loop like libuv or even a while-true loop with a series of file descriptors and a call to epoll.

To summarize, here's a complete execution trace through a particular async operation:

1. We create a main async task with local main = a.sync(function() end) and start running it with a.run(main()).
2. Within main, we call a.wait(Ble:scan(central)). This Ble:scan function returns a future.
3. a.wait yields to this future, pausing the main coroutine and passing the future back to the task.
4. The task takes the future and executes it, providing its own internal poll function as the callback.
5. The future tells the CoreBluetooth framework to start scanning. This happens in the background.
6. Sometime later, when a device is found, the didDiscoverPeripheral delegate method is called. This method eventually calls the callback provided in step 4.
7. The task's poll function is now executed. It takes the result (the discovered peripheral) and calls coroutine.resume, passing the peripheral back into the coroutine.
8. Execution inside main continues from exactly where it left off. The a.wait call returns the peripheral, and the script proceeds to the next line.

This model allows us to write asynchronous, event-driven code in a clean, sequential style, avoiding the complexity of nested callbacks entirely.

Note: There are a few useful async types, such as queues and channels that are part of async.lua, but are not used in switchbot.lua. I'm particularly chuffed with the channel type, that swaps out the send and receive methods with functions that either transfer a value or await a future, depending on which side gets to the channel first.

objc.lua

The final piece of this project is objc.lua. Lua doesn't have a built-in library for controlling Bluetooth. On MacOS, this is handled by the CoreBluetooth framework, which is written in Objective-C.

objc.lua uses LuaJIT's FFI library to interact directly with the Objective-C runtime API. This allows Lua to act as if it were a first-class Objective-C citizen, providing access to all the native capabilities of MacOS.

Here’s a deeper look at how it works:

1. Message Passing with objc_msgSend

The core of the Objective-C language is message passing. The syntax [myObject myMethod:arg] is compiled down to a C function call: objc_msgSend(myObject, "myMethod:", arg). The challenge is that objc_msgSend is variadic and has no fixed type signature. The types of the arguments and the return value depend on the method being called.

The msgSend function in objc.lua is a wrapper that solves this. When you call object:method(...) in Lua, it:

1. Looks up the Objective-C method on the object's class.
2. Uses Objective-C runtime API functions like class_getInstanceMethod, method_copyReturnType, and method_copyArgumentType to determine the Objective-C type signature.
3. Converts the Objective-C type signature into a C function signature string for LuaJIT's FFI on the fly (e.g. "id (*)(id, SEL, id)").
4. Casts the generic C.objc_msgSend function pointer to this specific, dynamically-created function signature.
5. Calls the now correctly-typed function with the provided arguments.

2. Metatables for Syntactic Sugar

The module uses ffi.metatype to attach __index and __newindex metamethods to all Objective-C objects. This is what translates natural Lua syntax into Objective-C message sends. When you write peripheral:discoverServices(...), the __index metamethod intercepts the access for "discoverServices", creates a function that wraps an objc_msgSend call, and executes it.

3. Creating Objective-C Classes in Lua

Additionally, objc.lua allows us to define new Objective-C classes and implement their methods with Lua functions. This is needed for handling delegate callbacks. switchbot.lua uses objc.newClass("CentralManagerDelegate") to create a new class and objc.addMethod(...) to attach Lua functions to it.

-- Create a new Objective-C class
local class = objc.newClass("CentralManagerDelegate")

-- Add a method to it, implemented by a Lua function
local scan_cb = objc.addMethod(class, "centralManager:didDiscoverPeripheral:advertisementData:RSSI:",
    "v@:@@@@", didDiscoverPeripheral())

-- Create an instance of our new class
local delegate = objc.CentralManagerDelegate:alloc():init()

When we pass this delegate object to the CBCentralManager, the CoreBluetooth framework holds a reference to a real Objective-C object. When a Bluetooth event occurs, the framework sends a message to our delegate object. The objc.lua bridge intercepts this message and invokes the corresponding Lua function (didDiscoverPeripheral), seamlessly bridging the gap between the two languages.

For switchbot.lua, didDiscoverPeripheral and the other CBCentralManager delegate methods are function factories, which generate cdata callback functions with stored upvalues for the future completion callbacks. When a new future is generated from a wrapped CBCentralManager Objective-C method, the cdata function's associated Lua function is replaced, e.g. scan_cb:set(didDiscoverPeripheral(cb)). This changes the dispatching of Objective-C's delegate method to Lua function, without having to update the Objective-C class.

Putting it all together

Now that we've written over 600 cumulative lines of code, it's time to push a button! There are two hardcoded constants for discovering the SwitchBot: the expected service data and the expected manufacturer data. Once a scan is started for nearby BLE devices, if we find a device advertising the expected service data, we know it is a SwitchBot, if it is also advertising the expected manufacturing data (which I believe includes the device serial number) we know it is our SwitchBot. CoreBluetooth does not provide access to a peripheral's Bluetooth address, so we need to find something uniquely identifying in the advertising data itself.

local function didDiscoverPeripheral(cb)
    return function(id, sel, central, peripheral, advertisement_data, rssi)
        local service_data = advertisement_data:objectForKey(CBAdvertisementDataServiceDataKey)
        local mfg_data = advertisement_data:objectForKey(CBAdvertisementDataManufacturerDataKey)
        if service_data and mfg_data then
            local data = service_data:objectForKey(ServiceDataUuid)
            if data and data:isEqualToData(ExpectedServiceData) == BOOL(true)  -- peripheral is a SwitchBot
                and mfg_data:isEqualToData(ExpectedMfgData) == BOOL(true) then -- peripheral is our Switchbot
                NSLog("Discovered the SwitchBot")
                central:stopScan()                    -- no more delegate callbacks
                return cb and cb(peripheral:retain()) -- complete the future with the peripehral
            end
        end
    end
end

Next we ask CoreBluetooth to discover services on the peripheral, searching for a specific service UUID. The same is done for a specific characteristic UUID. Once we have a handle to the required characteristic, we use it to write a static command for a button push action, then wait for the message confirmation.

I've discovered that if you do not remain connected until the SwitchBot's arm has finished moving, then the action is canceled. We use an async sleep function to wait for just a second. This is done by using a NSTimer to call a callback after a timeout. We conveniently reuse the single created custom class object as a delegate for both CBCentralManager and NSTimer.

Now we're done.

This project is totally over engineered for this trifling task. However, the core pattern of using a high-level scripting language for the main logic, a foreign function interface (objc.lua) to access platform-specific native APIs, and an async library (async.lua) to keep the code clean and manageable, is a valid one. What results is a simple main script with clear logic that can control a physical device in the real world, and save me from walking across the room.

---

Appendix: How to bundle Lua into a bytecode executable

I made a bundler script, named bundle.lua, to package an application and its dependencies into a single, easy-to-run file. It transforms our set of Lua files (switchbot.lua, async.lua, objc.lua) into one self-contained, executable LuaJIT bytecode file.

The process can be broken down into three main steps:

Step 1: Recursive Dependency Discovery

When you run luajit bundle.lua switchbot.lua, the script first reads the content of the entry point, switchbot.lua.

It then uses a regular expression to find all require("...") calls within that file. For each module it finds (like "async" and "objc"), it uses the standard package.searchpath function to locate the corresponding file on the filesystem (e.g., async.lua).

If the file is found, its content is read, and the process repeats. The bundler scans the content of async.lua for more require calls. This recursive process continues until every dependency has been found and its source code has been read into memory. This builds a complete map of all Lua modules needed to run the application.

Step 2: Assembling the Source with a Custom Loader

Simply concatenating the files isn't enough, because the require function is designed to look for files on disk. The solution is to hijack the require mechanism.

The bundler generates a single, large Lua script in memory that starts with a "pre-loader" section. For each dependency found in Step 1 (e.g., async.lua), it generates a small snippet of code like this:

package.preload["async"] = function()
  -- The entire content of async.lua goes here
end

The package.preload table is a special table in Lua that require checks before searching the filesystem. By pre-populating it, we are telling require: "When someone asks for the async module, don't look for a file. Instead, just execute this function I'm giving you."

The bundler creates these package.preload entries for every dependency and prepends them to the original content of switchbot.lua. The result is one large, in-memory source file that effectively looks like this:

-- Pre-load all dependencies
package.preload["async"] = function() ... end
package.preload["objc"] = function() ... end

-- Original main script
local a = require("async")
local objc = require("objc")
-- ... rest of switchbot.lua

When this combined script runs, the require("async") call will find the async key in package.preload, execute the associated function, and cache the result in package.loaded, perfectly mimicking the behavior of the real require but without ever touching the filesystem.

Step 3: Compiling to Bytecode and Packaging

This is the final step that creates the executable file.

1. Compilation: The entire in-memory Lua source string generated in Step 2 is compiled into LuaJIT bytecode using the string.dump() function. This function takes Lua source code, pre-compiles it into a series of binary op-codes and arguments, and returns it as a string.

2. Creating the Loader Script: The bundler prepends a small shell script "header" to the bytecode. This is a neat trick to make the file directly executable.

   #!/bin/sh
   tail -n +4 "$0" | luajit - "$@"
   exit

This shell script does the following:

• #!/bin/sh: This shebang identifies it as a shell script.
• tail -n +4 "$0": Takes its own file ($0), seeks to the 4th line, (skipping the loader script itself), and prints the rest of the file (the LuaJIT bytecode) to standard output.
• | luajit - "$@": Pipes that bytecode directly into the luajit interpreter. The - tells LuaJIT to read code from standard input, and "$@" passes along any command-line arguments.

3. Final Assembly: The bundler writes this shell header, followed immediately by the raw binary bytecode, into the output file (e.g., switchbot.ljbc). Finally, it makes the file executable using chmod +x.

The result is a single executable file that you can drop anywhere in your $PATH, without worrying about setting $LUA_PATH or package.path. This approach only bundles Lua source code and doesn't support native C modules. However, this is ok for switchbot.lua, since we're exclusively using the LuaJIT FFI for native code. The compiled bytecode file is also smaller than the original Lua sources:

❯ wc -c *.lua
 5345 async.lua
10256 objc.lua
10463 switchbot.lua
26064 total

❯ wc -c switchbot.ljbc
14554 switchbot.ljbc

Footnotes

1. Influenced and inspired by fjolnir/TLC and luapower/objc ↩

2. This module is an extended modification of shelbyd/lua-async-await ↩