Using libraries which add searchers/loaders from C++

[mothfuzz]

Hello!
I'm not sure if this counts as a bug or a feature request, but I discovered while trying to introduce fennel into my project that the behaviour of Lua's require is different from Sol's require (or perhaps even Lua's luaL_requiref).

When loading the fennel library, you have the option to invoke install() which will add its own searcher to package.searchers.
more info is here, under "Use Lua's built-in require function".

A minimal example of the code I'm using is here:

#include <exception>
#include <iostream>

#include <sol/sol.hpp>

int main(int argc, char *argv[]) {
    sol::state lua;
    lua.open_libraries(sol::lib::base,
                    sol::lib::package,
                    sol::lib::table,
                    sol::lib::math,
                    sol::lib::string,
                    sol::lib::io);
    try{
        sol::table fennel = lua.require_file("fennel", "resources/fennel.lua");
        fennel["install"]();

        //doesn't work...
        //lua.require_file("fenneltest", "resources/fenneltest.fnl");

        //does work
        //fennel["dofile"]("resources/fenneltest.fnl");

        //notably, does work
        lua.script("require('resources/fenneltest')");

    } catch(std::exception& e) {
        std::cout << e.what() << std::endl;
    }

    return 0;
}

where resources/fenneltest.fnl is simply the following:

(print "hello fennel!")

When I say "doesn't work", what I mean is that Sol's require function seems to be ignoring the searchers added by fennel, and is trying to load fenneltest.fnl as a lua script, leading to obvious syntax errors. Meanwhile, calling require from within Lua behaves as expected, passing it on to the fennel compiler.

If it makes any difference, I'm using CMake 3.29.3 with clang 17, and both sol2 and lua are installed from vcpkg. This is all on Windows.
I have also attached the fennel library I'm using.
fennel.txt

[mothfuzz]

Another interesting quirk, which has me completely stymied: if I add the 'resources' directory to package.path, and then change require('resources/fenneltest') to simply require('fenneltest'), it fails as it tries to parse a lua script instead of a fennel script.

Here is the code I am using to change package.path from C++:

auto package_path = lua["package"]["path"].get<std::string>();
auto resources_path = std::filesystem::current_path() / "resources";
lua["package"]["path"] = package_path
+ ";" + (resources_path / "?.lua").string()
+ ";" + (resources_path / "?.fnl").string();

I have no idea why simply changing the path would cause it to be interpreted as lua or as fennel, but it might be related to the above issue.