New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

require('libraryName') #90

Closed
EToreo opened this Issue May 13, 2016 · 22 comments

Comments

Projects
None yet
4 participants
@EToreo
Copy link

EToreo commented May 13, 2016

Is there a way to have sol2 make available lua-written libraries for "include"?

For example, I have a library I would like to use called "differed". It consists of one file "differed.lua" but expects to be required: local deferred = require('deferred'). I would like a way to do something similar to lua.open_libraries on the file and have it available for require. Is this possible with sol2?

@ThePhD ThePhD added this to the Bike.Shed milestone May 13, 2016

@ThePhD ThePhD added the Bike.Shed label May 13, 2016

@ThePhD ThePhD self-assigned this May 13, 2016

@ThePhD

This comment has been minimized.

Copy link
Owner

ThePhD commented May 13, 2016

I have thought about making a lua.require( ... ); function. It seems like it would take key argument (the key for the module / whatever you want to open) and then the next argument would either be a script that returns or loads an argument or a file. Note that lua has something similar for the C API, but it does not take script files and the like. Potential API:

// essentially wraps up luaL_requiref
lua.require("woofers", open_woofers_library ); // opens using any kind of function

lua.require_script("woofers", "return { bark = 20 }" ); 
// or something, no idea how that would work
lua.require_file("deferred", "deferred.lua" ); // fictional

I have no idea how I'd make the other 2 work. In all honestly, I'd probably just call lua.open_libraries( sol::lib::require); and then do lua.script( stringkey + "require(" + filename + ")" ); internally, like the true plebeian I am.

@EToreo

This comment has been minimized.

Copy link

EToreo commented May 13, 2016

I would love to see lua.require_script("woofers", "return { bark = 20 }" );. Your suggestion about adding sol::lib::require doesn't work for me because I'm being very cautious about what I expose to my lua users. Anything that can cause any IO, or open up the potential for IO can't be given to them. Its a great suggestion otherwise.

@ThePhD

This comment has been minimized.

Copy link
Owner

ThePhD commented May 19, 2016

Question: what happens if this fails? Do we just toss an error?

@ThePhD

This comment has been minimized.

Copy link
Owner

ThePhD commented May 19, 2016

Nevermind, I'm a dumb. Of course we just lua_error out.

@EToreo

This comment has been minimized.

Copy link

EToreo commented May 19, 2016

Lol, sounds good to me. I can't imagine it failing for too many reasons.

@ThePhD ThePhD closed this in 9f019ae May 19, 2016

@ThePhD

This comment has been minimized.

Copy link
Owner

ThePhD commented May 19, 2016

You're all set:

    std::string code = "return { modfunc = function () return 221 end }";

    sol::state lua;
    sol::table thingy1 = lua.require_script("thingy", code);
    sol::table thingy2 = lua.require_script("thingy", code);

    int val1 = thingy1["modfunc"]();
    int val2 = thingy2["modfunc"]();
    // val1 == val2 == 221
    bool reference_the_same_thing = thingy1 == thingy2; // true
@ThePhD

This comment has been minimized.

Copy link
Owner

ThePhD commented May 19, 2016

You can also require_file and just plain require. The plain one takes a function to call in case it needs to open the module.

@EToreo

This comment has been minimized.

Copy link

EToreo commented May 21, 2016

This is really cool and I will pick it up once I get sol compiling on windows.

@syed-ahmed

This comment has been minimized.

Copy link

syed-ahmed commented Aug 2, 2016

Hi, I think I have a similar issue. I need to run a torch/luajit script from a C++ program and have just started learning about lua C api and sol2 library. My script initially looks like:

require 'nn'
require 'cunn'
require 'cudnn'

model = torch.load("<path to serialized file>")
#take an input from C++
#do more stuff
#return an output from C++ 

Can you give an example how the 'require' portion of my script would work using sol2?
Thank you!

@ThePhD

This comment has been minimized.

Copy link
Owner

ThePhD commented Aug 2, 2016

I don't know where require when called from Lua goes looking, so I can't duplicate the functionality of it exactly. IIRC, it has code to load C libraries of certain names and also to check other local settings, so I am fairly unsure if I can give you an example. What are 'nn', 'cunn', and 'cudnn' ? Script files? C Libraries DLLs?

If you need exactly those libraries, try just running lua.script("require 'nn'");. I know this is not ideal, but since I don't exactly know what those are I can't offer anymore sound advice.

Note that even if sol2 doesn't do exactly what you want, if you do it yourself using the C API and find out how you can show me and I can spend some time working it into the library.

@syed-ahmed

This comment has been minimized.

Copy link

syed-ahmed commented Aug 2, 2016

Hi @ThePhD . Thank you for your reply. So these nn, cunn, cudnn are packages that gets installed by luarocks during the torch installation process. Luarocks calls 'make' on these folders: https://github.com/torch/distro/tree/master/extra. I am not sure where exactly these packages are lying. Here is the installation script: https://github.com/torch/distro/blob/master/install.sh

I don't know if that helps (i'm a beginner) but my end goal is to write a C++ command line interface that does the following:

---TODO:
---Write the lua C++ interface that passes an input tensor from c++ to lua and then fetches
---the output tensor from lua

1. Get input image from source
2. Feed processed image as a tensor in the lua program
    a. Get Input tensor
    b. Feed into model
    c. return output tensor
3. Get output tensor

An example that does something similar is the torch-android demo and it's relevant code is here. The torch android demo says

So I'm wondering if there is a way I can use package.loader in sol2 that can load the require 'nn', 'cunn' packages before my lua script is run.

@syed-ahmed

This comment has been minimized.

Copy link

syed-ahmed commented Aug 2, 2016

Here's an error I get when I compiled the C++ program loading a lua script with those require 'nn', require 'cunn' lines: attempt to call global 'require' (a nil value), when I just use lua.script("require 'nn'");.
Thinking if there is a to load 'nn', 'cunn' packages like you do in lua.open_libraries(sol::lib::base)

@Nava2

This comment has been minimized.

Copy link
Contributor

Nava2 commented Aug 2, 2016

Here is how I load libraries:

sol::state lua;
        lua.open_libraries(sol::lib::base, sol::lib::package);
        const std::string package_path = lua["package"]["path"];
        lua["package"]["path"]
            = package_path + (!package_path.empty() ? ";" : "") + test::scripts_path("proc/valid/") + "?.lua";

The lua["package"]["path"] variable is what the Lua engine uses to look for scripts or files. You'll need to adjust the path to look for your libraries in the LuaRocks packages.

@syed-ahmed

This comment has been minimized.

Copy link

syed-ahmed commented Aug 2, 2016

Hi @Nava2 , thanks for your reply. I did a luarocks path and got the following:

export LUA_PATH='/home/ubuntu/.luarocks/share/lua/5.1/?.lua;/home/ubuntu/.luarocks/share/lua/5.1/?/init.lua;/home/ubuntu/torch/install/share/lua/5.1/?.lua;/home/ubuntu/torch/install/share/lua/5.1/?/init.lua;./?.lua;/home/ubuntu/torch/install/share/luajit-2.1.0-beta1/?.lua;/usr/local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua'

export LUA_CPATH='/home/ubuntu/.luarocks/lib/lua/5.1/?.so;/home/ubuntu/torch/install/lib/lua/5.1/?.so;/home/ubuntu/torch/install/lib/?.so;./?.so;/usr/local/lib/lua/5.1/?.so;/usr/local/lib/lua/5.1/loadall.so'

I browsed to /home/ubuntu/torch/install/share/lua/5.1/?.lua and found the .lua scripts for all the 'nn', 'cunn' packages installed by lua rocks. Can you help me modify your lines using these path? I don't quite understand how the lua["package"]["path"] is working.

@Nava2

This comment has been minimized.

Copy link
Contributor

Nava2 commented Aug 2, 2016

Does this help?

@syed-ahmed

This comment has been minimized.

Copy link

syed-ahmed commented Aug 2, 2016

@Nava2 Sorry, that didn't really help. Like I understand in my case, it could be package.path = package.path .. ';/usr/local/share/lua/5.1/?/?.lua'? So integrating with your code, should it be:

const std::string package_path = lua["package"]["path"];
        lua["package"]["path"]
            = package.path .. ';/usr/local/share/lua/5.1/?/?.lua';
@Nava2

This comment has been minimized.

Copy link
Contributor

Nava2 commented Aug 3, 2016

Close, you'd want to do:

lua["package"]["path"] = package_path + ";/usr/local/share/lua/5.1/?/?.lua";
@syed-ahmed

This comment has been minimized.

Copy link

syed-ahmed commented Aug 10, 2016

@Nava2 Sorry for a late reply. Was working on something different. I tried your suggestion. However, it loads the exact same paths as this line does

lua.open_libraries(sol::lib::base, sol::lib::package);

Eitherway, I get the following error:

terminate called after throwing an instance of 'sol::error'
  what():  lua: error: caught (...) exception
Aborted (core dumped)

The error specifically happens in the lines - require 'nn', require 'cunn', require 'cudnn' - in the lua script. I have made sure the path for those modules are lib package. Any help is appreciated.

@Nava2

This comment has been minimized.

Copy link
Contributor

Nava2 commented Aug 10, 2016

What does the lua code print(package.path) print if you run it after you add the extra package_paths?

Could you show your full (relevant) code?

@syed-ahmed

This comment has been minimized.

Copy link

syed-ahmed commented Aug 10, 2016

@Nava2
Here is my code for the lua script called avg.lua:

print (package.path)
require 'nn'
require 'cudnn'
require 'cunn'
require 'torch'

M = torch.load("/home/ubuntu/Desktop/model.net")

Here is the C++ code:

#include <iostream>
#include "sol.hpp"

int main() {

    sol::state lua;

    lua.open_libraries(sol::lib::base, sol::lib::package);
    const std::string package_path = lua["package"]["path"];
    lua["package"]["path"] = package_path + ";/usr/local/share/lua/5.1/?/?.lua";

    lua.script_file("avg.lua");

    std::string important_string = lua["M"];
    std::cout << important_string << std::endl;

}

And here's my build command:

g++-4.9 -std=c++14 -W -Wall -g -o torchwrapper torchwrapper.cpp -I/home/ubuntu/torch/install/include -L/home/ubuntu/torch/install/lib -lluajit

Following is the output of print(package.path) before I get the error:

/home/ubuntu/.luarocks/share/lua/5.1/?.lua;/home/ubuntu/.luarocks/share/lua/5.1/?/init.lua;/home/ubuntu/torch/install/share/lua/5.1/?.lua;/home/ubuntu/torch/install/share/lua/5.1/?/init.lua;./?.lua;/home/ubuntu/torch/install/share/luajit-2.1.0-beta1/?.lua;/usr/local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua;/usr/local/share/lua/5.1/?/?.lua
@Nava2

This comment has been minimized.

Copy link
Contributor

Nava2 commented Aug 10, 2016

And you've verified the package nn lives at: /usr/local/share/lua/5.1/nn/nn.lua?

@syed-ahmed

This comment has been minimized.

Copy link

syed-ahmed commented Aug 10, 2016

@Nava2 I actually don't have a /usr/local/share/lua/5.1 (it gets added with the extra package path). nn is living in /home/ubuntu/torch/install/share/lua/5.1/nn/init.lua

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment