Conflict between Lua opcodes and the rest of Csound on Linux

[gogins]

In the system packages for Csound on Linux, the Lua interfaces for Csound and CsoundAC appear to be linked with the standard Lua shared library, which is OK.

However, the Lua opcodes plugin for Csound in the system packages for bionic beaver is also linked with the standard Lua shared library, which is NOT OK. The opcodes will not function unless they are linked with the LuaJIT library.

On my system (Ubuntu 18.04 "bionic beaver") the Lua opcodes are linked with:

mkg@bodhimandala:~/csound-extended$ ldd /usr/lib/csound/plugins64-6.0/libLuaCsound.so 
	linux-vdso.so.1 (0x00007ffd1df82000)
	liblua5.1.so.0 => /usr/lib/x86_64-linux-gnu/liblua5.1.so.0 (0x00007f91127b1000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f9112428000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f9112210000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9111e1f000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9111a81000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f911187d000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f9112be5000)

On my system the installed Lua libraries are:

/usr/lib/x86_64-linux-gnu/liblua5.1.so.0.0.0
/usr/lib/x86_64-linux-gnu/liblua5.1-c++.a
/usr/lib/x86_64-linux-gnu/libluajit-5.1.a
/usr/lib/x86_64-linux-gnu/liblua5.2-c++.so.0.0.0
/usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0
/usr/lib/x86_64-linux-gnu/liblua5.2.so.0.0.0
/usr/lib/x86_64-linux-gnu/liblua5.3-c++.so.0.0.0
/usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2.1.0
/usr/lib/x86_64-linux-gnu/liblua5.1.a
/usr/lib/x86_64-linux-gnu/liblua5.1-c++.so.0.0.0

The Lua opcodes must be linked wth libluajit-5.1.so instead of liblua5 in any form.

Or, they could be statically linked, which would probably be best in this case.

There is a workaround for users, namely to replace /usr/lib/x86_64-linux-gnu/liblua5.1.so.0 with a symbolic link:

liblua5.1.so.0 -> libluajit-5.1.so

[vlazzarini]

Why is this labelled as bug in Csound? It seems to be a packaging issue, which is a distro issue.
I can't see how this can be fixed in the Csound sources, but maybe I have not understood the problem.

[gogins]

It is a bug in both.

…

----------------------------------------------------- Michael Gogins Irreducible Productions http://michaelgogins.tumblr.com Michael dot Gogins at gmail dot com

On Mon, Nov 5, 2018 at 10:28 AM vlazzarini ***@***.***> wrote: Why is this labelled as bug in Csound? It seems to be a packaging issue, which is a distro issue. I can't see how this can be fixed in the Csound sources, but maybe I have not understood the problem. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1059 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGCDJZm1GEAQBsmDmFOaIdIS9vCDfmGyks5usFkBgaJpZM4YOqEz> .

[vlazzarini]

A bug where in Csound? CMake script?
If it's a bug in the distros you should open a ticket there.

[gogins]

Yes, of course in the CMake script.

…

----------------------------------------------------- Michael Gogins Irreducible Productions http://michaelgogins.tumblr.com Michael dot Gogins at gmail dot com

On Mon, Nov 5, 2018 at 1:59 PM vlazzarini ***@***.***> wrote: A bug where in Csound? CMake script? If it's a bug in the distros you should open a ticket there. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1059 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGCDJSqWa3hl64KwjqGFZuB7DOt01Tldks5usIp2gaJpZM4YOqEz> .

[fsateler]

The Lua opcodes must be linked wth libluajit-5.1.so instead of liblua5 in any form

Why? This should probably be documented somewhere.

Looks like cmake/Modules/FindLUAJIT.cmake is wrong in trying to use find_library.

[gogins]

@fsateler I believe you are correct, find_library does not work.

Mike Pall is not focused on LuaJIT any more. Others are maintaining it. I wonder if they have messed up the packaging.

I was only able to get things to work for me by manually fiddling around in /usr after installing Lua and LuaJIT.

There are old pieces of mine that I may wish to update or adapt that use LuaJIT, and for that, I can do this manual fiddling. I generally write new pieces in C++, JavaScript, or Common Lisp. So I am not personally going to pursue this issue.

[kunstmusik]

These opcodes sound like a candidate to be moved out from Csound core.

…

On Sun, May 26, 2019, 21:06 Michael Gogins ***@***.***> wrote: @fsateler <https://github.com/fsateler> I believe you are correct, find_library does not work. Mike Pall is not focused on LuaJIT any more. Others are maintaining it. I wonder if they have messed up the packaging. I was only able to get things to work for me by manually fiddling around in /usr after installing Lua and LuaJIT. There are old pieces of mine that I may wish to update or adapt that use LuaJIT, and for that, I can do this manual fiddling. I generally write new pieces in C++, JavaScript, or Common Lisp. So I am not personally going to pursue this issue. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1059?email_source=notifications&email_token=AAMMA65DPEFMGZYHDKO5FCLPXMX2RA5CNFSM4GB2UEZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWIRJCY#issuecomment-496047243>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMMA6YEBF6YUO2WAA6CVA3PXMX2RANCNFSM4GB2UEZQ> .

[gogins]

I do not object to removing the LuaJIT opcodes from the Csound repository. In fact, I will do it myself. However, I do not see fast enough progress towards a package manager that enables Csound to install extensions from the Internet. I find this troubling. In my view the long-term health of Csound, especially now that many extensions are externally maintained, depends upon the availability of a functional package manager. SuperCollider already has Quarks, a package manager for extensions written in the SuperCollider language. Pure Data already has a built-in package manager called deken. This includes binary externals, and organizes them in folders by architecture. Max, the main alternative to Csound, already has a built-in package manager that is advertised as handling packages containing a variety of data, including binary extensions. I have posted some concrete suggestions here: #1130 (comment).

…

----------------------------------------------------- Michael Gogins Irreducible Productions http://michaelgogins.tumblr.com Michael dot Gogins at gmail dot com

On Mon, May 27, 2019 at 7:58 AM Steven Yi ***@***.***> wrote: These opcodes sound like a candidate to be moved out from Csound core. On Sun, May 26, 2019, 21:06 Michael Gogins ***@***.***> wrote: > @fsateler <https://github.com/fsateler> I believe you are correct, > find_library does not work. > > Mike Pall is not focused on LuaJIT any more. Others are maintaining it. I > wonder if they have messed up the packaging. > > I was only able to get things to work for me by manually fiddling around > in /usr after installing Lua and LuaJIT. > > There are old pieces of mine that I may wish to update or adapt that use > LuaJIT, and for that, I can do this manual fiddling. I generally write new > pieces in C++, JavaScript, or Common Lisp. So I am not personally going to > pursue this issue. > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > < #1059?email_source=notifications&email_token=AAMMA65DPEFMGZYHDKO5FCLPXMX2RA5CNFSM4GB2UEZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWIRJCY#issuecomment-496047243 >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AAMMA6YEBF6YUO2WAA6CVA3PXMX2RANCNFSM4GB2UEZQ > > . > — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#1059?email_source=notifications&email_token=ABQIGJKRWNLJUAL3UK7Y56DPXPEIBA5CNFSM4GB2UEZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWJTU5A#issuecomment-496188020>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQIGJJ5MRWIGJK2PSWMCZLPXPEIBANCNFSM4GB2UEZQ> .

[rorywalsh]

Hi Mike. While a functional package manager will be great, for now, there is nothing stopping us from simply including all of these opcodes in the current installers. For example, the Csound-Pd library has its own release page. The AppVeyor installer build script can grab the binaries for inclusion in the installer. As long as maintainers provide release candidates for their libraries, users should have access to them via the standard installers.

[gogins]

I must respectfully but firmly disagree. The point of moving a lot of things out of the Csound repository was to reduce its complexity. That has worked, but also has created a situation which where many things are no longer as easily available. Doing as you suggest would make more things easily available again, but at the cost of re-introducing a good deal of the complexity. The only solution in the long term is to do as the maintainers of large systems such as Debian, node, Python, LaTeX, and so on have done and to create a DECENTRALIZED package system. There's a reason these systems have all adopted a similar approach, and Csound would be well advised to follow this pattern. In short: Individual projects maintain their own packages. The packages follow a standard format. There are tools to build binary packages from source packages. These tools work the same on all platforms and architectures. The package installer is built into the system, and can install, upgrade, or remove packages locally or from remote repositories.

…

----------------------------------------------------- Michael Gogins Irreducible Productions http://michaelgogins.tumblr.com Michael dot Gogins at gmail dot com

On Mon, May 27, 2019 at 10:02 AM Rory Walsh ***@***.***> wrote: Hi Mike. While a functional package manager will be great, for now, there is nothing stopping us from simply including all of these opcodes in the current installers. For example, the Csound-Pd library has its own release page. The AppVeyor installer build script can grab the binaries for inclusion in the installer. As long as maintainers provide release candidates for their libraries, users should have access to them via the standard installers. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#1059?email_source=notifications&email_token=ABQIGJJQ77POMPGT3VXKKHLPXPSVNA5CNFSM4GB2UEZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWJ4GUY#issuecomment-496223059>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQIGJOA6L2US6FQ2K26TRDPXPSVNANCNFSM4GB2UEZQ> .

[gogins]

I am going to remove the Lua opcodes from the Csound repository, and host them in the extended-csound repository.

[gogins]

Problems building the Emscripten directory for WebAssembly.

To fix the problems, it is necessary that the following code appear at the top of libcsound-worklet.js:

// Bug in Emscripten?
var tempDouble = 0.;
var tempI64 = 0;

[gogins]

I have merged the remove_luajit branch back into develop and am closing this issue.

Please note, the Emscripten problems are completely unrelated to removing LuaJIT, and turn up in this issue only because I tested the Emscripten build and it was not working. It works for me now with the current Emscripten SDK.

The fix was extremely simple, and implies that the current Emscripten SDK assumes that some variables such as tempDouble are defined in JavaScript ahead of loading the WebAssembly module.

One other fix was to remove the Pointer_stringify function, which has been removed from the Emscripten SDK, also from Csound code.

Other issues appear to have been red herrings.

[kunstmusik]

I didn't see anything in the commit regarding Pointer_stringify, but Emscripten is building here now too. The out_t changes seem relevant after Emscripten's change in 1.38.31. Glad that this is cleared up.

…

On Thu, May 30, 2019, 19:22 Michael Gogins ***@***.***> wrote: I have merged the remove_luajit branch back into develop and am closing this issue. Please note, the Emscripten problems are completely unrelated to removing LuaJIT, and turn up in this issue only because I tested the Emscripten build and it was not working. It works for me now with the current Emscripten SDK. The fix was extremely simple, and implies that the current Emscripten SDK assumes that some variables such as tempDouble are defined in JavaScript ahead of loading the WebAssembly module. One other fix was to remove the Pointer_stringify function, which has been removed from the Emscripten SDK, also from Csound code. Other issues appear to have been red herrings. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1059?email_source=notifications&email_token=AAMMA67MVTRH5SKKGODPK33PYB4U7A5CNFSM4GB2UEZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWT6ALQ#issuecomment-497541166>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMMA6Y3YLKL3K2VLKKQWILPYB4U7ANCNFSM4GB2UEZQ> .