Lua 5.1 -> 5.2 fallout #32354
Comments
|
@DomT4 if you get the chance to look at this it would be great. |
|
@MikeMcQuaid On it. |
If I remove Lua, Lua51, Luajit & Luarocks and start completely fresh with It looks like it's ignoring the Lua51 directory installation with luajit in favour of looking at the Lua52 directory regardless. I'll take a deeper poke at this ASAP. |
|
Great, having the same problem right now. |
|
I've got a few solid ideas on this, and am working as quickly as I can to test them and issue a fix. |
|
Still working on this. I don't think I've done much else for the last 3 hours, heh. The failure seems to be that |
|
The failure isn't actually caused by I think. Every few hours I make a little progress only to go "Ah. I see. This problem is actually caused by a bigger problem. Okay then, Let's go look at that". |
|
Making some progress on this. Some. I'm tempted to push an interim 'fix' where I cull the Luajit & Lua51 options entirely from the formula till I can sort it out, just to avoid tangling anyone else up in the issue. FWIW I haven't managed to find any other conflicts yet, partly because there are extremely few formula that have the option of which lua they prefer to tap. This problem doesn't crop up if there's no option on which Lua you get to pick, because then the formula only looks for I considered renaming the |
|
Just so folks are absolutely sure, #32459 does not and is not intended to close this in any way, shape or form. I'm working on a significantly larger fix for this issue, a fix that is likely to end up revisioning quite a few other formula, so I wanted to work on that from a clean tree without the version bump causing unnecessary noise in testing. Hopeful of having the Lua51 & Luajit builds back in Luarocks within a week tops. |
|
Thanks a lot for your work. I really appreciate it.
|
|
What is the Should be something like: vs |
|
I'm pretty sure I know what the problem is, at least with the Luarocks config. I'll be pushing some fairly major changes in the next few days to deal with this. It's going to require a lot of local testing though, which is what is slowing me down at the moment. Right now inside Homebrew attempting to build |
|
For someone looking for a workaround: I had to solve this by uninstalling Luarocks, then rolling back the Homebrew 'lua' package to a 5.1 version (for which I thankfully still had a package in the cache, despite running a cleanup), then build and install Luarocks from source. The 'lua51' version does not seem to add headers or libraries without the 5.1 version attached somewhere in the name, and so Rocks will fail installing when they require compilation |
|
Still working on this. Haven't abandoned it. Hope to get the fix pushed by Sunday evening. A lot of work & revisions to be done & tested & checked over hence the delay here. Sent from my iPhone
|
|
Not sure if this is related, but I can not get luarocks to install on my OSX 10.10 homebrew. I had homebrew before upgrading to 10.10, it died and I completely removed homebrew and just got it to successfully install today. Now, I am trying to get lua 5.2 and luarocks to install. lua works, but luarocks says that it can not find /usr/local/Cellar/luarocks/2.1.1/share/lua in the linking stage. Therefore, the script for linking is still trying to use the old version of lua. I need 5.2 for my Mjolnir installation. |
|
Looks like a user config issue rather than a Lua issue. Try the usual brew update, brew remove Lua, brew remove luarocks, brew update, brew prune, brew cleanup and finally brew install luarocks. See if that works? Sent from my iPhone
|
|
Done all that at least twice. Nothing helped. That is why I figure the brew Richard On Wed, Oct 1, 2014 at 10:31 PM, Dominyk Tiller notifications@github.com
|
|
Hey, the third time worked. Weird. Maybe I need to do a full wipe of On Wed, Oct 1, 2014 at 10:39 PM, Richard Guay raguay@customct.com wrote:
|
|
@raguay Check out this script. |
|
Thanks! |
|
FWIW, I'm still working on this. Quite a few delays have managed to sneak in, from the gtk+3 update the other day that threw a few packages off, to the general way this is going to have to be fixed. Essentially, I'm going to have to very slightly but importantly rebuild both the Luas we now ship and then once again check absolutely everything that depends on the Luas to check the changes don't require a revision elsewhere. At least both Luas, luarocks & luajit are going to require rebuilding & revisioning. The actual changes themselves aren't overly difficult, but testing them to make sure we don't wander into another situation like this one is taking a while. Appreciate the wait is tedious, but I'd quite like to not spend too much more time staring at the word "Lua" after I fix this issue 😸. |
|
Hi there, I do need lua5.1 as the only lua on my machine, is there an easy way to get that working? and get luarocks against 5.1 instead of 5.2? Thank you |
|
Alright. I'm too ill to do much today so I might as well spend the last few hours ironing this out. I'll finish this off finally today and solve this issue. |
|
Hilariously, lua-users where we pull the patches from is currently down. Terrific. And the website is too: Sigh. Once the website is back up I'll probably go vendor these patches manually to avoid this happening in future. |
|
Ah, this is confirmed upstream. They're looking into it:
|
|
Okay. I'm back to the drawing board, again. 6 hours work and we end up in the same place ugh. Remember before when I said you get to the bottom of one issue and it turns out something else was causing that issue? Well, yeah. It turns out the error message isn't directly from Lua, Lua51, Luarocks or Luajit itself. It comes from That is what is throwing the This is what the env command should look like, and does look like using Lua/Lua52: So when we do But it doesn't. We hit the We seem to have gotten around this problem with And that works. But trying to do something similar with Lua doesn't work. And I'm kinda stumped by that, so I'll tag in @MikeMcQuaid who wrote the language requirement document, and @tdsmith who is the resident Python expert around here and see if I'm missing or doing something stupid here. |
|
If anyone wants to see the work in progress and offer up some tips/hints/etc, I've just pushed it here so it can be examined in the hope someone finds an answer to the above issue. |
|
Is the problem that luarocks needs to support multiple luas and that doing that with a single luarocks formula is hard? Each flavor of python (python, python3, pypy, pypy3) bundles its own suffixed pip. Would that be easier? |
|
Prosody ship their own Homebrew formulae, with Lua51 and Luarocks as a specified dependency. Given they ship their own formula, with dependencies on other formula they ship, they could probably just add a modified luarocks formula to the ones they ship already and that would fix the problem for them easily enough. Their setup actually rubs against Homebrew a little because they don't use the |
|
On the Prosody front; It'd take 15 minutes to set up a tap and sort this issue out. This particular issue has a very easy fix. |
|
Yes - I actually installed lua51 and the prosody from source. I can help with it when I get back to my computer later, @metalaureate |
|
@stuartcarnie thank you! |
|
@DomT4 agree – a |
|
@stuartcarnie Yeah. Honestly, For the sake of knowing it works with all three luas and the sake of making life easier on you guys, you're more than welcome to use the Luarocks formula I drew up should we later decide we need to write back in It shouldn't take long to set up at all. 'Tis the great beauty of the |
|
Prosody author here. Our homebrew formula was a contribution, and does various things sub-optimally. I'd like to clean it up, and get Prosody into homebrew proper. Regarding Lua 5.1 vs LuaJIT vs Lua 5.2, the situation is similar to Python 2 vs PyPy/etc vs Python 3. Lua 5.1 is the most popular, widely deployed, widely supported. LuaJIT has full compatibility with 5.1, but is not fully compatible with 5.2 (it just implements a subset of 5.2's features, and on any conflict between 5.1 and 5.2, chooses the 5.1 feature). 5.2 is not binary or source compatible with 5.1 (it's theoretically a very similar but separate programming language, much like python2 vs 3). Don't let the version numbers fool you, it's not a 'minor' compatible release. Mike Pall (LuaJIT author) has described Lua 5.2 as the Windows Vista of Lua releases, but I have no opinions on that :) I'd just like to note that 5.1 is very much in widespread use, more so than 5.2, and wont be dying too soon. Note that 5.3 is in the works now, and while it's mostly backwards compatible (not 100%), the Lua team makes no promises that it will remain compatible. Luarocks packages with binary components installed for 5.2 wont magically work on 5.3 without a recompile :) Prosody cannot drop support for 5.1 or LuaJIT. Prosody is attempting to gain support for 5.2 as well, but this will take significant work, and then will be in a future Prosody release. I assume you folks want to drop 5.1 support in luarocks? In my opinion this is wrong: side by side 5.1 and 5.2 installs should work in much the same way python 2 and 3 installs work. But I wont argue too much about this, if others feel differently. If luarocks can only do one or the other, it might make sense for Prosody to have a Prosody-local install of luarocks, or for Prosody to not use luarocks, and do local installs of dependencies directly (this is e.g., what happens on Debian/Ubuntu, as they have official non-luarocks packages of our dependencies). I don't want to lock a Prosody user to specific version of luarocks for their non-Prosody projects. Another complication is Prosody has plugin support, and plugins can have their own rocks based dependencies. Should I create a new GitHub issue around a Prosody homebrew formula? |
With your level of customization you're probably better setting up a tap rather than trying to push things into the master branch here. I'll let the maintainers have some more say on that, but looking at what you do with the formulae I think a tap is the way to go and offers you the kind of setup you're looking for.
We're aware of that, and some of the significant differences. Lua 5.1 was released in 2006 though, and Lua 5.2 in 2011, and Lua 5.3 is under development. There was and is a real risk of hampering software by pegging absolutely everything to Lua 5.1 which is why we created this setup where they function side by side. For clarification - You can absolutely install Lua 5.1 and 5.2 without issue, side-by-side with Homebrew. That is done, and works. The only thing is the choice not to support a Luarocks formula that runs under multiple Luas.
Some further information on this would be handy. As lovely as widespread use sounds, some penetration statistics would be genuinely useful here, especially in regards to Luarocks, which is the affected party here.
This is known and understood.
You don't need to. Homebrew supports side-by-side installations of Lua51 and Lua52 (And Luajit too). All that Homebrew does not currently support is Luarocks using that side-by-side installation. You could go away and create a new tap with a customised
Correct. @MikeMcQuaid
I think you misunderstand the way Homebrew handles Python here. We ship Python 2 & Python 3, but any formula can only generally use one of those at once, and they have separate
Yes. This is what I refer to when I talk about setting up a Prosody |
|
Thanks for the clarification around homebrew current plans and around python. My point is, that customization isn't actually needed. A separate Tap would work, certainly, but I don't think it's actually warranted. Prosody depends on some standard things, like OpenSSL and lua51, etc, and these are not an issue at all. Prosody then depends on some packages installed via luarocks. These are installed system-global in our current homebrew script, but should be Prosody-local. This is true regardless of whether we have a custom tap or not. I already have a fix for this, which is telling luarocks an install path while installing these packages. The only problem is luarocks. Our script should either stop using luarocks, or should do a Prosody-local install of luarocks. Note that this is true regardless of whether we have a custom tap or not. In the end I want a good user experience, and I don't want to force Prosody's choice of Lua or luarocks versions onto the user for their non-Prosody usage. I'd very much appreciate feedback, review or assistance in this, and am open to making changes in Prosody to support these things. I do want to support 5.2, but that's potentially months away. |
Fix some issues with luarocks looking for the wrong version and cleanup the lua and lua51 formulae. Closes Homebrew#32354.
|
You had asked for statistics. Stats hard to find, but the Debian popularity contest is one (potentially unreliable) source: https://qa.debian.org/popcon.php?package=lua5.1 Ubuntu has similar numbers. They don't have an online viewer, but here's what I got from the large data dump they provide: As a sidenote: Luarocks in Debian and Ubuntu still depends on Lua 5.1, and not 5.2, despite them having a Lua 5.2 package for years now: http://packages.ubuntu.com/trusty/luarocks |
Well, unless the maintainers here have a shift of mind on the
Generally, yes. The Homebrew way is to sandbox things inside their own prefix, unless required globally.
Indeed. You could just use a customised |
I think it depends on whatever is called |
|
@DomT4 My comment was not meant to in any way spur your development, just to provide info on what I found worked for me. If that came over differently I apologise. For what it's worth, from my point of view, side-by-side Lua is not very important, but providing a way to get a complete 5.1 version is. Many modules fail to compile against the Anyway, I believe it's an error to keep a real About the luarocks 5.1 in Debian: If you compile your own luarocks on Debian, you won't have any trouble, but if you use the one from the Debian repository (https://packages.debian.org/wheezy/luarocks) it will depend on Lua 5.1. However, this is not a real issue, as the Lua version Luarocks is compiled against does not dictate which Lua versions it can install packages for. |
|
Oh the development push predated your comment, no worries. I initially wanted to support the multiple luarocks configurations natively anyway, but this thread kickstarted my desire to get that done quicker. After we found a solution the maintainers decided we couldn't shove that much potentially troublesome code onto all users unless the demand arises, but should it do so we at least now have the answer as to how to achieve that. On the headers issue: That should be wrong? The headers should be named entirely normally except they're inside "homebrew_prefix/include/lua5.1" instead of "homebrew_prefix/include". The pkg-config file points to that directory for headers so everything should be finding them okay. The headers themselves shouldn't be renamed; the formula isn't written that way. If you can give me some examples of things broken by this change I can look into them individually and push fixes where practical. I am actually looking into having Lua52 as Lua52 in the executables but having a symlink from Lua52 to Lua, which is how Debian does this. I'm not too worried about 5.3 yet. Based on Lua's historical development timetable I wouldn't expect 5.3 for at least another 2 years. I'm all for future proofing but Lua is less urgent than things with a hastier release timetable ;). I'm pretty sure my apt-get installed luarocks uses the Lua package, which is symlinked to 5.2. I'll double check that later today. Maybe I'm wrong on that point. |
|
Comment earlier didn't format correctly. Have fixed that now, apologies. |
|
What is homebrew's official status on luajit + luarocks currently? I'll admit I'm still pretty new to lua, but running some initial benchmarks, it looks like luaijt is multitudes faster, so I would prefer to use luajit over normal lua. It's reassuring to know it's first choice for projects like openresty. |
|
Okay, so luajit + luarocks is not supported (sorry, I didn't catch that first time around, I was trying to read it this too fast). Is there any official unofficial workarounds? For example, how could I use the formula in this luarocks.rb gist? Should I be looking at setting up a new |
|
@naartjie Replied to your comment on the Gist I set up. Hopefully the reply is helpful to you. |
|
@DomT4 Yes it's very helpful, amazing, thank you for such a detailed response. 👍 |
The first public "work" version of 5.3 was published about a year and a half ago. They've been through alpha, beta and 3 release candidates now. Also, the versions page on lua.org says:
|
Yes, Indeed. They've made huge progress in releases in the past few months. When I made the comment, they'd only just released the Alpha of 5.3, and based on the release schedule of the previous two Luas (2006 for 5.1, and 2011 for 5.2), I wasn't particularly expecting a quick release. But what we do from here is largely dictated by adoption rates. If everything currently using As ever, I note that I'm not a maintainer, and I have almost no say over what goes in and what doesn't, but based on previous maintainer comments and discussion, what I said above is what I believe likely to occur in x situation. |
|
5.3 came out this morning. I'll start another issue |
|
Hi - I have a situation where I need lua5.1/lua5.2/lua5.3 and luajit to co-exist on my system - can anyone explain to me how to do this with luarocks from homebrew, or do i need to resort to something like luajit-luarocks as a solution? Its not clear to me how homebrew expects us to just not use lua5.1 - for some projects this is not viable. |
|
In my work on a Prosody homebrew formula, I resorted to basically prosody-luarocks. See: https://github.com/prosody/homebrew-prosody/blob/master/prosody.rb @DomT4, etc, a code review of the formula would be appreciated. I would like Prosody officially in homebrew as soon as we support Lua 5.2 and 5.3. |
|
So if I get this right, I can use prosody-luarocks to get 5.1 lua packages installed on my system? If this is the case: thanks. But I'd still like to register a protest against whoever decided that the homebrew luarocks shouldn't be functional for us users who would like to rely on it to keep a 'clean' system .. |
|
I've actually proposed and detailed a different way of handling Luarocks which should suit just about everyone here. It took a long time to iron out and test properly, but seems to be something the maintainers aren't against. It internalises Luarocks into each Lua formulae in the same way |
It seems that luarocks + luajit (and possibly other combos) have been made non-working by this recent change:
The text was updated successfully, but these errors were encountered: