build-pkg.lua failure in comparePasses (bad arg #1 to 'ipairs') on item i686-w64-mingw32.static~libical

[master5597]

I'm compiling using:
$ git clone https://github.com/mxe/mxe /usr/lib/mxe
$ cd /usr/lib/mxe
$ lua tools/build-pkg.lua &> build-pkg.log

on Ubuntu 16.04 with the dependencies as specified here: https://github.com/mxe/mxe-apt
and after about 48 hours on my system (2nd pass of the 1671) the compile dies because prev_files is nil so the ipairs on line 647 in local function comparePasses(...) (line 634) throws an exception:

lua: tools/build-pkg.lua:647: bad argument #1 to 'ipairs' (table expected, got nil)
stack traceback:
[C]: in function 'ipairs'
tools/build-pkg.lua:647: in function 'comparePasses'
tools/build-pkg.lua:686: in function 'buildItem'
tools/build-pkg.lua:915: in function 'buildPackages'
tools/build-pkg.lua:1099: in function 'main'
tools/build-pkg.lua:1113: in main chunk
[C]: in ?

Inside the failing comparePasses function item=i686-w64-mingw32.static~libical, new _files has 495 items and prev_file2item has more items than that in it. if I put a 'if prev_files not null' around the for loop at 647 (below) the compile completes, I’m just not sure what I’m glossing over or covering up.

tools/build-pkg.lua - line 647: for _, file in ipairs(prev_files) do

I would put the whole log up on https://gist.github.com/, but it's huge (2.7M or so). Let me know if you want it. I’ll have to rerun it without all my debug code (48 hours of compiling on my system) to get a clean log. If you just want the output of a smaller (sub) command let me know what to run.

[starius]

It looks like prev_files is nil, if the first pass failed. If i686-w64-mingw32.static~libical is broken in the first pass, it confirms the hypothesis. Could you post output of grep -C 20 -w broken your.log as well as the following files: log/*/libical_i686-w64-mingw32.static, please?

[master5597]

Thanks, I didn’t know what to look for, libical is definitely broken. The grep is from my log file where I put the "if not null" around the for loop and the libical_i686-w64-mingw32.static file is from my last build attempt.

grep -C 20 -w broken your.log : https://gist.github.com/master5597/dab0bb371d6dd70d15eda094edc4d4c4

log/20160801_050332/libical_i686-w64-mingw32.static - https://gist.github.com/master5597/955a7d50e83a8731048a65a8c34b9659

[starius]

#1473 should fix the crash of build-pkg. Can you confirm this, please?

I propose to open additional issues for packages broken (at least for those which are broken in first pass.)

[master5597]

That is the exact change (minus the comment) that I made to make my compile complete. (ran a diff) so I can verify the change completes the build. Does this mean that everything compiled correctly? I thought I was just skipping over the problem and not fixing it.

[starius]

Does this mean that everything compiled correctly? I thought I was just skipping over the problem and not fixing it.

It fixes a bug in build-pkg.lua. It does item2files[item] = files in buildPackages only if the package was built successfully. item2files from first pass becomes prev_item2files in the second pass. So prev_item2files[item] is not defined for an item broken in the first pass.

[master5597]

Makes sense, thanks for your time.

[starius]

I have merged #1473

[tonytheodore]

Inside the failing comparePasses function item=i686-w64-mingw32.static~libical, new _files has 495 items and prev_file2item has more items than that in it.

@starius @master5597, how does one inspect variables in a lua program? Any recommendations from the list of debuggers?

[starius]

@tonytheodore
I don't use a debugger for Lua. My favorite "debugger" is print on most languages :)
To explore internals of data structures interactively I use luaprompt, it is similar to ipython.
People on lua mailing list are discussing Visual Studio plugin for Ravi (a language derived from Lua).
By the way, can you ask your question on the Lua mailing list?

[master5597]

@tonytheodore
I don't know the lua language at all. So I just printed out what was going on. For example if you look at starius's changes (#1473) for this ticket, I put the following code right before it. (Note: The output log is huge 42G vs 2.7M) It's ugly, but it worked.

    log('comparePasses(item=%s, new_files, prev_file2item, prev_files)', item)
    if new_files then
        for k, v in pairs(new_files) do
            log("new_files: %s=%s", k, v)
        end
    else
       log("new_files is nil")
    end
    if prev_file2item then
        for k, v in pairs(prev_file2item) do
            log("prev_file2item: %s=%s", k, v)
        end
    else
       log("prev_file2item is nil")
    end
    if prev_files then
        for k, v in pairs(prev_files) do
            log("prev_files: %s=%s", k, v)
        end
    else
       log("prev_files is nil")
    end