Hardening build results

[amdei]

Debian 10 Buster x64

# gcc --version
gcc (Debian 8.3.0-6) 8.3.0

Debian hardening-check tool complains on default build result like that:

# hardening-check /opt/luajit21/bin/luajit-2.1.0-beta3
/opt/luajit21/bin/luajit-2.1.0-beta3:
 Position Independent Executable: yes
 Stack protected: no, not found!
 Fortify Source functions: no, only unprotected functions found!
 Read-only relocations: yes
 Immediate binding: no, not found!

I've tried to play with different build options to overcome the situation:

export LUAJIT_PREFIX=/opt/luajit21
export LUAJIT_COMMON_XCFLAGS="-DLUAJIT_ENABLE_LUA52COMPAT -DLUA_USE_APICHECK -DLUA_USE_ASSERT -DLUAJIT_NUMMODE=2 -msse4.2 -O1"
export LUAJIT_XCFLAGS="$LUAJIT_COMMON_XCFLAGS"

export CFLAGS="-fstack-protector -fstack-protector-strong --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Werror=implicit-function-declaration -Winit-self -Wp,-D_FORTIFY_SOURCE=2 -fPIC" 
export LDFLAGS="-Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie"

make -j `nproc` PREFIX=$LUAJIT_PREFIX XCFLAGS="$LUAJIT_XCFLAGS"

make install PREFIX=$LUAJIT_PREFIX

Bright side: hardening-check complains less:

# hardening-check /opt/luajit21/bin/luajit-2.1.0-beta3
/opt/luajit21/bin/luajit-2.1.0-beta3:
 Position Independent Executable: yes
 Stack protected: no, not found!
 Fortify Source functions: yes (some protected functions found)
 Read-only relocations: yes
 Immediate binding: yes

Dark side: 5 tests has failed:

=== test/misc/meta_comp_jit.lua
/opt/luajit21/bin/luajit-2.1.0-beta3: meta_comp_jit.lua:91: assertion failed!
stack traceback:
        [C]: in function 'assert'
        meta_comp_jit.lua:91: in main chunk
        [C]: at 0x55ebe02c40af
Failed test when running /opt/luajit21/bin/luajit-2.1.0-beta3 meta_comp_jit.lua 1: 256
...
=== test/misc/parse_misc.lua
/opt/luajit21/bin/luajit-2.1.0-beta3: parse_misc.lua:10: assertion failed!
stack traceback:
        [C]: in function 'assert'
        parse_misc.lua:10: in main chunk
        [C]: at 0x55a43ac3c0af
Failed test when running /opt/luajit21/bin/luajit-2.1.0-beta3 parse_misc.lua 1: 256
...
=== test/misc/goto.lua
/opt/luajit21/bin/luajit-2.1.0-beta3: goto.lua:9: assertion failed!
stack traceback:
        [C]: in function 'assert'
        goto.lua:9: in function 'expect'
        goto.lua:33: in main chunk
        [C]: at 0x55a3051030af
Failed test when running /opt/luajit21/bin/luajit-2.1.0-beta3 goto.lua 1: 256
...
=== test/misc/meta_len.lua
/opt/luajit21/bin/luajit-2.1.0-beta3: meta_len.lua:8: assertion failed!
stack traceback:
        [C]: in function 'assert'
        meta_len.lua:8: in function '__len'
        meta_len.lua:27: in main chunk
        [C]: at 0x55b3025850af
Failed test when running /opt/luajit21/bin/luajit-2.1.0-beta3 meta_len.lua 1: 256
...
=== test/misc/libfuncs.lua
/opt/luajit21/bin/luajit-2.1.0-beta3: libfuncs.lua:24: got: "_G:_VERSION:arg:assert:collectgarbage:coroutine:debug:dofile:error:gcinfo:getfenv:getmetatable:io:ipairs:load:loadfile:loadstring:math:module:newproxy:next:os:package:pairs:pcall:print:rawequal:rawget:rawlen:rawset:require:select:setfenv:setmetatable:string:table:tonumber:tostring:type:unpack:xpcall"
expected: "_G:_VERSION:arg:assert:collectgarbage:coroutine:debug:dofile:error:gcinfo:getfenv:getmetatable:io:ipairs:load:loadfile:loadstring:math:module:newproxy:next:os:package:pairs:pcall:print:rawequal:rawget:rawset:require:select:setfenv:setmetatable:string:table:tonumber:tostring:type:unpack:xpcall"
stack traceback:
        [C]: in function 'error'
        libfuncs.lua:8: in function 'check'
        libfuncs.lua:24: in main chunk
        [C]: at 0x5628a55890af
Failed test when running /opt/luajit21/bin/luajit-2.1.0-beta3 libfuncs.lua 1: 256
...

Is it worth to continue investigation?
Are there any chances to success?
Or such kind of hardenization contradict with LuaJIT internals?

[agentzh]

@amdei stack-protector is never supported in LuaJIT. It's explicitly disabled in LuaJIT's own build system (including the official one). It requires extra effort to add support if not impossible.

[amdei]

@agentzh Thanks a lot for clarification!

Ok, let's leave stack-protector aside then.

With your generous help in #90, following configuration passed all tests on both Debian 9&10 x64 for me:

export LUAJIT_PREFIX=/opt/luajit21
export LUAJIT_COMMON_XCFLAGS="-DLUAJIT_ENABLE_LUA52COMPAT -DLUA_USE_APICHECK -DLUA_USE_ASSERT -DLUAJIT_NUMMODE=2 -msse4.2 -O1"
export LUAJIT_XCFLAGS="$LUAJIT_COMMON_XCFLAGS" LUA52=1

export CFLAGS="--param=ssp-buffer-size=4 -Wformat -Werror=format-security -Werror=implicit-function-declaration -Winit-self -Wp,-D_FORTIFY_SOURCE=2 -fPIC" 
export LDFLAGS="-Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie"

make -j `nproc` PREFIX=$LUAJIT_PREFIX XCFLAGS="$LUAJIT_XCFLAGS"
make install PREFIX=$LUAJIT_PREFIX

How do you think, is it safe to use it in production, aiming to embed such LuaJIT into NGINX?