Use typed null pointers instead of i64 0-values

[sppalkia]

Fixes #473 when using Weld with an LLVM 6.0 distribution that has LLVM_ENABLE_ASSERTIONS enabled.

Also fixes some README issues.

The issue was that some places in the code generation used an i64 0 literal as a substitute for null, which was okay with LLVM's module verifier but caused certain debug assertions to complain.

[sppalkia]

@boegel mind giving this a try?

[boegel]

@sppalkia will do ASAP (hopefully tomorrow)

[zao]

I'm not boegel, but with af1e767 on Clang/6.0.1 one test suite has two failing tests:

zao@freja:/scratch/zao/buildtests/weld$ target/debug/deps/serialize_tests-e9aa7bd8833ed150 dict_nested_pointers

running 1 test
serialize_tests-e9aa7bd8833ed150: /hp/eb/build/Clang/6.0.1/GCC-8.3.0-2.32/llvm-6.0.1.src/lib/IR/Constants.cpp:2055: static llvm::Constant *llvm::ConstantExpr::getInsertValue(llvm::Constant *, llvm::Constant *, ArrayRef<unsigned int>, llvm::Type *): Assertion `ExtractValueInst::getIndexedType(Agg->getType(), Idxs) == Val->getType() && "insertvalue indices invalid!"' failed.
Aborted (core dumped)

Backtrace for dict_nested_pointers

#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007ffff6e31801 in __GI_abort () at abort.c:79
#2  0x00007ffff6e2139a in __assert_fail_base (fmt=0x7ffff6fa87d8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
    assertion=assertion@entry=0x555557f83877 "ExtractValueInst::getIndexedType(Agg->getType(), Idxs) == Val->getType() && \"insertvalue indices invalid!\"",
    file=file@entry=0x555557f80844 "/hp/eb/build/Clang/6.0.1/GCC-8.3.0-2.32/llvm-6.0.1.src/lib/IR/Constants.cpp", line=line@entry=2055,
    function=function@entry=0x555557f837f3 "static llvm::Constant *llvm::ConstantExpr::getInsertValue(llvm::Constant *, llvm::Constant *, ArrayRef<unsigned int>, llvm::Type *)") at assert.c:92
#3  0x00007ffff6e21412 in __GI___assert_fail (assertion=0x555557f83877 "ExtractValueInst::getIndexedType(Agg->getType(), Idxs) == Val->getType() && \"insertvalue indices invalid!\"",
    file=0x555557f80844 "/hp/eb/build/Clang/6.0.1/GCC-8.3.0-2.32/llvm-6.0.1.src/lib/IR/Constants.cpp", line=2055,
    function=0x555557f837f3 "static llvm::Constant *llvm::ConstantExpr::getInsertValue(llvm::Constant *, llvm::Constant *, ArrayRef<unsigned int>, llvm::Type *)") at assert.c:101
#4  0x0000555556f640b6 in llvm::ConstantExpr::getInsertValue(llvm::Constant*, llvm::Constant*, llvm::ArrayRef<unsigned int>, llvm::Type*) ()
#5  0x00005555561621e5 in weld::codegen::llvm2::CodeGenExt::zero (self=0x7ffff6de9398, ty=0x7ffff00b4bb0) at weld/src/codegen/llvm2/mod.rs:376
#6  0x00005555560aafeb in _$LT$weld..codegen..llv$u6d$2..LlvmGenerator$u20$as$u20$weld..codegen..llv$u6d$2..serde..DeHelper$GT$::gen_deserialize_helper::h229a57c490ed160f (self=0x7ffff6de9398,
    builder=0x7ffff006f900, position=0x7ffff00cc040, output=0x7ffff00cb5d8, ty=0x7ffff00a51a0, buffer=0x7ffff00e0678, run=0x7ffff00cb708) at weld/src/codegen/llvm2/serde.rs:808
#7  0x00005555560a2679 in _$LT$weld..codegen..llv$u6d$2..LlvmGenerator$u20$as$u20$weld..codegen..llv$u6d$2..serde..SerDeGen$GT$::gen_deserialize::hc51fb39ce1c09e4c (self=0x7ffff6de9398, ctx=0x7ffff6de8d98,
    statement=0x7ffff00a5620) at weld/src/codegen/llvm2/serde.rs:95
#8  0x00005555560b6bed in weld::codegen::llvm2::LlvmGenerator::gen_statement (self=0x7ffff6de9398, context=0x7ffff6de8d98, statement=0x7ffff00a5620) at weld/src/codegen/llvm2/mod.rs:1206
#9  0x00005555560b453e in weld::codegen::llvm2::LlvmGenerator::gen_sir_function (self=0x7ffff6de9398, program=0x7ffff6ded4d0, func=0x7ffff00a21a0) at weld/src/codegen/llvm2/mod.rs:1107
#10 0x00005555560af523 in weld::codegen::llvm2::LlvmGenerator::generate (conf=..., program=0x7ffff6ded4d0) at weld/src/codegen/llvm2/mod.rs:781
#11 0x00005555560ac914 in weld::codegen::llvm2::compile (program=0x7ffff6ded4d0, conf=0x7ffff6dec490, stats=0x7ffff6dec440) at weld/src/codegen/llvm2/mod.rs:144
#12 0x0000555555ffaa6f in weld::codegen::compile_program (program=0x7ffff6ded4d0, conf=0x7ffff6dec490, stats=0x7ffff6dec440) at weld/src/codegen/mod.rs:92
#13 0x0000555555dc0ddf in weld::WeldModule::compile (code=..., conf=0x7ffff6dee4c0) at /scratch/zao/buildtests/weld/weld/src/lib.rs:734
#14 0x0000555555dd5235 in serialize_tests::common::_compile_and_run (code=..., conf=0x7ffff6dee4c0, ptr=0x7ffff6dee4f0) at weld/tests/common/mod.rs:54
#15 0x0000555555dd5515 in serialize_tests::common::compile_and_run (code=..., conf=0x7ffff6dee4c0, ptr=0x7ffff6dee4f0) at weld/tests/common/mod.rs:68
#16 0x0000555555df2847 in serialize_tests::dict_nested_pointers () at weld/tests/serialize_tests.rs:180
#17 0x0000555555df0a2a in serialize_tests::dict_nested_pointers::{{closure}} () at weld/tests/serialize_tests.rs:149
#18 0x0000555555dd59de in core::ops::function::FnOnce::call_once () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libcore/ops/function.rs:231
#19 0x0000555555e00d0f in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/liballoc/boxed.rs:746
#20 0x0000555557a81c8a in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:82
#21 0x0000555555e1bf38 in std::panicking::try () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/panicking.rs:275
#22 std::panic::catch_unwind () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/panic.rs:394
#23 test::run_test::run_test_inner::{{closure}} () at src/libtest/lib.rs:1466
#24 0x0000555555df5c95 in std::sys_common::backtrace::__rust_begin_short_backtrace () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/sys_common/backtrace.rs:77
#25 0x0000555555df9d55 in std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}} () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/thread/mod.rs:470
#26 <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/panic.rs:315
#27 std::panicking::try::do_call () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/panicking.rs:296
#28 0x0000555557a81c8a in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:82
#29 0x0000555555dfa942 in std::panicking::try () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/panicking.rs:275
#30 std::panic::catch_unwind () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/panic.rs:394
#31 std::thread::Builder::spawn_unchecked::{{closure}} () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/thread/mod.rs:469
#32 core::ops::function::FnOnce::call_once{{vtable-shim}} () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libcore/ops/function.rs:231
#33 0x0000555557a5464f in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/liballoc/boxed.rs:746
#34 0x0000555557a809b0 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/liballoc/boxed.rs:746
#35 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#36 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#37 0x00007ffff75876db in start_thread (arg=0x7ffff6df0700) at pthread_create.c:463
#38 0x00007ffff6f1288f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

zao@freja:/scratch/zao/buildtests/weld$ target/debug/deps/serialize_tests-e9aa7bd8833ed150 dict_pointers

running 1 test
serialize_tests-e9aa7bd8833ed150: /hp/eb/build/Clang/6.0.1/GCC-8.3.0-2.32/llvm-6.0.1.src/lib/IR/Constants.cpp:2055: static llvm::Constant *llvm::ConstantExpr::getInsertValue(llvm::Constant *, llvm::Constant *, ArrayRef<unsigned int>, llvm::Type *): Assertion `ExtractValueInst::getIndexedType(Agg->getType(), Idxs) == Val->getType() && "insertvalue indices invalid!"' failed.
Aborted (core dumped)

Backtrace for dict_pointers

#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007ffff6e31801 in __GI_abort () at abort.c:79
#2  0x00007ffff6e2139a in __assert_fail_base (fmt=0x7ffff6fa87d8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
    assertion=assertion@entry=0x555557f83877 "ExtractValueInst::getIndexedType(Agg->getType(), Idxs) == Val->getType() && \"insertvalue indices invalid!\"",
    file=file@entry=0x555557f80844 "/hp/eb/build/Clang/6.0.1/GCC-8.3.0-2.32/llvm-6.0.1.src/lib/IR/Constants.cpp", line=line@entry=2055,
    function=function@entry=0x555557f837f3 "static llvm::Constant *llvm::ConstantExpr::getInsertValue(llvm::Constant *, llvm::Constant *, ArrayRef<unsigned int>, llvm::Type *)") at assert.c:92
#3  0x00007ffff6e21412 in __GI___assert_fail (assertion=0x555557f83877 "ExtractValueInst::getIndexedType(Agg->getType(), Idxs) == Val->getType() && \"insertvalue indices invalid!\"",
    file=0x555557f80844 "/hp/eb/build/Clang/6.0.1/GCC-8.3.0-2.32/llvm-6.0.1.src/lib/IR/Constants.cpp", line=2055,
    function=0x555557f837f3 "static llvm::Constant *llvm::ConstantExpr::getInsertValue(llvm::Constant *, llvm::Constant *, ArrayRef<unsigned int>, llvm::Type *)") at assert.c:101
#4  0x0000555556f640b6 in llvm::ConstantExpr::getInsertValue(llvm::Constant*, llvm::Constant*, llvm::ArrayRef<unsigned int>, llvm::Type*) ()
#5  0x00005555561621e5 in weld::codegen::llvm2::CodeGenExt::zero (self=0x7ffff6de93b8, ty=0x7ffff00b3900) at weld/src/codegen/llvm2/mod.rs:376
#6  0x00005555560aafeb in _$LT$weld..codegen..llv$u6d$2..LlvmGenerator$u20$as$u20$weld..codegen..llv$u6d$2..serde..DeHelper$GT$::gen_deserialize_helper::h229a57c490ed160f (self=0x7ffff6de93b8,
    builder=0x7ffff006fa40, position=0x7ffff00ca0a0, output=0x7ffff00c9638, ty=0x7ffff00a3a20, buffer=0x7ffff00daee8, run=0x7ffff00c9768) at weld/src/codegen/llvm2/serde.rs:808
#7  0x00005555560a2679 in _$LT$weld..codegen..llv$u6d$2..LlvmGenerator$u20$as$u20$weld..codegen..llv$u6d$2..serde..SerDeGen$GT$::gen_deserialize::hc51fb39ce1c09e4c (self=0x7ffff6de93b8, ctx=0x7ffff6de8db8,
    statement=0x7ffff00a40b0) at weld/src/codegen/llvm2/serde.rs:95
#8  0x00005555560b6bed in weld::codegen::llvm2::LlvmGenerator::gen_statement (self=0x7ffff6de93b8, context=0x7ffff6de8db8, statement=0x7ffff00a40b0) at weld/src/codegen/llvm2/mod.rs:1206
#9  0x00005555560b453e in weld::codegen::llvm2::LlvmGenerator::gen_sir_function (self=0x7ffff6de93b8, program=0x7ffff6ded4f0, func=0x7ffff00a1390) at weld/src/codegen/llvm2/mod.rs:1107
#10 0x00005555560af523 in weld::codegen::llvm2::LlvmGenerator::generate (conf=..., program=0x7ffff6ded4f0) at weld/src/codegen/llvm2/mod.rs:781
#11 0x00005555560ac914 in weld::codegen::llvm2::compile (program=0x7ffff6ded4f0, conf=0x7ffff6dec4b0, stats=0x7ffff6dec460) at weld/src/codegen/llvm2/mod.rs:144
#12 0x0000555555ffaa6f in weld::codegen::compile_program (program=0x7ffff6ded4f0, conf=0x7ffff6dec4b0, stats=0x7ffff6dec460) at weld/src/codegen/mod.rs:92
#13 0x0000555555dc0ddf in weld::WeldModule::compile (code=..., conf=0x7ffff6dee4c0) at /scratch/zao/buildtests/weld/weld/src/lib.rs:734
#14 0x0000555555dd5235 in serialize_tests::common::_compile_and_run (code=..., conf=0x7ffff6dee4c0, ptr=0x7ffff6dee4f0) at weld/tests/common/mod.rs:54
#15 0x0000555555dd5515 in serialize_tests::common::compile_and_run (code=..., conf=0x7ffff6dee4c0, ptr=0x7ffff6dee4f0) at weld/tests/common/mod.rs:68
#16 0x0000555555df2113 in serialize_tests::dict_pointers () at weld/tests/serialize_tests.rs:121
#17 0x0000555555df074a in serialize_tests::dict_pointers::{{closure}} () at weld/tests/serialize_tests.rs:100
#18 0x0000555555dd5a5e in core::ops::function::FnOnce::call_once () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libcore/ops/function.rs:231
#19 0x0000555555e00d0f in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/liballoc/boxed.rs:746
#20 0x0000555557a81c8a in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:82
#21 0x0000555555e1bf38 in std::panicking::try () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/panicking.rs:275
#22 std::panic::catch_unwind () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/panic.rs:394
#23 test::run_test::run_test_inner::{{closure}} () at src/libtest/lib.rs:1466
#24 0x0000555555df5c95 in std::sys_common::backtrace::__rust_begin_short_backtrace () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/sys_common/backtrace.rs:77
#25 0x0000555555df9d55 in std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}} () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/thread/mod.rs:470
#26 <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/panic.rs:315
#27 std::panicking::try::do_call () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/panicking.rs:296
#28 0x0000555557a81c8a in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:82
#29 0x0000555555dfa942 in std::panicking::try () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/panicking.rs:275
#30 std::panic::catch_unwind () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/panic.rs:394
#31 std::thread::Builder::spawn_unchecked::{{closure}} () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/thread/mod.rs:469
#32 core::ops::function::FnOnce::call_once{{vtable-shim}} () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libcore/ops/function.rs:231
#33 0x0000555557a5464f in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/liballoc/boxed.rs:746
#34 0x0000555557a809b0 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/liballoc/boxed.rs:746
#35 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#36 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#37 0x00007ffff75876db in start_thread (arg=0x7ffff6df0700) at pthread_create.c:463
#38 0x00007ffff6f1288f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

The other test suites seem to pass.

[sppalkia]

Aha, all the tests should pass now -- try it out.

FYI, I noticed that building Weld with LLVM compiled from source takes much longer for some reason compared to using the standard LLVM 6.0 distribution without various flags enabled (seems like there's some link time optimization that uses a ton of memory and is also pretty expensive) -- I couldn't even build it on my laptop because it used so much memory, so I'd definitely recommend using the apt-get or brew distribution if possible.

[zao]

All tests pass now for me with the aforementioned setup.

As a tangent as to why I've got my own LLVM build, on the clusters we prefer having as much software as we can via the module system instead of from OS packaging, to ensure that we have all the versions needed and that it's built the way we need it with architecture optimizations.

[sppalkia]

Makes sense, I'll investigate why the builds are so slow when building LLVM from source when I get a chance. Thanks for testing this!

[boegel]

FWIW: I can confirm that these changes fix the problem reported in #473, thanks for the quick fix @sppalkia!