Crystal does not compile with the latest LLVM version

[anatol]

I am trying to compile Crystal 0.8.0 on Linux Arch with LLVM 3.7 installed and get following error.

src/llvm/ext/llvm_ext.cc:31:25: note:   template argument deduction/substitution failed:
src/llvm/ext/llvm_ext.cc:87:35: error: template argument 1 is invalid
       unwrapDI<DIDescriptor>(Scope), unwrapDI<DIFile>(File), Line, Column);
                                   ^
In file included from /usr/include/llvm/IR/DebugInfo.h:25:0,
                 from /usr/include/llvm/IR/DIBuilder.h:20,
                 from src/llvm/ext/llvm_ext.cc:7:
/usr/include/llvm/IR/DebugInfoMetadata.h:463:3: error: ‘llvm::DIFile::~DIFile()’ is private
   ~DIFile() = default;
   ^
src/llvm/ext/llvm_ext.cc:87:59: error: within this context
       unwrapDI<DIDescriptor>(Scope), unwrapDI<DIFile>(File), Line, Column);
                                                           ^
In file included from /usr/include/llvm/IR/DebugInfo.h:25:0,
                 from /usr/include/llvm/IR/DIBuilder.h:20,
                 from src/llvm/ext/llvm_ext.cc:7:
/usr/include/llvm/IR/DebugInfoMetadata.h:1479:3: error: ‘llvm::DILexicalBlock::~DILexicalBlock()’ is private
   ~DILexicalBlock() = default;
   ^
src/llvm/ext/llvm_ext.cc:87:74: error: within this context
       unwrapDI<DIDescriptor>(Scope), unwrapDI<DIFile>(File), Line, Column);
                                                                          ^
src/llvm/ext/llvm_ext.cc:89:17: error: no matching function for call to ‘wrap(llvm::DILexicalBlock&)’
   return wrap(LB);
                 ^
In file included from /usr/include/llvm/IR/Use.h:30:0,
                 from /usr/include/llvm/IR/Value.h:19,
                 from /usr/include/llvm/IR/User.h:24,
                 from /usr/include/llvm/IR/Constant.h:17,
                 from /usr/include/llvm/IR/Metadata.h:23,
                 from src/llvm/ext/llvm_ext.cc:3:
src/llvm/ext/llvm_ext.cc:18:1: note: candidate: LLVMOpaqueDIBuilder* wrap(const llvm::DIBuilder*)
 DEFINE_SIMPLE_CONVERSION_FUNCTIONS(DIBuilder, LLVMDIBuilderRef)
 ^
src/llvm/ext/llvm_ext.cc:18:1: note:   no known conversion for argument 1 from ‘llvm::DILexicalBlock’ to ‘const llvm::DIBuilder*’
src/llvm/ext/llvm_ext.cc:24:1: note: candidate: LLVMOpaqueMetadata* wrap(const llvm::Metadata*)
 DEFINE_ISA_CONVERSION_FUNCTIONS(Metadata, LLVMMetadataRef)
 ^
src/llvm/ext/llvm_ext.cc:24:1: note:   no known conversion for argument 1 from ‘llvm::DILexicalBlock’ to ‘const llvm::Metadata*’
/usr/include/llvm/IR/IRBuilder.h:1677:1: note: candidate: LLVMOpaqueBuilder* llvm::wrap(const llvm::IRBuilder<>*)
 DEFINE_SIMPLE_CONVERSION_FUNCTIONS(IRBuilder<>, LLVMBuilderRef)
 ^
/usr/include/llvm/IR/IRBuilder.h:1677:1: note:   no known conversion for argument 1 from ‘llvm::DILexicalBlock’ to ‘const llvm::IRBuilder<>*’
/usr/include/llvm/IR/Module.h:688:1: note: candidate: LLVMOpaqueModule* llvm::wrap(const llvm::Module*)
 DEFINE_SIMPLE_CONVERSION_FUNCTIONS(Module, LLVMModuleRef)
 ^
/usr/include/llvm/IR/Module.h:688:1: note:   no known conversion for argument 1 from ‘llvm::DILexicalBlock’ to ‘const llvm::Module*’
In file included from /usr/include/llvm/IR/Module.h:26:0,
                 from src/llvm/ext/llvm_ext.cc:6:
/usr/include/llvm/Support/CodeGen.h:76:24: note: candidate: LLVMCodeModel llvm::wrap(llvm::CodeModel::Model)
   inline LLVMCodeModel wrap(CodeModel::Model Model) {
                        ^
/usr/include/llvm/Support/CodeGen.h:76:24: note:   no known conversion for argument 1 from ‘llvm::DILexicalBlock’ to ‘llvm::CodeModel::Model’
In file included from /usr/include/llvm/IR/Use.h:30:0,
                 from /usr/include/llvm/IR/Value.h:19,
                 from /usr/include/llvm/IR/User.h:24,
                 from /usr/include/llvm/IR/Constant.h:17,
                 from /usr/include/llvm/IR/Metadata.h:23,
                 from src/llvm/ext/llvm_ext.cc:3:
/usr/include/llvm/IR/BasicBlock.h:347:1: note: candidate: LLVMOpaqueBasicBlock* llvm::wrap(const llvm::BasicBlock*)
 DEFINE_SIMPLE_CONVERSION_FUNCTIONS(BasicBlock, LLVMBasicBlockRef)
 ^
/usr/include/llvm/IR/BasicBlock.h:347:1: note:   no known conversion for argument 1 from ‘llvm::DILexicalBlock’ to ‘const llvm::BasicBlock*’
In file included from /usr/include/llvm/IR/Module.h:20:0,
                 from src/llvm/ext/llvm_ext.cc:6:
/usr/include/llvm/IR/DataLayout.h:469:26: note: candidate: LLVMOpaqueTargetData* llvm::wrap(const llvm::DataLayout*)
 inline LLVMTargetDataRef wrap(const DataLayout *P) {
                          ^
/usr/include/llvm/IR/DataLayout.h:469:26: note:   no known conversion for argument 1 from ‘llvm::DILexicalBlock’ to ‘const llvm::DataLayout*’
In file included from /usr/include/llvm/IR/Use.h:30:0,
                 from /usr/include/llvm/IR/Value.h:19,
                 from /usr/include/llvm/IR/User.h:24,
                 from /usr/include/llvm/IR/Constant.h:17,
                 from /usr/include/llvm/IR/Metadata.h:23,
                 from src/llvm/ext/llvm_ext.cc:3:
/usr/include/llvm/PassRegistry.h:96:1: note: candidate: LLVMOpaquePassRegistry* llvm::wrap(const llvm::PassRegistry*)
 DEFINE_STDCXX_CONVERSION_FUNCTIONS(PassRegistry, LLVMPassRegistryRef)
 ^
/usr/include/llvm/PassRegistry.h:96:1: note:   no known conversion for argument 1 from ‘llvm::DILexicalBlock’ to ‘const llvm::PassRegistry*’
In file included from /usr/include/llvm/IR/DerivedTypes.h:21:0,
                 from /usr/include/llvm/IR/DataLayout.h:25,
                 from /usr/include/llvm/IR/Module.h:20,
                 from src/llvm/ext/llvm_ext.cc:6:
/usr/include/llvm/IR/Type.h:483:21: note: candidate: LLVMOpaqueType** llvm::wrap(llvm::Type**)
 inline LLVMTypeRef *wrap(Type **Tys) {
                     ^
/usr/include/llvm/IR/Type.h:483:21: note:   no known conversion for argument 1 from ‘llvm::DILexicalBlock’ to ‘llvm::Type**’
In file included from /usr/include/llvm/IR/Use.h:30:0,
                 from /usr/include/llvm/IR/Value.h:19,

With LLVM 3.6 compilation is fine.

[asterite]

Yes, this is a known "issue": LLVM 3.7 changes the way debug metadata is emitted

[asterite]

Yes, this is a known "issue": LLVM 3.7 changes the way debug metadata is emitted

[jhass]

Well, Arch has 3.7 in testing (https://www.archlinux.org/packages/extra/x86_64/llvm/), so it would be great to have support for it when it lands in the stable repositories (probably in the next two weeks, I guess).

[jhass]

Well, Arch has 3.7 in testing (https://www.archlinux.org/packages/extra/x86_64/llvm/), so it would be great to have support for it when it lands in the stable repositories (probably in the next two weeks, I guess).

[jhass]

I was kinda optimistic I guess, llvm 3.7 just moved to community.

[jhass]

I was kinda optimistic I guess, llvm 3.7 just moved to community.

[bararchy]

Yap... Got the same issue now :/

[bararchy]

Yap... Got the same issue now :/

[bararchy]

@asterite Any updates on this ?

[bararchy]

@asterite Any updates on this ?

[waj]

No progress yet, but 3.7 is not complete compatible with 3.6. Can't you install both versions side by side?

In Homebrew they didn't updated the main package because that would break many apps.

[waj]

No progress yet, but 3.7 is not complete compatible with 3.6. Can't you install both versions side by side?

In Homebrew they didn't updated the main package because that would break many apps.

[ozra]

Just taking a glance at it, it seems like the DataLayout should be moved from a pass to directly on the LLVM module. If it helps some one off in to digging deeper ;-)

[ozra]

Just taking a glance at it, it seems like the DataLayout should be moved from a pass to directly on the LLVM module. If it helps some one off in to digging deeper ;-)

[jhass]

Arch doesn't provide a side by side install, however there's a llvm35 package, but that conflicts. So as soon as somebody wants to compile Crystal and use something else that depends on 3.7 already they'll be out of luck.

[jhass]

Arch doesn't provide a side by side install, however there's a llvm35 package, but that conflicts. So as soon as somebody wants to compile Crystal and use something else that depends on 3.7 already they'll be out of luck.

[vyp]

@jhass If you really need to, you could try using nix package manager for these sort of things which does allow side by side install, in addition to staying out of the way with the llvm installed by pacman (just adjust the $PATH). It has 3.6 and 3.7. Otherwise, maybe you still have 3.6 in your pacman cache, and you can just alternate between the two versions by rolling back/updating?

[vyp]

@jhass If you really need to, you could try using nix package manager for these sort of things which does allow side by side install, in addition to staying out of the way with the llvm installed by pacman (just adjust the $PATH). It has 3.6 and 3.7. Otherwise, maybe you still have 3.6 in your pacman cache, and you can just alternate between the two versions by rolling back/updating?

[vyp]

Actually, it does not have crystal so that's probably too big of an issue. Scrap that.

[vyp]

Actually, it does not have crystal so that's probably too big of an issue. Scrap that.

[waj]

@jhass I've never use Arch so I don't really know how things works there, but given that llvm doesn't have a backward compatibility policy I'd say that not supporting side by side installs it's a bug :-)

Anyway, I'll try to update Crystal ASAP so we can link to 3.7.

[waj]

@jhass I've never use Arch so I don't really know how things works there, but given that llvm doesn't have a backward compatibility policy I'd say that not supporting side by side installs it's a bug :-)

Anyway, I'll try to update Crystal ASAP so we can link to 3.7.

[ozra]

It would be super cool if Crystal could support one version backwards all the time. Especially from 3.8, since it drops support for Win XP (I don't know why I care so much for those poor windows users, haha)

[ozra]

It would be super cool if Crystal could support one version backwards all the time. Especially from 3.8, since it drops support for Win XP (I don't know why I care so much for those poor windows users, haha)

[lbguilherme]

Fedora 23 already comes with LLVM 3.7 only, and installing 3.6 or 3.5 requires building from sources (there are no prebuilt packages for other versions).

[lbguilherme]

Fedora 23 already comes with LLVM 3.7 only, and installing 3.6 or 3.5 requires building from sources (there are no prebuilt packages for other versions).

[waghanza]

@lbguilherme there is https://copr.fedorainfracloud.org/coprs/alonid/llvm-3.6

[waghanza]

@lbguilherme there is https://copr.fedorainfracloud.org/coprs/alonid/llvm-3.6

[asterite]

We only support LLVM 3.6 for now, so I'll close this. Eventually we will update the compiler to work with 3.8, but probably never with 3.7.

[asterite]

We only support LLVM 3.6 for now, so I'll close this. Eventually we will update the compiler to work with 3.8, but probably never with 3.7.

[anatol]

Instead of closing this issue I consider it tracking as "porting to the latest LLVM version". I renamed it, please reopen.

[anatol]

Instead of closing this issue I consider it tracking as "porting to the latest LLVM version". I renamed it, please reopen.

[asterite]

Sounds good.

[asterite]

Sounds good.

[ysbaddaden]

With LLVM 3.8 released, some Linux distributions are upgrading or have already, sadly without support for installing different versions of llvm/clang in parallel. I'm trying to build an llvm36 package for Alpine so I can still build/provide a Crystal compiler for Alpine, but compiling llvm/clang takes hours 😢

Is it a lot of hard work to upgrade? It seems that:

• DataLayout is now required by Module (removed LLVMGetTargetMachineData);
• JIT seems to become deprecated (but we use MCJIT);
• Changed debug info IR. I guess this is the hard part?

References:

• http://llvm.org/releases/3.7.1/docs/ReleaseNotes.html
• http://llvm.org/releases/3.8.0/docs/ReleaseNotes.html

[ysbaddaden]

With LLVM 3.8 released, some Linux distributions are upgrading or have already, sadly without support for installing different versions of llvm/clang in parallel. I'm trying to build an llvm36 package for Alpine so I can still build/provide a Crystal compiler for Alpine, but compiling llvm/clang takes hours 😢

Is it a lot of hard work to upgrade? It seems that:

• DataLayout is now required by Module (removed LLVMGetTargetMachineData);
• JIT seems to become deprecated (but we use MCJIT);
• Changed debug info IR. I guess this is the hard part?

References:

• http://llvm.org/releases/3.7.1/docs/ReleaseNotes.html
• http://llvm.org/releases/3.8.0/docs/ReleaseNotes.html

[asterite]

@ysbaddaden I can try and see how hard it's to migrate to LLVM 3.8. Debug info is the most difficult part, yes, though maybe we can comment all that functionality for now (it's very basic so not super useful). LLVM 3.8 (well, 3.7) comes with some goodies like being able to mark method arguments as nonnull, and dereferenceable(bytes), which might make code more performant (for non-nilable reference types we are sure they are non-null)

[asterite]

@ysbaddaden I can try and see how hard it's to migrate to LLVM 3.8. Debug info is the most difficult part, yes, though maybe we can comment all that functionality for now (it's very basic so not super useful). LLVM 3.8 (well, 3.7) comes with some goodies like being able to mark method arguments as nonnull, and dereferenceable(bytes), which might make code more performant (for non-nilable reference types we are sure they are non-null)

[ysbaddaden]

That would be much appreciated! Even if it means no support for debugging for now when Crystal is built against LLVM 3.7+

[ysbaddaden]

That would be much appreciated! Even if it means no support for debugging for now when Crystal is built against LLVM 3.7+

[asterite]

Well, I just tried upgrading to LLVM 3.8 and the only thing that's necessary is upgrading the debug info. The problem is that this file, llvm_ext.cc must be upgraded. I think most of the original code came from here. Some functions can be upgraded by copy-paste but some disappeared from llvm's source code and I don't know how the upgrade should be done (@waj did that part).

So we have three options:

1. Don't upgrade, stick with LLVM 3.6
2. Upgrade, but drop debug info support.
3. Upgrade by keeping debug support, but for that @waj or somebody that understands LLVM and that C++ code must upgrade it.

[asterite]

Well, I just tried upgrading to LLVM 3.8 and the only thing that's necessary is upgrading the debug info. The problem is that this file, llvm_ext.cc must be upgraded. I think most of the original code came from here. Some functions can be upgraded by copy-paste but some disappeared from llvm's source code and I don't know how the upgrade should be done (@waj did that part).

So we have three options:

1. Don't upgrade, stick with LLVM 3.6
2. Upgrade, but drop debug info support.
3. Upgrade by keeping debug support, but for that @waj or somebody that understands LLVM and that C++ code must upgrade it.

[jhass]

Well, we should only consider 2 if we can retain compatibility with 3.5-3.7 in addition to 3.8 and conditionally drop it on 3.7 & 3.8 only, so one can at least get 3.5 or 3.6 and get debug support back until it's ported to 3.7/3.8.

[jhass]

Well, we should only consider 2 if we can retain compatibility with 3.5-3.7 in addition to 3.8 and conditionally drop it on 3.7 & 3.8 only, so one can at least get 3.5 or 3.6 and get debug support back until it's ported to 3.7/3.8.

[ysbaddaden]

Yes, what about 4. disable debug for LLVM > 3.7 until someone has time to spend on it?

[ysbaddaden]

Yes, what about 4. disable debug for LLVM > 3.7 until someone has time to spend on it?

[felixbuenemann]

This also affects the homebrew formula, now that llvm 3.8.1 is current in homebrew.

[felixbuenemann]

This also affects the homebrew formula, now that llvm 3.8.1 is current in homebrew.

[jhass]

#2993 should fix this very soon :)

[jhass]

#2993 should fix this very soon :)