Support dynamically linked glslang

[faulesocke]

The build script currently assumes that libSPIRV from glslang is always available in a static version (libSPIRV.a). However, this is not true on my system, I only have libSPIRV.so and my libshaderc_combined.a relies on symbols from that .so.

Would it be possible to add a check for libSPIRV.so to the build-script so that the crate compiles correctly on my system?

[antiagainst]

Hey @faulesocke, I don't have the exact configuration as yours; so please feel free to send a pull request for it.

[faulesocke]

@antiagainst Okay, I'm working on it. My system (void linux) looks like this:

% ll /usr/lib/libSPIRV*
-rw-r--r-- 1 root root 336K Apr  2 05:52 /usr/lib/libSPIRV-Tools-comp.a
-rw-r--r-- 1 root root 151K Apr  2 05:52 /usr/lib/libSPIRV-Tools-link.a
-rw-r--r-- 1 root root 6.8M Apr  2 05:52 /usr/lib/libSPIRV-Tools-opt.a
-rw-r--r-- 1 root root 360K Apr  2 05:52 /usr/lib/libSPIRV-Tools-reduce.a
-rwxr-xr-x 1 root root 975K Apr  2 05:52 /usr/lib/libSPIRV-Tools-shared.so
-rw-r--r-- 1 root root 2.0M Apr  2 05:52 /usr/lib/libSPIRV-Tools.a
-rwxr-xr-x 1 root root 522K Jul  9 19:02 /usr/lib/libSPIRV.so
% ll /usr/lib/libshaderc*
-rw-r--r-- 1 root root  64K Jun  8 10:18 /usr/lib/libshaderc.a
-rw-r--r-- 1 root root 221K Jun  8 10:18 /usr/lib/libshaderc_combined.a
lrwxrwxrwx 1 root root   22 Jun  8 10:18 /usr/lib/libshaderc_shared.so -> libshaderc_shared.so.1
-rwxr-xr-x 1 root root 3.4M Jun  8 10:18 /usr/lib/libshaderc_shared.so.1
% ll /usr/lib/libglslang*
-rwxr-xr-x 1 root root  26K Jul  9 19:02 /usr/lib/libglslang-default-resource-limits.so
-rwxr-xr-x 1 root root 1.7M Jul  9 19:02 /usr/lib/libglslang.so

It's especially confusing that libSPIRV is only available as a .so while libSPIRV-Tools is available as a .a.

What way would you consider "proper" to detect the state of the system? I would prefer detection for every single library and if it exists, link it. However, I don't want to mess with the detection code too much so that I don't break everyone elses setups.

I will file a pull request shortly showing you the code with minimal changes that works on my system but I wouldn't consider that a proper solution.

[antiagainst]

That is strange. libSPIRV-Tools-opt should also have a shared library. It seems something is not correctly configured in upstream.

[faulesocke]

Can't find one, neither on Void nor on Arch.

[Abendstolz]

May I jump in here?

I am running Archlinux and to me it looks like I have a similar but not the same problem. I get

Compiling shaderc-sys v0.5.1 error: could not find native static library SPIRV-Tools-opt, perhaps an -L flag is missing? error: aborting due to previous error warning: Checking for system installed libraries. Use --features > = build-from-source to disable this behavior warning: Found SPIRV. Linking libSPIRV & libglslang

error: Could not compile shaderc-sys.

I checked my /usr/lib for libSPIRV and found these:

.rwxr-xr-x 100k root 19 Jul 18:38 libSPIRV-Tools-link.so
.rwxr-xr-x 2.6M root 19 Jul 18:38 libSPIRV-Tools-opt.so
.rwxr-xr-x 181k root 19 Jul 18:38 libSPIRV-Tools-reduce.so
.rwxr-xr-x 1.1M root 19 Jul 18:38 libSPIRV-Tools-shared.so
.rwxr-xr-x 1.2M root 19 Jul 18:38 libSPIRV-Tools.so
.rw-r--r-- 1.0M root 19 Jul 19:46 libSPIRV.a
.rwxr-xr-x 538k root 19 Jul 19:46 libSPIRV.so
lrwxrwxrwx 11 root 19 Jul 19:46 libSPIRV.so.0 -> libSPIRV.so

looks like expected https://www.archlinux.org/packages/extra/x86_64/spirv-tools/files/

To me it's strange that a small cargo update broke everything while the only "suspicious" thing I could see in the Cargo.lock is a version increase from v0.5.0 to v0.5.1 - happening via https://github.com/vulkano-rs/vulkano

If you know of any workaround, I'd appreciate. Should I (we) wait until upstream is "fixed" and libSPIRV-Tools.a is included or what exactly is there to do?

[faulesocke]

I know two workarounds: At least on Void Linux I fixed my issues by building a static version of glslang. And my PR (see above) also fixed the issue for me.

[Abendstolz]

Thanks @faulesocke

Just for the record, here is my not-as-elegant fix:

[dependencies]
... 
# Fixing broken detection of glslang on my arch machine, see: https://github.com/google/shaderc-rs/pull/59
shaderc = "0.5.1"
shaderc-sys = "0.5.1"

[features]
shaderc_fix = ["shaderc/build-from-source"]

#[replace]
#"shaderc:0.5.1" = { git = "https://github.com/faulesocke/shaderc-rs", branch = "voidlinux"}
#"shaderc-sys:0.5.1" = { git = "https://github.com/faulesocke/shaderc-rs", branch = "voidlinux"}

as I couldn't get [replace] to work (*) I have to build with --features=shaderc_fix

(*) Meaning I still got the same error as in my previous comment

[faulesocke]

Try this ;)

[patch.crates-io]
shaderc-sys = { path = "../shaderc-rs/shaderc-sys" }

[mitchmindtree]

Just wanted to mention that I'm also running into the same issue as @Abendstolz on Arch Linux. I haven't been able to get the aforementioned patch working just yet but I haven't had a lot of time to tinker. I'll revisit this tomorrow and report back.

[antiagainst]

I've submitted 69adfd6, which should address this. Please try out and let me know (sorry I don't have a Linux box with the same configuration for this issue). If it's working I'll cut a release. Otherwise please reopen this issue. :)

[mitchmindtree]

Thanks @antiagainst!

Unfortunately, I'm still personally getting a build error when running cargo build, though cargo build --features "build-from-source" seems to work OK.

[mindtree@mindtree shaderc-sys]$ cargo clean                                                                                                  
[mindtree@mindtree shaderc-sys]$ cargo build --features "build-from-source"                                                                   
   Compiling cc v1.0.38                                                
   Compiling libc v0.2.60                                              
   Compiling cmake v0.1.40                                             
   Compiling shaderc-sys v0.6.0 (/home/mindtree/programming/rust/shaderc-rs/shaderc-sys)                                                      
warning: Requested to build from source                                                                                                       
    Finished dev [unoptimized + debuginfo] target(s) in 5m 38s         
[mindtree@mindtree shaderc-sys]$ cargo build                           
   Compiling shaderc-sys v0.6.0 (/home/mindtree/programming/rust/shaderc-rs/shaderc-sys)                                                      
warning: Checking for system installed libraries.  Use --features = build-from-source to disable this behavior                                
warning: Found and linking shared system installed SPIRV libraries.    
error: could not find native static library `SPIRV-Tools-opt`, perhaps an -L flag is missing?                                                 
                                                                       
error: aborting due to previous error                                  

Strange as libSPIRV-Tools-opt.so seems to be in my /usr/lib:

[mindtree@mindtree shaderc-sys]$ ls -ahl /usr/lib/libSPIRV*            
-rw-r--r-- 1 root root 1010K Jul 19 19:46 /usr/lib/libSPIRV.a          
-rwxr-xr-x 1 root root  526K Jul 19 19:46 /usr/lib/libSPIRV.so                                                                                
lrwxrwxrwx 1 root root    11 Jul 19 19:46 /usr/lib/libSPIRV.so.0 -> libSPIRV.so                                                               
-rwxr-xr-x 1 root root   98K Jul 19 18:38 /usr/lib/libSPIRV-Tools-link.so                                                                     
-rwxr-xr-x 1 root root  2.6M Jul 19 18:38 /usr/lib/libSPIRV-Tools-opt.so                                                                      
-rwxr-xr-x 1 root root  178K Jul 19 18:38 /usr/lib/libSPIRV-Tools-reduce.so                                                                   
-rwxr-xr-x 1 root root  1.1M Jul 19 18:38 /usr/lib/libSPIRV-Tools-shared.so                                                                   
-rwxr-xr-x 1 root root  1.2M Jul 19 18:38 /usr/lib/libSPIRV-Tools.so                                                                          

I might dig into this a little later to see if I can work out what's going on!

[antiagainst]

Ah, so you have a shared lib version of SPIRV-Tools-opt. The script right now only tries to find static version.

[antiagainst]

@mitchmindtree: Okay, submitted ff876b1. Let me know whether it works for you right now.

[mitchmindtree]

@antiagainst thanks so much, it works beautifully! ✨

Going from 5min source build to a 2sec compile time is a really great win :)

[antiagainst]

@mitchmindtree: Glad to hear that! I'll cut a release then. :)