Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows ARM64 source build - fails if parallel make jobs used (winlibs download race condition) #340

Closed
andrjohns opened this issue Jul 16, 2024 · 2 comments

Comments

@andrjohns
Copy link

Currently experiencing this on Windows ARM64 (so not sure if present in x86) with all packages that need winlibs static libs (where the library isn't already in rtools).

If I have MAKEFLAGS=-j2 (or any value > 1) set in my .Renviron, the installation will fail at the linking stage due to libraries not being found:

install.packages("curl", repos = "https://cloud.r-project.org")
...
"C:/PROGRA~1/R-AARC~1/R-devel/bin/Rscript.exe" "../tools/winlibs.R"
clang -shared -s -static-libgcc -o curl.dll tmp.def callbacks.o curl.o download.o escape.o fetch.o findport.o form.o getdate.o handle.o ieproxy.o init.o interrupt.o multi.o nslookup.o options.o reflist.o split.o ssl.o typechecking.o utils.o version.o winidn.o writer.o -L../windows/libcurl/libclang-17.0.6 -L../windows/libcurl/lib -lwinhttp -lcurl -lnghttp2 -lssh2 -lz -lssl -lcrypto -pthread -lgdi32 -lws2_32 -lcrypt32 -lbcrypt -lwldap32 -L/aarch64-w64-mingw32.static.posix/lib -L/aarch64-w64-mingw32.static.posix/lib -LC:/PROGRA~1/R-AARC~1/R-devel/bin -lR
lld: error: unable to find library -lnghttp2
clang-17: error: linker command failed with exit code 1 (use -v to see invocation)
[1] TRUE
echo '#include <curl/curl.h>' | clang  -I../windows/libcurl/include -DCURL_STATICLIB -DSTRICT_R_HEADERS -DR_NO_REMAP -std=gnu99 -E -xc - | grep "^[ \t]*CURLOPT_.*," | sed s/,// > ../tools/option_table.txt
no DLL was created
ERROR: compilation failed for package 'curl'

Which appears to be caused by $(SHLIB) target reaching the linking stage before the winlibs target has completed. I'd guess that this hasn't been seen to-date on x86_64 because compilation is generally slower than under ARM, so the winlibs target has usually finished before $(SHLIB) reaches linking.

From local testing, an easy fix appears to be just making the dependency explicit in Makevars.win:

- all: clean winlibs
+ $(SHLIB): $(OBJECTS)
+ $(OBJECTS): winlibs

But I'm not sure whether there's an easier fix which doesn't require updating all winlibs R packages. Thanks!

@jeroen
Copy link
Owner

jeroen commented Jul 16, 2024

Yes I think that is the only solution.

FWIW this is unrelated to arm but about using parallel make. Afaik parallel make on Windows was never really supported because there are many other problems with it, so we didn't bother to test it.

@jeroen jeroen closed this as completed in 5ae10a9 Oct 24, 2024
@jeroen
Copy link
Owner

jeroen commented Oct 24, 2024

I'm sorry I tried it but it just broke the build, so I hard to revert. The Windows tooling is a mess these days, I don't care enough to fix this. Just don't use parallel make.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants