Work around the infamous locked DLL issue

[gaborcsardi]

This issue is currently a place that I'll dump my notes about the investigation of this problem, and also thoughts about possible workarounds.

What R does

R 3.6.0, install binary from remote repo, or from a local ZIP file

• Staged installation does not matter here.

• If the package is loaded in the current R session, installations is refused with a warning:

> install.packages("curl")
Installing package into ‘C:/Users/Gabor/Documents/R/win-library/3.6’
(as ‘lib’ is unspecified)
Warning: package ‘curl’ is in use and will not be installed

• Now if we unload the package, but not the DLL, then we get a warning and a broken package is left around, i.e. only the dll is there, the rest is removed:

> unloadNamespace("curl")
> install.packages("curl")
Installing package into ‘C:/Users/Gabor/Documents/R/win-library/3.6’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/bin/windows/contrib/3.6/curl_3.3.zip'
 length 2996751 bytes (2.9 MB)
downloaded 2.9 MB

package ‘curl’ successfully unpacked and MD5 sums checked
Warning: cannot remove prior installation of package ‘curl’

The downloaded binary packages are in
        C:\Users\Gabor\AppData\Local\Temp\Rtmp2JeKkE\downloaded_packages
> library(curl)
Error in library(curl) : there is no package called ‘curl’

• The same happens in the package is loaded in another R session. Note that even if we install the exact same binary, we'll end up with a broken installation.

R 3.6.0, install source from remote repo, or from local tar.gz file, or from a local directory

• If the package is loaded in the current session, R still tries to install it. With the default staged install, moving the package to its final place produces a warning, but the installation still continues, and then the load-test fails, and the originally installed package is restored. So for some time the library contains a broken package. Finally, the install.packages() call gives a warning that the installation had a non-zero exit status:

** testing if installed package can be loaded from temporary location
*** arch - i386
*** arch - x64
WARNING: moving package to final location failed, copying instead
Warning in file.copy(instdir, dirname(final_instdir), recursive = TRUE,  :
  problem copying C:\Users\Gabor\Documents\R\win-library\3.6\00LOCK-curl\00new\curl\libs\x64\curl.dll to C:\Users\Gabor\Documents\R\win-library\3.6\curl\libs\x64\curl.dll: Permission denied
** testing if installed package can be loaded from final location
*** arch - i386
*** arch - x64
Error: package or namespace load failed for 'curl' in FUN(X[[i]], ...):
 no such symbol R_new_file_writer in package C:/Users/Gabor/Documents/R/win-library/3.6/curl/libs/x64/curl.dll
Error: loading failed
Execution halted
ERROR: loading failed for 'x64'
* removing 'C:/Users/Gabor/Documents/R/win-library/3.6/curl'
* restoring previous 'C:/Users/Gabor/Documents/R/win-library/3.6/curl'
Warning in file.copy(lp, dirname(pkgdir), recursive = TRUE, copy.date = TRUE) :
  problem copying C:\Users\Gabor\Documents\R\win-library\3.6\00LOCK-curl\curl\libs\x64\curl.dll to C:\Users\Gabor\Documents\R\win-library\3.6\curl\libs\x64\curl.dll: Permission denied

The downloaded source packages are in
        ‘C:\Users\Gabor\AppData\Local\Temp\Rtmp4EIpFM\downloaded_packages’
Warning message:
In install.packages("curl", type = "source") :
  installation of package ‘curl’ had non-zero exit status

• The same happens if the package is loaded in another session.

• If the already installed package is of the same version as the one being installed, then the load-test succeeds and you only get the warnings about the failed move and the failed copy:

** testing if installed package can be loaded from temporary location
*** arch - i386
*** arch - x64
WARNING: moving package to final location failed, copying instead
Warning in file.copy(instdir, dirname(final_instdir), recursive = TRUE,  :
  problem copying C:\Users\Gabor\Documents\R\win-library\3.6\00LOCK-curl\00new\curl\libs\x64\curl.dll to C:\Users\Gabor\Documents\R\win-library\3.6\curl\libs\x64\curl.dll: Permission denied
** testing if installed package can be loaded from final location
*** arch - i386
*** arch - x64
** testing if installed package keeps a record of temporary installation path
* DONE (curl)

• If the already installed package is not of the same version as the one being installed, but they are close enough for the load-test to succeed, then you get a package that looks OK, but essentially has undefined behavior: it may crash, it may freeze, it may fail, or produce the wrong results. :(

R 3.6.0, non-staged install, source from repo

The end result is the same as for staged install, although the warnings are slightly different. If the load-test fails, then we only get the warning about the failed copy of the DLL file.

** testing if installed package can be loaded
*** arch - i386
*** arch - x64
Error: package or namespace load failed for 'curl' in FUN(X[[i]], ...):
 no such symbol R_new_file_writer in package C:/Users/Gabor/Documents/R/win-library/3.6/curl/libs/x64/curl.dll
Error: loading failed
Execution halted
ERROR: loading failed for 'x64'
* removing 'C:/Users/Gabor/Documents/R/win-library/3.6/curl'
* restoring previous 'C:/Users/Gabor/Documents/R/win-library/3.6/curl'
Warning in file.copy(lp, dirname(pkgdir), recursive = TRUE, copy.date = TRUE) :
  problem copying C:\Users\Gabor\Documents\R\win-library\3.6\00LOCK-curl\curl\libs\x64\curl.dll to C:\Users\Gabor\Documents\R\win-library\3.6\curl\libs\x64\curl.dll: Permission denied

The downloaded source packages are in
        ‘C:\Users\Gabor\AppData\Local\Temp\RtmpIlhhOz\downloaded_packages’
Warning message:
In install.packages("curl", type = "source") :
  installation of package ‘curl’ had non-zero exit status

What pkginstall does currently

pkginstall only installs binary packages. To install a source package, needs to be R CMD build-t first, and then pkginstall extracts the binary, and install the extracted binary:

This is the current code:
https://github.com/r-lib/pkginstall/blob/0ca61acc1028e1f8d399bbb175824219bcee12e5/R/install_binary.R#L51-L92

In a nutshell:

1. First move the existing package dir to a temporary place. This works even if a process has the DLL open. If the move fails, it aborts.
2. Then tries to delete the moved dir, which may fail if another process as the file open. If it fails, it gives a warning.
3. Then it tries to move the extracted new package into the library. If this fails, then it aborts.

[gaborcsardi]

Here is what should happen ideally, from the user's point of view. (It might not be possible to achieve this, though.)

• The user should get an error that the installation is aborted. Maybe this could be restricted to the packages that have a DLL, but even w/o the DLL the installation might break the current session, via the lazy-load DB changes. So is is better to do it for all packages.

• The user should get some hints about whether the package is loaded into the current session or another one, and possibly also which one (e.g. its PID).

• If the current session is interactive, and the package is loaded in another session, then the user has the choice to

  • abort and try again
  • try the installation anyway, knowing that this might break the other session
  • kill the other R process(es)
  • install the package(s) in a temporary library, that is added to .libPaths(), so they can still be used in the current session, without breaking the other session.

[jimhester]

Then it tries to move the extracted new package into the library. If this fails, then it gives a warning.

Is this right? Shouldn't we throw an error if the new package can't be moved into the library?

Edit: it looks like it errors for this as well (to me)

https://github.com/r-lib/pkginstall/blob/0ca61acc1028e1f8d399bbb175824219bcee12e5/R/install_binary.R#L82-L83

[gaborcsardi]

@jimhester yes, sry, updated above!

[jimhester]

Having more diagnostic info would be nice, but in general what is wrong with what pkginstall does?

[gaborcsardi]

I think pkginstall is basically doing the right things. It could provide more info, and maybe even help you resolve the issue, see the edited #368 (comment) above.

I should have said it at the beginning, but there are two goals here, one is having a good way to handle this in remotes/pak/renv/etc. via a standalone file, the other is to see how we could write a patch to install.packages() that improves the situation.

[gaborcsardi]

@jimhester what do you think about #368 (comment) ?

[jimhester]

I think it sounds pretty good overall

[gaborcsardi]

One question is whether we want to do sg about source installation. pkginstall always runs R CMD build first, and maybe we could just do the same in remotes, but I am not sure.

Working around source installation is much more difficult, because we need to call install.packages().

[mattdowle]

More background to gather in one place :
Rdatatable/data.table#3056 (data.table issue)
Rdatatable/data.table#3237 (data.table permanent solution using compiled constant but each package would have to do this too and therefore isn't a general solution)
https://twitter.com/MattDowle/status/1084528873549705217 (twitter discussion)
https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17478 (R bug report)
https://tdhock.github.io/blog/2019/windows-dll/ (recent article by Toby Dylan Hocking)
http://r.789695.n4.nabble.com/R-pkg-install-should-fail-for-unsuccessful-DLL-copy-on-windows-tp4757281p4757302.html (problem on Bioconductor attributed to this)

[mattdowle]

I'm willing to help with a fix. But I don't like starting to work on something only to find that R-core are already working on it, or know of failed approaches they've tried before. I'm hoping for some kind of interaction with R-core on it before I look myself. But they haven't replied and the bug is still unconfirmed status.

[gaborcsardi]

@kalibera is working on it already, apparently:
wch/r-source@0434ff2
wch/r-source@ebef820

[mattdowle]

Seems like R-devel fixes it. 17478 has been closed.
https://twitter.com/MattDowle/status/1141443694735704065
I suppose I should retest latest R-devel on Windows then.

[kalibera]

FYI, the fixes have been ported to R-patched as well.