Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

data.table::update.dev.pkg() warning or error #4403

Closed
drag05 opened this issue Apr 27, 2020 · 16 comments · Fixed by #4404
Closed

data.table::update.dev.pkg() warning or error #4403

drag05 opened this issue Apr 27, 2020 · 16 comments · Fixed by #4404

Comments

@drag05
Copy link

@drag05 drag05 commented Apr 27, 2020

data.table v 1.12.8:

applying

data.table::update.dev.pkg()

gives the following message in R v. 4.0.0, R Studio v 1. 2.5042 both with Admin rights:

R data.table package has been updated to NA (1.12.8)
Warning message:
In file.copy(savedcopy, lib, recursive = TRUE) :
  problem copying C:\R\R-4.0.0\library\00LOCK\data.table\libs\x64\datatable.dll to C:\R\R-4.0.0\library\data.table\libs\x64\datatable.dll: Permission denied
> sessionInfo()
R version 4.0.0 (2020-04-24)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.0.0    tools_4.0.0       data.table_1.12.8
@jangorecki
Copy link
Member

@jangorecki jangorecki commented Apr 27, 2020

It is actually the same problem as I have seen on CRAN windows checks today. I was hoping it is a local CRAN issue related to new 4.0.0 setup, it seems it is not.
Thanks for reporting. @drag05 could you just confirm you are using rtools 4.0?
Problem is probably related to installing from source on Windows.

@tlapak
Copy link
Contributor

@tlapak tlapak commented Apr 27, 2020

I have the same issue on Windows 10 using R 3.6.2. update.dev.pkg() downloads the binaries from gitlab, no? I don't have rtools installed on that machine. Specifying a lib, which I put in my documents folder, let me run the "update" successfully.

Edit: Actually using R 3.6.2

> sessionInfo()
R version 3.6.2 (2019-12-12)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 17763)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.1252 
[2] LC_CTYPE=English_United Kingdom.1252   
[3] LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C                           
[5] LC_TIME=English_United Kingdom.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_3.6.2

Using

install.packages("data.table", type="source", repos="https://Rdatatable.gitlab.io/data.table")

works smoothly.

@mattdowle
Copy link
Member

@mattdowle mattdowle commented Apr 27, 2020

We've seen this before on CRAN checks several times; the workaround there is that we ping Uwe and he reruns. CRAN checks are run in parallel. I don't think data.table is reinstalled by CRAN each time but sometimes it is, and when it does, if another package that uses data.table is running its checks at the same time, the install fails because the .dll is in use by that other package. Prior discussions with Uwe and others, if I remember correctly, are essentially that this is a Windows feature and there is no known way to alleviate it (other than perhaps adding to the error message a hint to the user to close all other R sessions and perhaps reboot too). Windows locks its dll's. Unlike Linux and MacOS which have a mechanism to allow several versions of the same .so to be in use.

I don't think it is R 4.0.0 per se. It's just that the event of R 4.0.0 release triggers people to update who perhaps do not update very often.

Could the people who are having troubles ensure that all R sessions are shutdown, and then try upgrading/installing data.table again? If you have a .RProfile, and that .RProfile loads data.table, then you have to turn that off otherwise data.table loads on startup and the lock is opened on that dll. This applies to every R package with a .dll as far as I know.

@jangorecki
Copy link
Member

@jangorecki jangorecki commented Apr 27, 2020

I am able to reproduce problem. update.dev.pkg() seems to be broken in R 4.0.0 because R holds the data.table package (because function is in the package) more strictly than before.

To install latest master please use this for now

install.packages("data.table", repos="https://Rdatatable.gitlab.io/data.table")

Just for the record, building from source on windows for R 4.0.0

## download and install
# https://cran.r-project.org/bin/windows/base/R-4.0.0-win.exe
# https://cran.r-project.org/bin/windows/Rtools/rtools40-x86_64.exe
set PATH=C:\rtools40\usr\bin;%PATH%
R                 ## ensure R 4.0.0
Sys.which("make") ## ensure rtools40 path

# install 1.12.8
install.packages("data.table", type="source", repos="https://cran.r-project.org")
# install devel
install.packages("data.table", type="source", repos="https://Rdatatable.gitlab.io/data.table")

@jangorecki jangorecki self-assigned this Apr 27, 2020
@drag05
Copy link
Author

@drag05 drag05 commented Apr 27, 2020

@mattdowle
R 4.0.0 asks for all previously R v < 4 installed packages to be re-installed. My issue popped up while I was reading about fcase() in News: the data.table had already been installed anew as requested ...

@drag05
Copy link
Author

@drag05 drag05 commented Apr 27, 2020

@jangorecki

I have tried twice and got this every time:

> install.packages("data.table", repos="https://Rdatatable.gitlab.io/data.table")
trying URL 'https://Rdatatable.gitlab.io/data.table/bin/windows/contrib/4.0/data.table_1.12.9.zip'
Content type 'application/zip' length 2675878 bytes (2.6 MB)
downloaded 960 KB

Error in read.dcf(file.path(pkgname, "DESCRIPTION"), c("Package", "Type")) : 
  cannot open the connection
In addition: Warning messages:
1: In download.file(url, destfile, method, mode = "wb", ...) :
  downloaded length 983040 != reported length 2675878
2: In unzip(zipname, exdir = dest) : error 1 in extracting from zip file
3: In read.dcf(file.path(pkgname, "DESCRIPTION"), c("Package", "Type")) :
  cannot open compressed file 'data.table/DESCRIPTION', probable reason 'No such file or directory'

and

> sessionInfo()
R version 4.0.0 (2020-04-24)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.0.0 tools_4.0.0   

also

> Sys.which('make')
                              make 
"C:\\rtools40\\usr\\bin\\make.exe" 

@jangorecki
Copy link
Member

@jangorecki jangorecki commented Apr 27, 2020

@drag05 install.packages warnings seems like issue with your connection. It doesn't download complete zip archive. AFAIK it is not related to the issue reported in this thread, and should be possible to "fix" by a retry (or more retries, maybe router restarting can help as well?).
Original issue has been addressed in PR #4404.

@drag05
Copy link
Author

@drag05 drag05 commented Apr 27, 2020

@jangorecki

Thank you for the tip! I was inside my VPN. Now it works both ways - it seems:


> install.packages("data.table", repos="https://Rdatatable.gitlab.io/data.table")
trying URL 'https://Rdatatable.gitlab.io/data.table/bin/windows/contrib/4.0/data.table_1.12.9.zip'
Content type 'application/zip' length 2675878 bytes (2.6 MB)
downloaded 2.6 MB

package ‘data.table’ successfully unpacked and MD5 sums checked

The downloaded binary packages are in
        C:\Users\Dragos\AppData\Local\Temp\Rtmp2VNd5T\downloaded_packages

and

> data.table::update.dev.pkg()
R data.table package is up-to-date at dd7609e83132e19c9be80d71c73e8a8f95e19b27 (1.12.9)

So, I guess - no VPN allowed which may not be that good ...

@drag05
Copy link
Author

@drag05 drag05 commented Apr 27, 2020

Thank you for your prompt replies! I wonder if I should open a new issue involving VPN.
Please advise!

@drag05 drag05 closed this as completed Apr 27, 2020
@jangorecki
Copy link
Member

@jangorecki jangorecki commented Apr 27, 2020

I will re-open this issue, as I was able to reproduce the problem on windows. The issue was not just connection, but windows that locks dll, unloading namespace before reinstalling seems to remove the lock.

@jangorecki jangorecki reopened this Apr 27, 2020
@drag05
Copy link
Author

@drag05 drag05 commented Apr 28, 2020

In my case the data.table wasn't loaded at any time, nor other R sessions were open.
It could be that being inside a VPN triggered another issue.

@jangorecki
Copy link
Member

@jangorecki jangorecki commented Apr 28, 2020

when you call data.table:: it automatically loads package namespace, do not confuse with attaching a namespace.

@drag05
Copy link
Author

@drag05 drag05 commented Apr 28, 2020

@jangorecki
Now I see what you meant; if I understand you correctly, the functions after the data.table:: should unloadNamespace('data.table') before doing everything else.

@jangorecki
Copy link
Member

@jangorecki jangorecki commented Apr 28, 2020

not functions, but this particular function, because it attempts to upgrade this namespace

@drag05
Copy link
Author

@drag05 drag05 commented Apr 28, 2020

I have miswritten. I meant it could happen to other namespaces too. Couldn't it?

@jangorecki
Copy link
Member

@jangorecki jangorecki commented Apr 28, 2020

It is related only to reinstalling pkg. Using :: in general is fine, but when you upgrade pkg it is best to have its namespace unloaded. Locks during upgrades on Windows happens quite commonly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants