R integration with Dataverse

[hamidtam]

What steps does it take to reproduce the issue?
Upload a new datafile to in dataset, download said RData version of datafile

When does this issue occur?
When attempting to download RData version of datafile

Which page(s) does it occurs on?
datafile page (access file button -> download options)

What happens?

Frontend error message:
No download, redirect to page:
{"status":"ERROR","code":404,"message":"datafile access error: requested optional service (image scaling, format conversion, etc.) could not be performed on this datafile."}

Backend error message:
Cannot connect: Connection refused|#]

To whom does it occur (all users, curators, superusers)?
testing was done with Admin account

What did you expect to happen?
Successful RData file download

Which version of Dataverse are you using?
6.0

Any related open or closed issues to this bug report?
https://groups.google.com/g/dataverse-community/c/JG6NkJ7ZQW0

Workaround Research:
Integrate a different library to replace Rserve,
JRI / rJava: https://www.rforge.net/JRI/
RCaller: https://github.com/jbytecode/rcaller

[pdurbin]

Related:

• R data download doesn’t work #9490
• The option to download tabular data in RData format should not appear in the dropdown menu if a Dataverse installation has not been configured to handle RData #8711
• replace download option "RData" with RDS #6678

@hamidtam do you have Rserve installed? I see you wrote "Integrate a different library to replace Rserve".

[hamidtam]

Yes Rserve is installed, but as the other tickets indicate Rserve is unreliable. Given how RData is used by many on our side we were planning on looking at alternatives to fix this with other libraries. One of out plans is to integrate download options for SPSS, STATA, SAS files, something an existing R integration can do.

[donsizemore]

the error you're getting is "connection refused." have you verified that Rserve is running, listening on the correct port, and that your jvm-options tell Dataverse how to connect?

[hamidtam]

@donsizemore Thanks for your input

Yes Rserve running locally, port 6311 default. Same errors previously.

tameem@Tameems-MacBook-Pro dataverse % R CMD Rserve --vanilla

R version 4.3.2 (2023-10-31) -- "Eye Holes"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin20 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

Rserv started in daemon mode.

tameem@Tameems-MacBook-Pro bin % sudo lsof -i :6311 
COMMAND   PID   USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
Rserve  28766 tameem    3u  IPv4 0x930658fa76d9a547      0t0  TCP localhost:6311 (LISTEN)

$ asadmin list-jvm-options

jvm-options options set in through Payara

-Ddataverse.rserve.port=6311
-Ddataverse.rserve.host=localhost
-Ddataverse.rserve.user=rserve
-Ddataverse.rserve.password=rserve
Command list-jvm-options executed successfully.

[landreev]

As I just said on slack, the fact that you are getting a flat out "connection refused" is suspicious. Is it just some MacOS firewall issue, and/or some weirdness with what it routes as "localhost"?
What happens when you try
telnet localhost 6311 on the command line?
If you get the same "connection refused", please try 127.0.0.1 and ::1 instead and see if you have better luck with either of these?

This issue appears to have fallen through the cracks, apologies.

[landreev]

I'll try to have replacing Rserve with either rJava or RCaller prioritized. That part may not be difficult at all.
As I tried to explain on slack, the actual conversion may still be unreliable an flaky, because the relevant code underneath, both in Java and R, is very old and hasn't been actively maintained. But we can try and address it one step at a time.

[hamidtam]

Issue fixed by moving from Mac OS environment to Linux. Feature to integrate download files from SPSS, STATA, SAS files was completed locally and waiting for testing and integration.