• We assume this will be a small change, and we'll plan to reingest (and include release note that installations should reingest)

I picked this up and poked around a bit.

I found an RData file from 2007 at https://doi.org/10.7910/DVN/RWUY8G/90LTEO and it can be successfully ingested. Here's how the first 32 bytes look:

00000000: 5244 5832 0a58 0a00 0000 0200 0202 0100  RDX2.X..........
00000010: 0104 0000 0004 0200 0000 0100 0010 0900  ................

Here are just the first five bytes, which are what our code cares about:

00000000: 5244 5832 0a                             RDX2.

So yes, for these RData files from 2007 our code works well, the regex makes sense ("RDX2" is right there).

However, the first five bytes look pretty wildly different for modern RData files.

Here are the first 32 bytes look for an RData file from 2019 from https://doi.org/10.7910/DVN/FTYHPJ/JAJHKH

00000000: 1f8b 0800 0000 0000 0003 ecfd 3d73 1c77  ............=s.w
00000010: dee7 6be6 f4e0 a108 910c 46ac 0c19 3a11  ..k.......F...:.

Here are the first 32 bytes for an RData file I just created locally with R 3.6.3 following instructions from https://statisticsglobe.com/r-save-load-rdata-workspace-file

00000000: 1f8b 0800 0000 0000 0003 0b72 8930 e68a  ...........r.0..
00000010: e062 6060 6066 6066 0362 5620 9381 3534  .b```f`f.bV ..54

I haven't come across an RDX3 file yet but maybe I'll try to create one myself.

In short, the examples seem to require a different regex or perhaps even a different technique to detect that they're RData files.

Other thoughts:

• It would be nice to have some RData files to test with in https://github.com/IQSS/dataverse-sample-data
• There is interest in support for the RDS format at replace download option "RData" with RDS #6678.

However, the first five bytes look pretty wildly different for modern RData files.
...

You are looking at the compessed stream. You're supposed to uncompress it, then look at the first 5 bytes.

For an example of an "RDX3" file - see the dataset that triggered this: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/OFQUAN
(this was reported in #dv-general, on Apr. 7).
The .RData files that are recognized as "Gzip Archive" in that dataset are examples of this new RData flavor.

@landreev thanks! Yes, now I see "RDX3"...

gunzip -c S02-dataTurkey.RData | xxd -l 32

00000000: 5244 5833 0a58 0a00 0000 0300 0306 0100  RDX3.X..........
00000010: 0305 0000 0000 0555 5446 2d38 0000 0402  .......UTF-8....

I created pull request #6832 because it was enough to get Dataverse to attempt to ingest the file.

However, I don't have a working Rserve installation on my Mac. This is what I did:

Start R REPL:

R

Then from the R REPL:

> install.packages("Rserve")

Start RServe:

R CMD Rserve

But then I'm getting weird errors even on RData files that should ingest.

So I plan to spin up my pull request on Linux and try it there.

There's a section in the installation guide that explains how to install R components that you need to ingest R files.

Yes, I'd prefer that we test it before we move the issue along. There is a non-zero chance that the R code we use for ingest will somehow stop working on these newer files.
I could test it too, but would have to finish other things first.

@landreev thanks I ended up testing on a server but it's good to know that devs can use the installation guide instructions. I got an RDX3 file to ingest 🎉 so I moved pull request #6832 to code review. I'll take this issue off the board, as is our convention.

Good.