error when fetching package sources from private GitHub repo

[ras44]

I'm using packrat and install_github to pull a specific version of a package from a private github repo:

install_github('<MY_USERNAME>/<MY_REPO>@v0.2.0',
               host='https://<MY_INTERNAL_GITHUB_URL>/api/v3',
               auth_token= <MY_GITHUB_PAT>
)

When I run

packrat::snapshot()

I receive:

Adding these packages to packrat:
                          _      
    <MY_REPO>   0.2.0

Fetching sources for <MY_REPO> (0.2.0) ... Error in snapshotSources(project, activeRepos(project), allRecordsFlat) : 
  Errors occurred when fetching source files:
Error in untar2(tarfile, files, list, exdir, restore_times) : 
  incomplete block on file

I've checked the packrat/ directory in the project, and it contains what looks like the repo in

packrat/lib/x86_64-redhat-linux-gnu/3.4.3/<MY_REPO>

By that, I mean that directory contains DESCRIPTION, INDEX, NAMESPACE, extdata, help, html, Meta, R, tests files or directories.

On the other hand,

packrat/src/<MY_REPO> 

is empty.

I've checked the DESCRIPTION file in packrat/lib/x86_64-redhat-linux-gnu/3.4.3/<MY_REPO> and it contains the correct URL for the repo.

I've also tried setting the GITHUB_PAT environment variable to <MY_GITHUB_PAT>, but it still fails.

From what I've read, packrat uses the URL in description to find where the file was pulled from, and should be able to work with packages that were installed via devtools::install_github.

Any suggestions for what could be going on? Thanks in advance for your time.

[kevinushey]

Is devtools installed and available in this project? What's the output of

packrat:::canUseGitHubDownloader()

in that project?

[ras44]

Thanks for your quick response. yes, devtools is installed and available via:

install.packages('devtools')
library('devtools')

The output of packrat:::canUseGitHubDownloader() is:

> packrat:::canUseGitHubDownloader()
[1] TRUE

I think the issue here is that this is a repo hosted on a internal corporate github. By "private repo" I meant a repo hosted on our private internal github, not on github.com, but I realize this may cause confusion. I will update the title of the issue as well.

[kevinushey]

What do you see if you try to execute

url <- "https://api.github.com/repos/rstudio/packrat/tarball/cd0f9a4dae7ea0c79966b6784b44d7e4e4edadad"
file <- "package.tar.gz"
packrat:::githubDownload(url, file)

Is the archive downloaded as expected? (It's possible that Packrat instead receives a 403 error page). Replace the above url with the path to your private repository as appropriate.

[ras44]

closing, as I believe this is a duplicate of #442

[ras44]

@kevinushey I've submitted a pull request that illustrates what's going on and provides a limited fix. The githubDownload function you mentioned is working correctly. The issue was that the Archive URL was hard-coded as api.github.com, but in our case the internal github is not at api.github.com; it's accessible via an internal subdomain like gh.internal.acme.com. This is accessible in the DESCRIPTION file via the RemoteHost variable.

[ras44]

Have you tried using the local_repo option via something like:

cd ~
mkdir local_repo
cd ~/local_repo
git clone https://gh.corporate.repo.com/user/myRPackage.git
cd myRPpackage
git checkout v<RELEASE_VERSION>

then in your R script set the local_repo option to point to ~/local_repo:

## Set the local repositories for an existing packrat project
packrat::set_opts(local.repos = "~/local_repo")
packrat::install_local('myRPackage')

You might also want to make sure that you have all the correct internal IPs in your /etc/hosts file, so that when you attempt to run the bootstrap packrat command from within the docker container, it knows where to pull the packages from. I believe in ours we had to add a line with an IP for both gh.corporate.repo.com and codeload.gh.corporate.repo.com.