git remotes fail if the repository contains a submodule

[dgkf]

When trying to download a package from a git repo that contains a submodule, I'm encountering an error arising from an attempt to write out an object that doesn't exist from the downloaded git files:

pkgdepends/R/git-protocol.R

Line 549 in 2dffed6

writeBin(parsed[[tr$hash[l]]]$raw, opath)

pkgload::load_all()
sy(async_git_download_repo("https://enterprise.com/user/repo.git"))
# Error in writeBin(parsed[[tr$hash[l]]]$raw, opath) :
#   can only write vector objects

Debugging this line, the error occurs when opath is the path to the submodule, resulting in a parsed[..] that returns NULL.

[gaborcsardi]

git remotes indeed do not support submodules.

[dgkf]

Ah, I didn't realize this was intentional. Is this something pkgdepends might support in the future or is it considered decidedly out-of-scope?

The particular package that prompted the error doesn't even require the submodule to build (it's excluded via .Rbuildignore, but used for tracking related dependencies used to build distributed inst files).

At least for this narrow use case, tracking which packfile elements can't be unpacked and suppressing issues when they're .Rbuildignore'd would help to mitigate some of these failures. If they aren't included in .Rbuildignore, having a listing of which files failed to come over in the packfile would also improve the error message emitted when submodules are the culprit.

[andrewherren]

Hi there, I have a C++ library with separate R and python bindings, all within the same organization. The C++ code is included in the R package as a submodule.

Installation from GitHub works fine with remotes::install_github but as noted in this issue, pkgdepends does not download submodules which causes downstream pak::pkg_install to fail. I'd like to use r-lib's CRAN check and pkgdown actions which require me to set up a package that builds with pak::pkg_install.

Is there a "best practice" way to let my R package depend on C++ code from another repo that plays nicely with pkgdepends? I see for example that duckdb has a vendoring script which runs in a github action, but I'm curious if there is a simpler workaround?

Thank you in advance! r-lib's tools have been a huge upgrade to my workflow!

[andrewherren]

Quick update (in case anyone else with a similar project structure stumbles upon this thread), the workaround is in fact remarkably simple and doesn't have anything to do with pak or pkgdepends. Just modify the checkout@v4 action in the _pkgdown.yml workflow generated by usethis::use_pkgdown_github_pages() to check out submodules recursively via

- uses: actions/checkout@v4
  with:
    submodules: 'recursive'

[gaborcsardi]

git remotes now support submodules, but it is opt in via a config value. Do you think it should be the default? We can also keep it opt-in now, and make it the default later?

[andrewherren]

I'll let @dgkf weigh in as well, but I think it's probably fine to make it opt-in as long as it's clearly documented. Seems like only a handful of us are asking about it on Github issues.