FR: create a .rscIgnore files to ignore files from being bundled

[cderv]

It is already possible to provide a appFileManifest in deployApp to list the files that should be bundle.

I am often in the case like with .gitignore or .Rbuildignore : it is easier to tell what we don't want at all instead of listing everything we want.

I would find it interesting to have a feature like that, a .rscignore file that would allow to always exclude some files to be bundled by rsconnect.
This is like .Rbuildignore that prevent bundling in R CMD build.

We encountered this need in some scenarios where

• we have projects that push in RStudio Connect
• the development is done externally and the code is delivered in a gitlab. when tagged, it should be published in one of the url
• One person of the project in charge of deployment need to publish. This require to recreate the dev env and publish the file. Some files used in development do not need be be published but is is not easy to exclude them.

Publishing unneeded file is not a problem, but can cause issues because they are scanned for dependencies and can increase the bundle size.

What do you think ?

In a way this issue relates to #342 about how to manage bundle files.

[ColinFay]

Linked issue in the {golem} issue board: ThinkR-open/golem#110

[kevinushey]

FWIW, renv also has the concept of a .renvignore that basically works just like .gitignore. If rsconnect could switch from using Packrat to renv, this fix could come for free.

(As an intermediate step, renv could be used to generate a Packrat lockfile, so that all the existing deployment machinery could work as-is -- just the dependency discovery would happen with renv)

[jmcphers]

Wouldn't that just keep files from being included when computing dependencies? (The goal here is preventing files from being included in the app bundle.)

[kevinushey]

I thought (perhaps erroneously) that those were one and the same. If not, then please disregard :-)

[cderv]

I think there would be some usecase where you want to use renv for a project, and with some files to publish and some other you want ignore from being bundle, but they be part of your renv project.

For rsconnect, today this is only possible to select file to bundle with a file manifest only for file to include. You can't provide excluding pattern.

I really believe that providing a mechanism to specify which file to bundle (include and exclude) would be really useful - for such framework as golem, it would be for sure.

I think depending on renv for this kind of feature would not be so simple to understand for users. And what if you are using renv also yourself for your project? There would be a double usage: by the user and by rsconnect as a tool to create the bundle. could there be conflict ?

Otherwise, would it be possible (and a good idea?) to just take the mechanism you put in renv to be of use in rsconnect ? I am thinking about renv_renvignore_parse

[slopp]

I think Jonathan's take is accurate. The goal in this case is to specify the bundle more explicitly instead declaring its contents in the deployApp call or in the IDE publish pane.

We would need to be careful here, as any updates to rsconnect would also need to be made to the Git deployments supported by RSC (which does not use reconnect for the bundling step) and perhaps even the Jupyter deployment scenario (for consistency).

All that said, this is a relatively common request and I think it'd be a very handy feature!

cc: @aronatkins

[ColinFay]

Hey,

Any update on this issue?

I've noticed that this behaviour is possible if you do it "manually", i.e. when you click on the RStudio "blue button", and uncheck files, this will create an entry in the rsconnect/...dcf file :

ignoredFiles: dev/01_start.R|dev/02_dev.R|dev/03_deploy.R|vignettes/.gitignore|vignettes/graphs.html|vignettes/graphs.Rmd

Then if I call rsconnect::deployApp() after having done this manually, I'm under the impression that the files will be ignored.

Am I correct in understanding that ignoring files is only available if one does the deployment through point and click?

Happy to try and implement this, would you review a PR that does that?

[andyquinterom]

I just opened a pull request for including this feature

[cderv]

It seems like this has been added in #502 (thanks @andyquinterom!) and it now merged and available last version.

There is no wildcard support but that is like the manifest file in a way and it is a good start that this feature is available now.

We would need to be careful here, as any updates to rsconnect would also need to be made to the Git deployments supported by RSC (which does not use reconnect for the bundling step) and perhaps even the Jupyter deployment scenario (for consistency).

@slopp I don't know if you concerns here were checks. Probably.

I think this could be closed now, right ?

[pat-s]

Would be great to also document this in the RSC admin guide :)

[nick-youngblut]

Would be great to also document this in the RSC admin guide :)

I've been trying to finds docs on the .rscignore file. The only mention that I can find in the rsconnect package docs is for deployApp(), and that doesn't provide much info:

The files and directories to bundle and deploy (only if upload = TRUE). Can be NULL, in which case all the files in the directory containing the application are bundled, with the exception of any listed in an .rscignore file. Takes precedence over appFileManifest if both are supplied.

For instance, the docs don't state whether the .rscignore file works just like a .gitignore file. It would also be great to be able to specify the path to the .rscignore file (e.g., just use an existing .gitignore file instead of creating a new dot-ignore file).

Lastly, it appears that Workbench (2022.07.2) doesn't show the .rscignore file in the Files, Plots, Packages... panel. I'm guessing that this dotfile is not included in the acceptable list of dotfiles to show in the panel (unlike .Rprofile, .gitignore, etc). Does anyone know if the .rscignore file is visible in newer versions of Workbench?

[nick-youngblut]

For what I can tell from the code:

  # checks for .rscignore file and excludes the files and directories listed
  if (".rscignore" %in% contents) {
    ignoreContents <- readLines(".rscignore")
    contents <- setdiff(
      x = contents,
      y = ignoreContents
    )
  }

Wildcards will not work (e.g., *.png), which is a major feature of .gitignore files. It appears that absolute versus relative file paths are also not handled (Is contents a vector of relative file/directory paths? What if the developer uses absolute file paths?).

Are there any plans on updating this feature? Explicitly stating all files/directories that should be included/excluded in a deployment is hard to sustainably maintain.

[hadley]

Improved documentation will be part of #684. Otherwise, we don't have current plans to enhance this functionality.

@nick-youngblut you might be better of generating appFiles() yourself.

[nick-youngblut]

Thanks @hadley for updating the documentation.

So I either have to list all files/directories to include or all files/directories to ignore. I'm surprised that Connect content developers are generally OK with this functionality, given that everyone is used to the high functionality of .gitignore (and similar), and the burden of maintaining a full list of file/directories to include or exclude from publishing on Connect. I would think that many users would want to just symlink their .gitignore to .rscignore in order to ignore the same files for pushing & publishing (as I would like to do).

From the code shown above ((".rscignore" %in% contents) ..., it appears that paths are not normalized (as in os.path.normpath in python) prior to the setdiff, so /path/to/directory/ is considered different from /path/to/directory. Is that correct? If so, would you accept a PR in which I add path normalization (and maybe wildcards too)?

[hadley]

@nick-youngblut if you want to implement it, a PR would be awesome! But I think you should wait until after #684 has been merged, since that pulls out the bundling code into something easier to modify.

[trangdata]

Thank you all for the discussion! Adding that this would also be super helpful for deploying git-backed content. It seems that manually defining appFiles in writeManifest doesn't seem to ignore these files in my git-backed bundle.