Global search and replace

[Robinlovelace]

Another feature request (hence lack of system info), this one is for global search and replace functionality, perhaps implemented in the Window that pops-up when you press Ctrl-Shift-F. I opened this a while back on the old feature request site: https://support.rstudio.com/hc/en-us/community/posts/205333518-Global-search-and-replace

[ronblum]

Thanks @Robinlovelace. Just to spell it out—you'd like to do a replace (interactive or automatic) on all matching files in the directory tree?

[Robinlovelace]

Yes. Interactive would be a bonus. Currently I use regexxer for such tasks, as illustrated below:

[ronblum]

Thanks for the example! I've added this to the enhancements list to review for future releases.

[Robinlovelace]

There are also good command line programs that can solve this problem that may be worth thinking about, such as sed or rpl which I just discovered. The 'interactive mode' wasn't as interactive as I was hoping on that last one (as illustrated in my attempt to use it below). Unless you use such CLI tools regularly, the incantations are tricky to remember.

[jennybc]

Related but perhaps should be separate issue: with the current "Find in Files" functionality, I get results in a new tab, in the same pane as Console, Terminal, etc. But this text cannot be copied to the clipboard, which can be frustrating if I'm, say, working through a list of things I need to attend to. It is great that I can click on this text to go to specific loci and I imagine there is tension between that and the ability to copy.

[faridcher]

related to SO post:
https://stackoverflow.com/questions/45894069/find-and-replace-text-in-all-files-rstudio/52096986

[billdenney]

I have the same issue relatively commonly.

I found the follow SO answer helpful for an R function that does the search and replace for you: https://stackoverflow.com/a/53397980/3831096

I'll also occasionally switch over to emacs for dired mode.

When implementing the function, it would help to show all the found items before the replace as is done for the current ctrl-shift-f command (optionally) so that the user could confirm that they will not make a mistake across many files.

[NanisTe]

Hi,

I think it would already help if the existing find and replace function would not always empty its entries when switching to another file.

Thanks for considering a solution for us.

[ju-rgen]

What is the problem in resolving this basic issue?

[Robinlovelace]

Heads-up @NanisTe I've also wondered that, and I imagine that issue would be quicker to fix. Assuming RStudio developers (thanks for awesome open source software guys!) agree, I would suggest opening a separate issue on that topic.

@ju-rgen unsure what the problem is or if there is one, I imagine that this issue is not a priority but that it's climbing up the list based on the number of upvotes! Also I suspect it takes time to think over the option space of ways to implement this. I'd be interested to know what the options are and how much work it would be, suspect that members of the community may be up for helping out with ideas/testing and maybe even code. Unfortunately I cannot program in languages in which RStudio is written. Any C++/Java devs out there know how to solve this? I'm not even sure which language the solution would be written in!

[jmcphers]

We have a beginner's guide to feature writing here: https://github.com/rstudio/rstudio/wiki/A-Beginner's-Guide-to-Writing-RStudio-IDE-Features

And the code for the existing find-in-files system (which would be extended to do find and replace) is here:

https://github.com/rstudio/rstudio/blob/master/src/cpp/session/modules/SessionFind.cpp

[Robinlovelace]

Nice, thanks for the link to the source code 👍 Almost enough of an incentive to learn C++, on top of all the other ones.

[Robinlovelace]

Awesome work guys, many thanks @melissa-barca for programming this into RStudio, looking forward to testing it!

[ronblum]

@Robinlovelace Thank you for the kind note, and not only for testing it, but for suggesting it in the first place—it's clearly a popular request, with 43 👍s!

[melissa-barca]

Thank you @Robinlovelace I look forward to your feedback :)

[Robinlovelace]

On a related note I've found a command line solution which has become my go-to find and replace tool, the rust crate sd: https://github.com/chmln/sd has a relatively high barrier to entry. Having a solution built into RStudio will save lots of time for lots of people so thanks again, looking forward to giving it a spin.

[ronblum]

Verified in RStudio Desktop 1.3.743 on MacOS 10.15.3 and RStudio Server 1.3.730 on Red Hat 7.8.

[Robinlovelace]

Quick follow-up, how does one use it? Checking out Version 1.3.776 and trying Ctl-F and Ctl-Shift-F it's not immediately clear.

[Robinlovelace]

Update, apologies, it is clear. Ctl-Shift-F does it:

[Robinlovelace]

Feedback having tested it, it seems to work on the UI but according to my tests does not actually replace the files... Can share a screenshot of this. Any ideas @ronblum ?

[Robinlovelace]

Sorry @melissa-barca I tried it on the latest daily version and it worked. The behaviour shown in the screenshot above was on Ubuntu 18.04. I'm very happy with the performance of this feature on the daily version. One question though, how to replace items one-by-one? I'll create another GIF to show what I mean...

[Robinlovelace]

In that example, it would be useful to have a button to replace one instance of 'geofabrik' at a time, not all instances want to be be replaced. Is that possible? Also it seemed something went wrong in that case, not sure what though and it definitely worked in a previous test.

Hope this is useful and not a hassle for you, I really appreciate the work that has gone into it and it looks awesome very useful new feature.

[Robinlovelace]

Second attempt shown below, with a bit of a cryptic error message:

[melissa-barca]

Unfortunately at this time you cannot replace instances one at a time, but it's definitely an enhancement we'll consider for the future. The cryptic error message you're seeing is generated when we attempt to overwrite the original file with a file containing the replaced text. Is this something you're seeing regularly?

[Robinlovelace]

Thanks for the clarification. Is it worth opening a new issue to track ideas/development related to such an enhancement? I imagine the hard part is now done and that to add a new button 'Replace' (as opposed to 'Replace all') would increase the usefulness of this newfound functionality.

Regarding the error message, I've only tested the new functionality 3 times. The first time it worked fine, the second 2 times it failed.

Reproducible example starting from the command-line:

git clone git@github.com:ITSLeeds/geofabrik
rstudio geofabrik/geofabrik.Rproj
# then in RStudio try replacing all instances of 'geofabrik' with 'osmextractr' to rename it

[melissa-barca]

@Robinlovelace Yes, please open a new issue for this enhancement or I will open one. Unfortunately I was not able to reproduce the example you provided.

[Robinlovelace]

Job done, heads-up @melissa-barca and many thanks for great work on this awesome IDE.

[Robinlovelace]

Sorry to report this, but the 'Replace all' button is not seeming to update files on the HD on the latest daily version as far as I can tell, as documented in the more-or-less reproducible GIF below (from the itsleeds/pct github repo):

[Robinlovelace]

On RStudio
Version 1.3.776
© 2009-2020 RStudio, Inc.
"Middlemist Red" (1d8a572, 2020-01-23) for Unknown OS
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) QtWebEngine/5.12.6 Chrome/69.0.3497.128 Safari/537.36

[melissa-barca]

Hi @Robinlovelace if you close and reopen the file does it contain the changes? We are tracking that the file doesn't always automatically refresh with the changes on issue #6237.

[Robinlovelace]

if you close and reopen the file does it contain the changes?

Not on my computer. I do git diff after clicking 'Replace all' and nothing has changed on the HD it seems.

[melissa-barca]

@Robinlovelace are you able to provide an strace of the rsession from immediately before performing the replace to immediately after? Could you also run and provide a diagnostics report (https://support.rstudio.com/hc/en-us/articles/200321257-Running-a-Diagnostics-Report) after attempting a global replace? Thank you!

[Robinlovelace]

Sorry @melissa-barca it's a busy week so not sure I'll have time before next week.

[melissa-barca]

@Robinlovelace I understand, thanks again for your help looking into this!

[Robinlovelace]

I've generated a diagnostic report. All kinds of things in there about my set-up. How should I share it?

[melissa-barca]

Hey @Robinlovelace I really only need to see the Log file portion without any of your set up info. Could you share this portion here? Thank you.

[Robinlovelace]

Cannot see anything relevant in the log file but this shows in the find and replace tab, for example:

[melissa-barca]

Hey @Robinlovelace this looks different from the original video you posted. When a replace fails, we expect the Error to be returned as it did in your most recent screenshot, highlighted in red -- though I need more information to debug this error. In your original video, the output was green as if the replace had occurred but then the files weren't updated. These are two different issues. Have you had a successful find/replace since your first round of testing?

[Robinlovelace]

Have you had a successful find/replace since your first round of testing?

Afraid not, tested briefly on laptop and desktop although have gone back to using the released version of RStudio for now.

[ronblum]

@Robinlovelace and @melissa-barca I'm unable to reproduce the issue on Ubuntu 18.04.4—it works fine. Is that the correct OS? Also, I am still running into issue #6048 (though it works on MacOS).

[Robinlovelace]

Sorry to be the bearer of bad news but global replace just crashed my RStudio session. Screengrab below. Context I asked it to do a globaly replace in the robinlovelace/cyclestreets repo on Ubuntu 18.04, it seemed to hang for ~10 seconds before I hit 'Stop' and it crashed.

[Robinlovelace]

On Version 1.3.938

[Robinlovelace]

I ended up using the command-line sd tool as follows to achieve the desired result:

sd 'cutoff = 20' 'cutoff = 50' $(fd R)

[Robinlovelace]

Robinlovelace/cyclestreets@028ed1c

[melissa-barca]

Hi @Robinlovelace I attempted to reproduce your issue on Ubuntu 18.04 but was able to perform the replace as expected. Are you able to consistently reproduce an example? If so could you please open a new issue with that description? Thanks!

[Robinlovelace]

I will open an issue if I can reproduce. Just a comment here for now as may be a one-off.

[AltfunsMA]

I have a similar issue. The first two images show output for a search and replace that failed completely. The errors reported are rather cryptic. The overall count of successes vs fails seems totally random: 64 vs 11, when actually nothing seems to have been replaced. Replacement didn't happen in the first file, either, although it reports no error.

When I tried a really small and simple test with just two fake files and one word to be replaced, the whole replacing got stuck. See third image below.

Running version 1.3.1073 on Ubuntu 18.04.4

1

2

3

[kevinushey]

@AltfunsMA; I've filed that as a separate issue here: #7736. Let's please direct any follow-up and future correspondence there. Thanks!