saving the content of a file modified inline with to text erases the content

[amtoine]

Describe the bug

i wanted to write a script to modify a bunch of config files of mine at once.

for the example, let's say i want to remove all the empty lines from a file and save the file inplace with the lines removed.

i should be able to use the following script

#!/usr/bin/env nu

def main [file: path] {
    open $file --raw
    | lines
    | find --invert --regex '^\s*$'
    | to text
    | save --force $file
}

however the content of the file gets erased completely...

Warning
let's call the script above remove-empty-lines-inplace, make it executable and put it in our PATH

Note
this issue does not happen at all if you replace to text with str join "\n"

How to reproduce

1. save the following in your PATH as remove-empty-lines-inplace

#!/usr/bin/env nu

def main [file: path] {
    open $file --raw
    | lines
    | find --invert --regex '^\s*$'
    | to text
    | save --force $file
}

3. make the script executable
4. reload Nushell
5. let's take an example file.txt

foo
bar

baz

6. run replace-empty-lines-inplace file.txt
7. see that file.txt is now empty

Expected behavior

i expected the content of foo.txt to be

foo
bar
baz

after running remove-empty-lines-inplace

Screenshots

No response

Configuration

key	value
version	0.81.1
branch	main
commit_hash	a5dd93d
build_os	linux-x86_64
build_target	x86_64-unknown-linux-gnu
rust_version	rustc 1.68.2 (9eb3afe9e 2023-03-27)
rust_channel	1.68.2-x86_64-unknown-linux-gnu
cargo_version	cargo 1.68.2 (6feb7c9cf 2023-03-26)
build_time	2023-06-08 20:55:16 +02:00
build_rust_channel	release
allocator	standard
features	default, sqlite, trash, which, zip
installed_plugins

Additional context

No response

[WindSoilder]

Actually I don't think we have a way to fix it.

nushell handles opened file as an external stream, and lines, find, to text also return stream as well(it's consumed lazily).

So your pipeline is basically like this: open file.txt --raw | save -f file.txt, or something like the command in bash or fish shell: cat file.txt > file.txt.

So the result will be empty.

To work aound this, you have to either:

1. save the reuslt into a variable first, then invoke save take the variable as pipeline input

let x = (open file.txt --raw | lines | find --invert --regex '^\s*$' | to text)
$x | save -f file.txt

2. use complete to consume the external raw string first

open file.txt --raw | lines | find --invert --regex '^\s*$' | to text | complete | get stdout | save -f file.txt

[amtoine]

@WindSoilder
interesting 🤔

yeah i've been able to fix my issue with

#!/usr/bin/env nu

def main [file: path] {
    let content = (open $file --raw)

    $content
    | lines
    | find --invert --regex '^\s*$'
    | to text
    | save --force $file
}

Note
note the break with the content variable

[amtoine]

not sure how to qualify this issue then, as wontfix?

[WindSoilder]

Yeah, I'd vote for wontfix

[amtoine]

@WindSoilder
i think i've narrowed the issue even more and it's not with open and save 😏

in the main above, change to text to str join "\n" and bam it works 😋
could the issue come from to text? 🤔

i think it's worth reopening

[fdncred]

i wrote to text quickly so it wouldn't surprise me to see bugs in it. What is the fix that would need to happen to to text?

[amtoine]

@fdncred
maybe it has to do with this FIXME!

[fdncred]

@fdncred maybe it has to do with this FIXME!

maybe? i stole that from somewhere else probably.

I think the real question isn't is this a fix or won't fix. it's do we think this is working the way we want it to. if it's not, then we should fix it. if it is working the way we want it to, because the work-around is the idiomatic solution, then we shouldn't fix it.

[amtoine]

@fdncred
agree with you 👍

imo the fact that

#!/usr/bin/env nu

def main [file: path] {
    open $file --raw
    | lines
    | find --invert --regex '^\s*$'
    | to text
    | save --force $file
}

does not work but

#!/usr/bin/env nu

def main [file: path] {
    open $file --raw
    | lines
    | find --invert --regex '^\s*$'
    | str join "\n"
    | save --force $file
}

does is confusing 🤔