-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Edit files in place #105
Comments
+1 |
1 similar comment
+1 |
Scripting around this is not an option? In shell:
|
I'm asking why it's not acceptable. |
By analogy: Would it be acceptable if <insert your favorite word processor here> made you save to a different file every time you wanted to save your document? It's not that the problem can't be worked around, it's that doing so is cumbersome. The fact that something as fundamentally stream oriented as |
I see. I've generally not bothered with sed's -i option. For me the On Mon, Jun 9, 2014 at 10:23 AM, ipmcc notifications@github.com wrote:
|
Anyways, I can add this easily. My only concern is to limit the number of
which would run the given command with On Mon, Jun 9, 2014 at 10:58 AM, Nico Williams nico@cryptonector.com
|
Ah, there's a sponge(1) that almost works this way: http://joeyh.name/code/moreutils/ I think both, sponge(1) and my idea of an inplace(1) would be very useful Would that work for you? |
Regardless of how you feel about this issue, what think ye of this: https://github.com/nicowilliams/inplace ? |
Is the |
@drorata - It was (as best I can tell) never available in any official jq release, and is certainly not available in jq 1.4 or 1.5. It was added as an experimental feature on Jul 20, 2014, and removed on Mar 5, 2015. |
@svnpenn - Although there are obviously circumstances when -i would be convenient, jq is more like awk than sed. The earlier attempt to endow jq with -i confirms that such attempts will be fraught with difficulties. awks do not have an -i option, and I think it is asking too much of the jq developers/maintainers to devise and implement such an option, especially given there are easy workarounds. |
Some of the refactoring to move I/O from main.c was done with in-place
editing in mind, and is necessary to implement sed-style in-place editing.
But it's not a high priority for me at this time, meaning: send us a PR if
you want this done soon :)
|
@svnpenn - My apologies if I over-generalized, but the GNU awk documentation characterizes "inplace editing" as "an extension" (https://www.gnu.org/software/gawk/manual/html_node/Extension-Sample-Inplace.html). Also, your example does not work with the GNU awk installed on my machine (4.0.2), and the awk that comes with OS X 10.9.5 does not support a "-i" option at all. |
I agree that jq can and should do in-place. The earlier problems were due to a poor original implementation. |
The original implementation was poor and, in petticoat, had bad semantics.
I refactored the way I/O is done in main.c in order to make it easier to
later add sed-like `-i` -- it should be easy enough now. Send a PR or be
patient :)
|
This was an off-by one: the NUL terminator byte was not allocated on resize. This was triggered by JSON-encoded numbers longer than 256 bytes.
Is this feature in the next release? I found it in 'release candidate 1', or is that the version that was retracted? |
@danielbayley No, it got taken out. 1.5rc1 implemented the wrong |
There's no PR for it at the moment. Feel free to send one :)
|
+1 |
any action on this one? |
@omgdenzuko so GNU sed -i flag to you is a bad practice? |
I am honestly amazed. 😲 |
It's the modern authoritarian way, justifying itself with the days when
there was no choice.
"Must stop people using features that I don't know how to use"
And so missing features must be replaced by idiomatic expressions
everywhere they are needed, and are even harder to get right.
Those who don't know how to use -i don't need to use it, but they seemed so
determined to prevent anybody else succeeding with it.
Do they really look at people using sed -i and not know what is going on,
or think that they would be just as easily able to parse an idiomatic loop
and prove that it did the same thing? Surely these are the kind of people
who write interview questions.
1. Prove that this expression is functionally the same as sed -i on the
list of files. (Time allowed 10 minutes)
2. Explain why it is superior to sed -i
(Time allowed 5 minutes)
3. Explain how it is more obvious that the expression replaces the files in
place than it is that -i replaces the files in place (Time allowed 45
minutes)
Bonus points of the interviewee can come up with their own expression which
only fails on unlikely corner cases, or "rare" race conditions
…On Mon, 9 Aug 2021, 22:27 Abhiroj Panwar, ***@***.***> wrote:
I am honestly amazed. 😲
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#105 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABL7PTCLWIFX6ORL2KSJXQTT4BB4PANCNFSM4AEYTROA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email>
.
|
I would like this feature 😄 |
Although I use below, still want '-i' or '-w' jq 'del(.packageManager)' package.json | sponge package.json |
It can be done with:
|
Don't even need the echo.
…On Wed, Oct 20, 2021, 18:02 Iaroslav Pleten ***@***.***> wrote:
It can be done with:
echo "$(jq '. + {"registry-mirrors": ["https://docker-mirror"]}' /etc/docker/daemon.json)" > /etc/docker/daemon.json
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#105 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AUILSHGDJM7AUS5KQZ34FTTUH5DAJANCNFSM4AEYTROA>
.
|
@omgdenzuko Yes, you need, unless you are not using bash/zsh/... |
Ok.. Since when did any posix shell not support proper stdout redirecting
in the past 40-ought years?
No. Echoing a subshell's output as a string is, I repeat on K&R's graves,
not needed. Its like piping into cat from cat.
jq '<expression>' > output.json works on all posix systems. Even then
piping into tee output.json is acceptable. I should know its something I do
a fing lot for work. But hey.. I'd love to see the asciimina of where I'm
wrong.
…On Wed, Oct 20, 2021, 18:54 loynoir ***@***.***> wrote:
Don't even need the echo.
… <#m_-1820777542152595716_>
On Wed, Oct 20, 2021, 18:02 Iaroslav Pleten *@*.***> wrote: It can be
done with: echo "$(jq '. + {"registry-mirrors": ["https://docker-mirror"]}'
/etc/docker/daemon.json)" > /etc/docker/daemon.json — You are receiving
this because you were mentioned. Reply to this email directly, view it on
GitHub <#105 (comment)
<#105 (comment)>>, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AUILSHGDJM7AUS5KQZ34FTTUH5DAJANCNFSM4AEYTROA
.
@omgdenzuko <https://github.com/omgdenzuko> Yes, you need, unless you are
not using bash/zsh/...
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#105 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AUILSHENEPYRKJFKS5YGTK3UH5JDNANCNFSM4AEYTROA>
.
|
@omgdenzuko
$ echo '{"a":"b"}' > daemon.json && jq '. + {"registry-mirrors": ["https://docker-mirror"]}' daemon.json > daemon.json && cat daemon.json
$ echo '{"a":"b"}' > daemon.json && echo "$(jq '. + {"registry-mirrors": ["https://docker-mirror"]}' daemon.json)" > daemon.json && cat daemon.json
{
"a": "b",
"registry-mirrors": [
"https://docker-mirror"
]
} |
@omgdenzuko
But not agree with your
Because |
Yeah that's the usual *sh not holding a file buffer after a return from
subshell. It's exactly why tee(1) exists.
Just pipe stdout to tee or read so one gets stdout as a memory block to
flush to disk. That's what your echo is doing while adding complexity, it's
capturing stdout from the subshell then parsing it as a string, looking for
shell expansions to evaluate then dumping it to its stdout where the shell
then redirects it to a file.
jq &1 pipe tee file eliminates the evaluation and just writes to the file.
…On Wed, Oct 20, 2021, 22:33 loynoir ***@***.***> wrote:
@omgdenzuko <https://github.com/omgdenzuko>
Don't even need the echo.
$ echo '{"a":"b"}' > daemon.json && jq '. + {"registry-mirrors": ["https://docker-mirror"]}' daemon.json > daemon.json && cat daemon.json
$ echo '{"a":"b"}' > daemon.json && echo "$(jq '. + {"registry-mirrors": ["https://docker-mirror"]}' daemon.json)" > daemon.json && cat daemon.json
{
"a": "b",
"registry-mirrors": [
"https://docker-mirror"
]
}
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#105 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AUILSHHMYVGEGUUUQVFNLD3UH6CYLANCNFSM4AEYTROA>
.
|
@omgdenzuko
On ipleten's approach
To
But seems you actaully means
I prefer https://stackoverflow.com/questions/33638511/differences-between-sponge-and-tee |
Using > or tee is at best a bad idea unless your shell or version of tee specifically documents that it can be used in the desired way. For more on this, see http://askubuntu.com/questions/752174 Using |
Hey! |
@ipleten in any debian based distro sponge is apart of the moreutils and its not part of POSIX or heirloom *nix so that's why its not include by default. A list of supported package managers to install sponge and it copy-pasta command can be found at https://command-not-found.com/sponge |
This repeated discussion, the usual mistakes, the file races, repeated all
over the world (not only here) is why in-place option is needed and why it
is futile to argue about how the unix philosophy renders it not strictly
necessary.
It is needed, because too often people can't get the workarounds right.
The sheer number of hours wasted must be incredible and only get worse as
jq becomes more popular in spite of this obvious deficiency.
Sam
…On Thu, 21 Oct 2021, 20:42 Dwight Spencer, ***@***.***> wrote:
@ipleten <https://github.com/ipleten> in any debian based distro sponge
is apart of the moreutils and its not part of POSIX or heirloom *nix so
that's why its not include by default.
https://command-not-found.com/sponge
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#105 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABL7PTBNP4CXZ3O26VAQK7DUIBUJVANCNFSM4AEYTROA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
yq does support inplace editing and can produce normal JSON. CLI API is different however.
|
I use yq -iP '.Email.Port=3030' config.json -o json
|
After nearly 10 years of jq maintainers arguing against in-place editing on the grounds that there are difficult and obscure ways around that delightful limitation, we now also have jaq, written in rust, with in-place editing of multiple files, and fixing various jq bugs: |
I hope this PR can once and for all end this debate. It's a really simple change that follows exactly the solution that was explained in the Kernighan and Pink document as mentioned earlier, plus it doesn't impose any big changes on how the internals work. Re yq and jaq: https://xkcd.com/927 comes to mind :P |
While we are waiting ... Minify all find . \
-type f \
-name '*.json' \
-exec sh -c 'mv "$1" "$1.tmp"; jq -c . "$1.tmp" > "$1"; rm "$1.tmp"' shell {} \; |
@sdavids what does the |
https://unix.stackexchange.com/questions/389705/understanding-the-exec-option-of-find#answer-389706 I used |
This comment was marked as outdated.
This comment was marked as outdated.
github community is getting weird... |
With GNU Sed for example you can edit files in place:
However I see no option for this with JQ, currently I must do this:
Here is a workaround, Awk slurp:
The text was updated successfully, but these errors were encountered: