Can't pipe into yq

[andrewsav-bt]

Piping into yq with my.yml | yq does not seem to work, which limits usability for scripting on linux severely. This needs to be supported.

[mikefarah]

http://mikefarah.github.io/yq/read/#from-stdin

[spuder]

You have to do this yq r -. Really wish this was simpler.

echo -e '---
foo: bar' | yq r -

Also it doesn't work reliably with tools like helm. It only returns the first yaml document, even if there are more.

helm template foo

[mikefarah]

That's what it does by default, you can specify a specific document or all documents:

http://mikefarah.github.io/yq/read/#multiple-documents-specify-a-single-document

[spuder]

So looks like yq r - -d '*' is the solution to pipe to yq.

echo -e '---
foo:bar
---
buzz:42
' | yq r - -d '*'

[ovidiubuligan]

I hope the defaults will change to something smarter . To just do mycommand | yq just like jq.

[RSwarnkar]

Hi guys, I see It does not do the syntax highlighting when piped. Please help!

[lindhe]

Was this functionality removed? I can't get it to work...

EDIT: Yep! Seems like this changed with v4 of yq. Now we have to run yq eval for things to work from stdin. I find it incredibly annoying that eval is not the default behaviour when just running stuff | yq since that's how jq works...

[czerasz]

echo -e '---
foo: bar
buzz: 42
' | yq e '.foo' -

returns bar

In version:

$ yq --version
yq version 4.5.0

[kubosuke]

@RSwarnkar
I got hilighted yaml by:

cat tmp.yaml | yq -C r -

yq version 3.4.1

[tliron]

The use of - to specify stdin processing does not seem to be mentioned in the command help. I had to find this issue to learn about it.

[mikefarah]

Good idea - I'll put it into the command help (for v4)

[fabiopaiva]

if you just need something to print nice YML

alias yq="yq eval -P"

[mabushey]

Has anyone figured out how to do this with V4? yq e by itself pretty prints, but then assumes anything (ie .name) else is a filename. It's IMPOSSIBLE to find correct docs because there are so many incompatable versions and different releases.

[mikefarah]

Sorry to hear that, the documentation is referenced from this page, you can select the correct version on the RHS, the default is version 4.

yq's built in help docs also document how to do read from stdin:

yq e --help

For what it's worth, I've just added reading from STDIN as an example in the quick guide of the Github readme.

[sezu]

Has anyone figured out how to do this with V4? yq e by itself pretty prints, but then assumes anything (ie .name) else is a filename. It's IMPOSSIBLE to find correct docs because there are so many incompatable versions and different releases.

Try yq e '.' -

[mabushey]

Thanks @sezu . I keep coming back here via google trying to figure out how to do this repeatedly as this is so anti-intuitive.

[mikefarah]

Did some black magic, and with the latest release (4.18.1) you no longer have to specify e or -:

cat file.yml | yq

[petrmvala]

@mikefarah Can you explain why you've deleted my post?

[mikefarah]

It was a rude and entitled post, but i deleted soecifically because you said no one sane would use yq, which I take as a personal atttack against me and users of yq. I understand, and am sorry that you were frustrated, but I do not tolerate posts like that. See the the code of conduct in the repo for more information. Fixed: repo not report

…

________________________________ From: petrmvala ***@***.***> Sent: Sunday, January 30, 2022 11:43:18 PM To: mikefarah/yq ***@***.***> Cc: Mike Farah ***@***.***>; Mention ***@***.***> Subject: Re: [mikefarah/yq] Can't pipe into yq (#113) @mikefarah<https://github.com/mikefarah> Can you explain why you've deleted my post? — Reply to this email directly, view it on GitHub<#113 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAIZHNOAAUEE3WE72I4S2DLUYUW6NANCNFSM4EJX4FEQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[tliron]

@mikefarah Thank you for volunteering your time to work on this project. This latest fix is very welcome and makes it more pleasant to use for us pseudo-sane people. :)

For those unhappy few, I'm sure you are welcome to contribute patches, fork, or start your own alternative project.

[petrmvala]

There was nothing rude about the post I made. I barely mentioned that 99.9% of macos/linux tools work the same way and use the same verb structure.
This way, all the tools can do things as piping commands and using CLI a coherent experience.

You have made a tool to manipulate yaml. That is a good idea, and thank you for that. However, you have named the tool yq, which obviously reflects on a very popular tool jq. It is quite natural to expect the CLI tool to behave like most CLI tools do. Furthermore, it is quite natural to expect very similar behavior to jq.

Your tool, however, doesn't behave like jq and doesn't behave like all other CLI tools either. If this behavior were so sane, why is there this thread filled with several attempts to understand basic functionality?

The fact that this is probably a bad functionality decision is not a personal attack.

[jeffsani]

I was able to Anyone know which switch to use to remove the "-" from the output? FYI - I am on 4.16.1 and I still had to use the "e" eval directive to get yq working with piped curl output.

[btpka3]

Found when using yq v4.40.4 with pipe / stdin, flag -p is needed, otherwise there's no output
( input format is guessed by file name suffix but there no filename for stdin?) .

# check yq version
yq -V

# prepare test data   
xml='
<project>
  <profiles>
    <profile><id>aaa</id></profile>
    <profile><id>bbb</id></profile>
    <profile><id>ccc</id></profile>
  </profiles>
</project>
'
echo $xml > /tmp/a.xml

# test 1 :  worked
echo $xml | yq eval -p xml -o json '.project.profiles.profile[].id'
# test 2 :  worked
yq eval -p xml -o json '.project.profiles.profile[].id' < /tmp/a.xml

[dbsanfte]

I feel like if no file is specified, yq should be like jq and default to reading from stdin.