Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

"Working with XML" documentation example does not work for version 4.17.2 #1095

Closed
app opened this issue Feb 4, 2022 · 18 comments
Closed

"Working with XML" documentation example does not work for version 4.17.2 #1095

app opened this issue Feb 4, 2022 · 18 comments
Labels
bug v4

Comments

@app
Copy link

app commented Feb 4, 2022

Describe the bug
Trying "Parse xml: simple" from top of this page
https://mikefarah.gitbook.io/yq/usage/xml

Version of yq: 4.17.2
$ yq --version
yq (https://github.com/mikefarah/yq/) version 4.17.2

Operating system: Ubuntu 20.04.3 LTS
Installed via: snap
$ snap list |grep yq
yq 4.17.2 1557 latest/stable mikefarah

Input XML
test.xml:

<?xml version="1.0" encoding="UTF-8"?>
<cat>
  <says>meow</says>
  <legs>4</legs>
  <cute>true</cute>
</cat>

Command
The command you ran:

yq -p=xml "." test.xml

Actual behavior

Error: unknown command "." for "yq"
Run 'yq --help' for usage.

Expected behavior

cat:
  says: meow
  legs: "4"
  cute: "true"

Additional context
no
@mikefarah
Copy link
Owner

Yep sorry, in versions prior to 4.18 you still need to use the 'e/eval' command - you can either update to the latest one, or

yq -p=xml e '.' file.xml

@mikefarah
Copy link
Owner

FWIW There was a notification on the readme - but that's easy to miss, I've now added a notification banner to the individual docs.

@app
Copy link
Author

app commented Feb 6, 2022

:) Thanks for quick reply!

@app
Copy link
Author

app commented Feb 6, 2022

I'm trying upgraded to version 4.18.1 yq with following results

$ yq --version
yq (https://github.com/mikefarah/yq/) version 4.18.1

Whithout e

$ yq -p=xml '.' sample.xml
Error: open sample.xml: no such file or directory

With e

$ yq  e -p=xml '.' sample.xml
Error: open sample.xml: no such file or directory

File permissions

$ ls -l sample.xml
-rw-rw-r-- 1 app app 109 фев  6 14:42 sample.xml

With stdin

$ cat sample.xml | yq  -p=xml '.'
Error: open .: permission denied

And finally it works :)

$ cat sample.xml | yq  -p=xml
cat:
  says: meow
  legs: "4"
  cute: "true"

Yes, yq installed from snap. As I described when open the issue.

@mikefarah mikefarah reopened this Feb 6, 2022
@mikefarah
Copy link
Owner

Hmm - that's not right :/

@mikefarah
Copy link
Owner

I couldn't replicate the permissions errors with 'samle.xml' - (even when I installed snap) it must be a snap confinement thing. If you're able to download the binary from the github releases - I'd be interested to see if those command then work.

The stdin example cat sample.xml | yq -p=xml '.' is a bug and I was able to replicate that. Will have a fix in the next release.

@app
Copy link
Author

app commented Feb 7, 2022

You are right. This is snap related issue.

$ ./yq_linux_amd64 --version
yq (https://github.com/mikefarah/yq/) version 4.18.1
$ ./yq_linux_amd64 -p=xml '.' sample.xml
cat:
  says: meow
  legs: "4"
  cute: "true"

@mikefarah
Copy link
Owner

Fixed STD bug in 4.20.1

@app
Copy link
Author

app commented Apr 1, 2022 •
edited

@mikefarah

XML does not work once again :(

$ yq --version
yq (https://github.com/mikefarah/yq/) version 4.23.1

$ cat sample.xml 
<?xml version="1.0" encoding="UTF-8"?>
<cat>
  <says>meow</says>
  <legs>4</legs>
  <cute>true</cute>
</cat>

cat sample.xml|yq -p=xml '.'
empty line in output

@mikefarah
Copy link
Owner

Weird - it works for me...

cat examples/mike.xml| yq -p=xml '.'
cat:
  says: meow
  legs: "4"
  cute: "true"
 ~/dev/yq  ➦ 03f57b7 ±  yq --version          
yq (https://github.com/mikefarah/yq/) version 4.23.1

@app
Copy link
Author

app commented Apr 1, 2022 

$ cat sample.xml | yq -v -p=xml '.'
12:02:39 evaluateSequence [DEBU] pipingStdIn: true
12:02:39 evaluateSequence [DEBU] stat.Mode(): prw-------
12:02:39 evaluateSequence [DEBU] ModeDir: ----------
12:02:39 evaluateSequence [DEBU] ModeAppend: ----------
12:02:39 evaluateSequence [DEBU] ModeExclusive: ----------
12:02:39 evaluateSequence [DEBU] ModeTemporary: ----------
12:02:39 evaluateSequence [DEBU] ModeSymlink: ----------
12:02:39 evaluateSequence [DEBU] ModeDevice: ----------
12:02:39 evaluateSequence [DEBU] ModeNamedPipe: p---------
12:02:39 evaluateSequence [DEBU] ModeSocket: ----------
12:02:39 evaluateSequence [DEBU] ModeSetuid: ----------
12:02:39 evaluateSequence [DEBU] ModeSetgid: ----------
12:02:39 evaluateSequence [DEBU] ModeCharDevice: ----------
12:02:39 evaluateSequence [DEBU] ModeSticky: ----------
12:02:39 evaluateSequence [DEBU] ModeIrregular: ----------
12:02:39 evaluateSequence [DEBU] ModeType: p---------
12:02:39 evaluateSequence [DEBU] ModePerm: -rw-------
12:02:39 maybeFile [DEBU] checking '.' is a file
12:02:39 maybeFile [DEBU] error: <nil>, dir: true
12:02:39 maybeFile [DEBU] result: false
12:02:39 processStdInArgs [DEBU] missing '-', adding it to the end
12:02:39 processArgs [DEBU] processed args: [. -]
12:02:39 maybeFile [DEBU] checking '.' is a file
12:02:39 maybeFile [DEBU] error: <nil>, dir: true
12:02:39 maybeFile [DEBU] result: false
12:02:39 processArgs [DEBU] assuming expression is '.'
12:02:39 ParseExpression [DEBU] Parsing expression: [.]
12:02:39 Tokenise [DEBU] Tokenising SELF (55)
12:02:39 handleToken [DEBU] processing SELF (55)
12:02:39 handleToken [DEBU]   adding token to the fixed list
12:02:39 ConvertToPostfix [DEBU] postfix processing currentToken SELF (55)
12:02:39 ConvertToPostfix [DEBU] put SELF (55) onto the opstack
12:02:39 ConvertToPostfix [DEBU] postfix processing currentToken )
12:02:39 popOpToResult [DEBU] popped SELF (55) from opstack to results
12:02:39 ConvertToPostfix [DEBU] opstackLen: 0
12:02:39 ConvertToPostfix [DEBU] PostFix Result:
12:02:39 ConvertToPostfix [DEBU] > SELF
12:02:39 createExpressionTree [DEBU] pathTree SELF 
12:02:39 decodeXML [DEBU] start element cat
12:02:39 decodeXML [DEBU] start element says
12:02:39 decodeXML [DEBU] chardata [meow] for says
12:02:39 decodeXML [DEBU] end element says
12:02:39 AddChild [DEBU] looking for says
12:02:39 AddChild [DEBU] not there, making a new one says
12:02:39 decodeXML [DEBU] start element legs
12:02:39 decodeXML [DEBU] chardata [4] for legs
12:02:39 decodeXML [DEBU] end element legs
12:02:39 AddChild [DEBU] looking for legs
12:02:39 AddChild [DEBU] not there, making a new one legs
12:02:39 decodeXML [DEBU] start element cute
12:02:39 decodeXML [DEBU] chardata [true] for cute
12:02:39 decodeXML [DEBU] end element cute
12:02:39 AddChild [DEBU] looking for cute
12:02:39 AddChild [DEBU] not there, making a new one cute
12:02:39 decodeXML [DEBU] end element cat
12:02:39 AddChild [DEBU] looking for cat
12:02:39 AddChild [DEBU] not there, making a new one cat
12:02:39 ParseExpression [DEBU] Parsing expression: [.]
12:02:39 Tokenise [DEBU] Tokenising SELF (55)
12:02:39 handleToken [DEBU] processing SELF (55)
12:02:39 handleToken [DEBU]   adding token to the fixed list
12:02:39 ConvertToPostfix [DEBU] postfix processing currentToken SELF (55)
12:02:39 ConvertToPostfix [DEBU] put SELF (55) onto the opstack
12:02:39 ConvertToPostfix [DEBU] postfix processing currentToken )
12:02:39 popOpToResult [DEBU] popped SELF (55) from opstack to results
12:02:39 ConvertToPostfix [DEBU] opstackLen: 0
12:02:39 ConvertToPostfix [DEBU] PostFix Result:
12:02:39 ConvertToPostfix [DEBU] > SELF
12:02:39 createExpressionTree [DEBU] pathTree SELF 
12:02:39 GetMatchingNodes [DEBU] Processing Op: SELF
12:02:39 GetMatchingNodes [DEBU] D0, P[], (doc)::
12:02:39 GetMatchingNodes [DEBU] >>
12:02:39 PrintResults [DEBU] PrintResults for 1 matches
12:02:39 PrintResults [DEBU] -- print sep logic: p.firstTimePrinting: false, previousDocIndex: 0, mappedDoc.Document: 0

12:02:39 PrintResults [DEBU] done printing results

@mikefarah
Copy link
Owner

Something weird is going on there after the last 'AddChild' - for some reasons it starts to parse the expression again 🤔

12:02:39 AddChild [DEBU] not there, making a new one cat
12:02:39 ParseExpression [DEBU] Parsing expression: [.]

I can't re-create that locally, so it's hard to diagnose.

Can you try with docker:

cat examples/mike.xml | docker run -i mikefarah/yq:4.23.1 -p=xml

Also, can you try from the binaries in github releases? I don't entirely trust snap and it's containment magic..

@mikefarah mikefarah reopened this Apr 1, 2022
@mikefarah
Copy link
Owner

Oh - and can you try with a yaml file too? wonder if this is specifically an XML thing or not.

@app
Copy link
Author

app commented Apr 2, 2022 •
edited 

$ cat sample.xml | docker run -i mikefarah/yq:4.23.1 -p=xml
Unable to find image 'mikefarah/yq:4.23.1' locally
4.23.1: Pulling from mikefarah/yq
3d2430473443: Pull complete 
b53b21a020ea: Pull complete 
fba71212744b: Pull complete 
150e3e8e6950: Pull complete 
74f05bae6540: Pull complete 
Digest: sha256:26bb2fb3f35e6522c43260b85d67e31043764d53a761ffaf957f8819e52b05ea
Status: Downloaded newer image for mikefarah/yq:4.23.1
cat:
  says: meow
  legs: "4"
  cute: "true"

@app
Copy link
Author

app commented Apr 2, 2022 •
edited 

$ cat sample.yml 
a:
  cool:
    foo: bar
    +id: hi

d: <foo>bar</foo>

$ cat sample.yml | yq '.b = (.d | from_xml)' 
a:
  cool:
    foo: bar
    +id: hi
d: <foo>bar</foo>
b:
  foo: bar

$ cat sample.yml | yq '.a | @xml'
<cool id="hi"><foo>bar</foo></cool>

Update. Version of yq is unknown here :)
May be it is 4.24.2

@app
Copy link
Author

app commented Apr 2, 2022 

$ ./yq_linux_amd64 --version
yq (https://github.com/mikefarah/yq/) version 4.23.1

$ ./yq_linux_amd64 -p=xml '.' sample.xml 
cat:
  says: meow
  legs: "4"
  cute: "true"

$ cat sample.xml|./yq_linux_amd64 -p=xml '.'
cat:
  says: meow
  legs: "4"
  cute: "true"

@app
Copy link
Author

app commented Apr 2, 2022

And I see no problems now in 4.24.2 (snap) version

$ yq --version
yq (https://github.com/mikefarah/yq/) version 4.24.2
$ cat sample.xml|yq -p=xml '.' 
cat:
  says: meow
  legs: "4"
  cute: "true"

Thank you!

P.S.
However, the old snap problem is here. But it is not yq problem.

$ yq -p=xml '.' sample.xml 
Error: open sample.xml: no such file or directory

@mikefarah
Copy link
Owner

Righto! not sure what was going on before ... glad it's working now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug v4
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@app @mikefarah