jq doesn't parse colorized JSON

[nexxai]

Describe the bug
When piping output from azure-cli (az) to jq it fails with a parse error. If I manually select the text and pipe it into jq by way of the 'echo' command, it works fine.

To Reproduce
az account list --query '[].id' | jq '.[]'

Expected behavior
// Doesn't work:

sh$ az account list --query '[].id' | jq '.[]'
"a86bc126-XXXX-XXXX-XXXX-7e546be7a509"                   // Expected
parse error: Invalid numeric literal at line 2, column 4 // Actual
sh$

// Works:

sh$ az account list --query '[].id'
[
  "a86bc126-XXXX-XXXX-XXXX-7e546be7a509"
]
sh$ echo '["a86bc126-XXXX-XXXX-XXXX-7e546be7a509"]' | jq '.[]'
"a86bc126-XXXX-XXXX-XXXX-7e546be7a509"
sh$

Environment (please complete the following information):

• OS and Version: macOS Mojave 10.14.2 (18C54)
• jq version 1.6

Additional context
n/a

[wtlangford]

Interesting. Can you pipe the output of az to a file and then upload it as an attachment to the issue? I wonder if there aren't some weird ASCII control characters in there for some reason. Feel free to scrub the ID out, but don't just paste it into github. It might try to sanitize the text (for example, copying your script's output from the issue works fine for me).

…

On Wed, Jan 9, 2019, 14:50 Justin Smith ***@***.***> wrote: *Describe the bug* When piping output from azure-cli (az) to jq it fails with a parse error. If I manually select the text and pipe it into jq by way of the 'echo' command, it works fine. *To Reproduce* az account list --query '[].id' | jq '.[]' *Expected behavior* // Doesn't work: sh$ az account list --query '[].id' | jq '.[]' "a86bc126-XXXX-XXXX-XXXX-7e546be7a509" // Expected parse error: Invalid numeric literal at line 2, column 4 // Actual sh$ // Works: sh$ az account list --query '[].id' [ "a86bc126-XXXX-XXXX-XXXX-7e546be7a509" ] sh$ echo '["a86bc126-XXXX-XXXX-XXXX-7e546be7a509"]' | jq '.[]' "a86bc126-XXXX-XXXX-XXXX-7e546be7a509" sh$ *Environment (please complete the following information):* - OS and Version: macOS Mojave 10.14.2 (18C54) - jq version 1.6 *Additional context* n/a — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1787>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADQ4V-TY8RNc56aPRSW3FEx1FXCMnk5Vks5vBjcKgaJpZM4Z4BeS> .

[nexxai]

I think you're on to something. I use a colored terminal (oh_my_zsh) and when I piped the output of az account to a file, it also included the ANSI(?) color codes like this:

[
  �[33m"a86bc126-XXXX-XXXX-XXXX-7e546be7a509"�[39;49;00m
]

I'm not sure how big an ask this is, but would it be possible to add filtering in jq to look for (and discard) those codes?

[wtlangford]

I'd rather not teach our parser to ignore those for a variety of reasons.

That said, it looks like az has an output mode that doesn't include the color codes, though. Try az -o json account list --query '[].id' | jq '.[]' and see if that works?

[nicowilliams]

Maybe the az command should not colorize output when it goes to a pipe... Maybe az should learn from jq...

[nicowilliams]

Good debugging. It actually might not be crazy to teach jq about color... But I too would rather not.