Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PowerShell: Wrap example does not work #747

Open
chrischu opened this issue Mar 12, 2021 · 11 comments
Open

PowerShell: Wrap example does not work #747

chrischu opened this issue Mar 12, 2021 · 11 comments
Labels

Comments

@chrischu
Copy link

Describe the bug
I tried running the sample from https://mikefarah.gitbook.io/yq/v/v4.x/operators/create-collect-into-object#wrap-prefix-existing-object in PowerShell but I can't seem to get it to work.

Version of yq: 4.6.1
Operating system: windows
Installed via: binary release

Input Yaml
Concise yaml document(s) (as simple as possible to show the bug, please keep it to 10 lines or less)
sample.yml:

name: Mike

Command
The command you ran:

yq eval '{""wrap"": .}' sample.yml

Actual behavior

Error: Parsing expression: Lexer error: could not match text starting at 1:1 failing at 1:2.
        unmatched text: "`"

Expected behavior

wrap:
  name: Mike

Additional context
I tried both the original code snippet with single ", but then found out that I apparently have to double the quotes in PowerShell, but that didn't not help either :(.

@mikefarah
Copy link
Owner

@chrischu
Copy link
Author

I tried both the original code snippet with single ", but then found out that I apparently have to double the quotes in PowerShell, but that didn't not help either :(.

As I said: yes.

@mikefarah
Copy link
Owner

Oh right sorry - no idea then, I got that advice from here https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_quoting_rules?view=powershell-7.1 for what it's worth - perhaps there's doc for whatever version of powershell you are using..

@phreed
Copy link

phreed commented May 27, 2021

The work around is to quote in the following way.

.\yq.exe e '{\"wrap\": .}' sample.yaml

You can see that this is strictly a Powershell issue by the following example.
There is a tool to help with this echoargs

 echoargs e '{\"wrap\": .}' sample.yaml
Arg 0 is <e>
Arg 1 is <{"wrap": .}>
Arg 2 is <.\examples\name.yaml>

Command line:
"C:\ProgramData\chocolatey\lib\echoargs\tools\EchoArgs.exe" e "{\"wrap\": .}" .\examples\name.yaml

echoargs is available as a chocolatey package choco install echoargs.

@phreed
Copy link

phreed commented May 28, 2021

Here is a yq.ps1 that seems to do the necessary escaping.

Param(
    [string]$command,
    [string]$yamlFile
  )
$escCommand =  $command -replace '"','\"' -replace ' ',''
$evalArgs = @("eval", $escCommand, $yamlFile)

Write-Output ".\yq.exe"  $evalArgs
Start-Process ".\yq.exe"  -ArgumentList $evalArgs -Wait -NoNewWindow -PassThru

I am not sure about the removing whitespace bit. But, without it Start-Process splits the evalArgs into 4 arguments instead of 3.
I would like to see evaluate_sequence_command.go have a log message in the evaluateSequence() function showing the individual arguments.

@phreed
Copy link

phreed commented May 28, 2021

Here is a yq.ps1 that seems to act better.

Param(
    [string]$command,
    [string]$yamlFile
  )
$escCommand =  $command -replace '"','\"' 
$evalArgs = @("eval", "$escCommand", $yamlFile)

Write-Output ".\yq.exe"  $evalArgs
& ".\yq.exe"  $evalArgs

@ferdnyc
Copy link

ferdnyc commented Jun 2, 2022

@phreed

The work around is to quote in the following way.

.\yq.exe e '{\"wrap\": .}' sample.yaml

You can see that this is strictly a Powershell issue by the following example. There is a tool to help with this echoargs

 echoargs e '{\"wrap\": .}' sample.yaml
Arg 0 is <e>
Arg 1 is <{"wrap": .}>
Arg 2 is <.\examples\name.yaml>

Command line:
"C:\ProgramData\chocolatey\lib\echoargs\tools\EchoArgs.exe" e "{\"wrap\": .}" .\examples\name.yaml

echoargs is available as a chocolatey package choco install echoargs.

Thanks for the pointer on echoargs, that's great to have.

Here's the one thing that confuses me, though: At least in PowerShell 6 (7 doesn't work under remote sessions), the "Arg..." output of echoargs is identical for these two cases, yet only one of them works. And in fact, the message yq spits out in the failure case has changed in subtle but interesting ways from @chrischu's OP:

PS C:\Users\***> yq -M '{""wrap"": .}' sample.yml
Error: parsing expression: Lexer error: could not match text starting at 1:2 failing at 1:7.
        unmatched text: "\"wrap:"

PS C:\Users\***> echoargs e '{""wrap"": .}' sample.yml
Arg 0 is <e>
Arg 1 is <{"wrap": .}>
Arg 2 is <sample.yml>

Command line:
"C:\ProgramData\chocolatey\lib\echoargs\tools\EchoArgs.exe" e "{""wrap"": .}" sample.yml

PS C:\Users\***> yq -M e '{\"wrap\": .}' sample.yml
wrap:
  name: Mike
PS C:\Users\***> echoargs e '{\"wrap\": .}' sample.yml
Arg 0 is <e>
Arg 1 is <{"wrap": .}>
Arg 2 is <sample.yml>

Command line:
"C:\ProgramData\chocolatey\lib\echoargs\tools\EchoArgs.exe" e "{\"wrap\": .}" sample.yml

In fact, another working form — as I just discovered — is this:

PS C:\Users\***> yq -M e "{\""wrap\"": .}" sample.yml
wrap:
  name: Mike

It doesn't work without the backslashes, and interestingly, echoargs also requires the backslashes to show the argument correctly, which is making me a bit suspicious of it now:

PS C:\Users\***> echoargs e "{""wrap"": .}" sample.yml
Arg 0 is <e>
Arg 1 is <{wrap: .}>
Arg 2 is <sample.yml>

Command line:
"C:\ProgramData\chocolatey\lib\echoargs\tools\EchoArgs.exe" e "{"wrap": .}" sample.yml

PS C:\Users\***> echoargs e "{\""wrap\"": .}" sample.yml
Arg 0 is <e>
Arg 1 is <{"wrap": .}>
Arg 2 is <sample.yml>

Command line:
"C:\ProgramData\chocolatey\lib\echoargs\tools\EchoArgs.exe" e "{\"wrap\": .}" sample.yml

(Notice how, without the backslashes, echoargs drops one set of quotes from its "Command line:" string, then the other set of quotes for its "Arg 1" output.)

@bschapendonk
Copy link

bschapendonk commented Jan 30, 2024

Powershell 7.4 breaks the behavior of "", a "seems to work just fine now
image

@chrischu
Copy link
Author

chrischu commented Jan 31, 2024

If you want the old behavior you can use $PSNativeCommandArgumentPassing = "Legacy". See Passing arguments that contain quote characters

@chrischu
Copy link
Author

But yq does some really weird stuff when using --%:

.\yq_windows_amd64.exe --% --verbose -n '.test = "something"'

prints

08:16:49 processArgs [DEBU] processed args: ['.test = "something"']
08:16:49 maybeFile [DEBU] checking ''.test' is a file
08:16:49 maybeFile [DEBU] error: CreateFile '.test: The system cannot find the file specified.
08:16:49 maybeFile [DEBU] result: false
08:16:49 processArgs [DEBU] assuming expression is ''.test'
Error: cannot pass files in when using null-input flag

So yq gets the correct parameters but for some reason thinks .test is a file.

@Aankhen
Copy link

Aankhen commented Jan 31, 2024

I believe yq is getting the literal string '.test as its first argument, not just .test, because the stop-parsing token (--%) prevents PowerShell from parsing the quoted string into a single argument. See the difference in the processed args here:

❯❯ echo '' | yq --% --verbose -n .
13:03:51 processArgs [DEBU] processed args: [.]
13:03:51 maybeFile [DEBU] checking '.' is a file
13:03:51 maybeFile [DEBU] error: <nil>, dir: true
13:03:51 maybeFile [DEBU] result: false
13:03:51 processArgs [DEBU] assuming expression is '.'
13:03:51 FormatFromFilename [DEBU] using default inputFormat 'yaml'
13:03:51 initCommand [DEBU] Using input format yaml
13:03:51 initCommand [DEBU] Using output format yaml
13:03:51 ParseExpression [DEBU] Parsing expression: [.]
13:03:51 handleToken [DEBU] processing SELF (55)
13:03:51 handleToken [DEBU]   adding token to the fixed list
13:03:51 ConvertToPostfix [DEBU] postfix processing currentToken SELF (55)
13:03:51 ConvertToPostfix [DEBU] put SELF (55) onto the opstack
13:03:51 ConvertToPostfix [DEBU] postfix processing currentToken )
13:03:51 popOpToResult [DEBU] popped SELF (55) from opstack to results
13:03:51 ConvertToPostfix [DEBU] opstackLen: 0
13:03:51 ConvertToPostfix [DEBU] PostFix Result:
13:03:51 ConvertToPostfix [DEBU] > SELF
13:03:51 createExpressionTree [DEBU] pathTree SELF
13:03:51 GetMatchingNodes [DEBU] Processing Op: SELF
13:03:51 GetMatchingNodes [DEBU] D0, P, ScalarNode (!!null)::0 kids
13:03:51 PrintResults [DEBU] PrintResults for 1 matches
13:03:51 PrintResults [DEBU] -- print sep logic: p.firstTimePrinting: false, previousDocIndex: 0
13:03:51 PrintResults [DEBU] D0, P, ScalarNode (!!null)::0 kids
13:03:51 Encode [DEBU] encoderYaml - going to print D0, P, ScalarNode (!!null)::0 kids

13:03:51 PrintResults [DEBU] done printing results

❯❯ echo '' | yq --% --verbose -n '.'
13:03:54 processArgs [DEBU] processed args: ['.']
13:03:54 maybeFile [DEBU] checking ''.'' is a file
13:03:54 maybeFile [DEBU] error: CreateFile '.': The system cannot find the file specified.
13:03:54 maybeFile [DEBU] result: false
13:03:54 processArgs [DEBU] assuming expression is ''.''
13:03:54 FormatFromFilename [DEBU] using default inputFormat 'yaml'
13:03:54 initCommand [DEBU] Using input format yaml
13:03:54 initCommand [DEBU] Using output format yaml
13:03:54 ParseExpression [DEBU] Parsing expression: ['.']
Error: 1:1: invalid input text "'.'"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

6 participants