Escape shell arguments correctly

[fritzmg]

• Bug fix #…?
• New feature?
• BC breaks?
• Tests added?
• Docs added?

This is my take on fixing #3478. The current usage of escapeshellarg is wrong, as it will executed on the local system and thus not escape the shell argument appropriately - since the argument is used for a command that is run on the remote system.

This PR basically creates an escape_shell_argument that can be made aware of the remote host's operating system. It is a copy of Symfony\Component\Process\Process::escapeArgument and allows to override the directory separator.

So this solution first resolves the directory separator on the remote system and then escapes the shell argument based on that detection.

[antonmedv]

Lets use quote function from my other project:

https://github.com/google/zx/blob/d1cd1aa02e3fab7ca4f048d9571b77b6ce25d559/src/util.ts#L31-L48

[fritzmg]

Lets use quote function from my other project:

https://github.com/google/zx/blob/d1cd1aa02e3fab7ca4f048d9571b77b6ce25d559/src/util.ts#L31-L48

But that code would create the following command, always:

echo '{"created_at":"2023-05-19T14:29:03+0000","release_name":"16","user":"foobar","target":"HEAD"}' >> .dep/releases_log

This would again produce a wrong releases_log if the target server is Windows based. Though I guess that use case is may be not supported anyway?

[fritzmg]

@antonmedv I have adjusted the PR now to the new escaping method, under the assumption that deployments onto a Windows server are not really supported (and not common place) anyway.

[antonmedv]

if the target server is Windows based

This is true. Only Linux.

[antonmedv]

Let's add some test for this function.

[fritzmg]

I added some basic tests.

[s4muel]

at a first glance it won't work with single quotes in string, will it? i mean, the escaped string will be OK (not making problems when running the shell command), but it won't produce valid JSON. taking Erik's name as an example

escape_shell_argument(json_encode(['name' => "Erik D'Ercole"]));

this will produce '{"name":"Erik D\'Ercole"}' which again is not JSON and will cause the same problem

oh, and to clarify, the TARGET OS is not an issue, the escapeshellarg problem is on the machine that DEPLOYS the code. just read the reasoning in the previous PR: #3569

one thing is to escape correctly when deploying from windows. but the other thing is to handle single quotes in the string, like the example with Erik's name - that is a problem when deploying from any OS.

[fritzmg]

at a first glance it won't work with single quotes in string, will it?

It will. Single quotes will be replaced by \'.

which again is not JSON

What makes you say that? Single quotes must be escaped by \ in JSON.

[s4muel]

well, AFAIK there is no escaping of single quotes in JSON. this is valid JSON:

{
  "name": "Erik D'Ercole"
}

this is invalid JSON:

{
  "name": "Erik D\'Ercole"
}

not a pro here, just used random JSON online validator in combination with https://www.json.org/json-en.html
i might be wrong

[fritzmg]

well, AFAIK there is no escaping of single quotes in JSON.

While it's not technically necessary, it's good practise to do so. PHP's json_encode does it as well. The info will be read by PHP anyway.

[s4muel]

function escape_shell_argument(string $argument): string
{
    if ('' === $argument || preg_match('~^[a-z0-9/_.\-@:=]+$~i', $argument)) {
        return $argument;
    }

    return "'".str_replace(
        ['\\', "'", "\f", "\n", "\r", "\t", "\v", "\0"],
        ['\\\\', "\\'", '\\f', '\\n', '\\r', '\\t', '\\v', '\\0'],
        $argument,
    )."'";
}

$json = escape_shell_argument(json_encode(['name' => "Erik D'Ercole"]));

echo $json, PHP_EOL;
var_dump(json_decode($json));

echo 'Last error: ', json_last_error_msg(), PHP_EOL, PHP_EOL;


$json2 = <<<EOI
{"name": "Erik D\'Ercole"}
EOI;

echo $json2, PHP_EOL;
var_dump(json_decode($json2));

echo 'Last error: ', json_last_error_msg(), PHP_EOL, PHP_EOL;

produces:

'{"name":"Erik D\'Ercole"}'
NULL
Last error: Syntax error

'{"name":"Erik D\'Ercole"}'
NULL
Last error: Syntax error

you sure? am i doing something wrong or missing something?

[fritzmg]

Sorry, you are right. And escaping with \' on the command line level is not sufficient.

Instead, the command needs to look like this:

echo '{"name":"Erik D'\''Ercole"}' >> .dep/releases_log

which will result in

{"name":"Erik D'Ercole"}

in the .dep/releases_log.

And this is exactly what Symfony\Component\Process\Process::escapeArgument does.

I have updated the PR accordingly and boiled it down to the strictly necessary pieces (assuming a Linux server target).

@s4muel can you please test these changes again?

[fritzmg]

@antonmedv this should be the correct solution. Any chance of getting this merged and released?

[underdpt]

This might seem a trivial bug, but it's having great impact on users deploying from Windows, because old releases are not deleted which might create issues (I've just hit another server that had no free inodes due to old releases not being cleaned).

Is there anything we can do to have this merged and released?

[ttk]

I've been hit by this bug. I've reviewed this PR and it looks good to me!