[9.x] Database queries containing JSON paths support array index references

[derekmd]

Fixes: #26415

When running queries that update JSON paths including an array index, Laravel compiles an SQL statement that escapes the array index as part of a JSON object key.

e.g.,

Database column owner.packages row with JSON payload:

[
    {
        "name":"laravel/dusk",
        "versions":["6.17.0", "6.16.0", "6.15.1", "6.15.0"]
    },
    {
        "name":"laravel/framework",
        "versions":["8.54.0", "8.53.1", "8.53.0"]
    }
]

DB::table('owner')
    ->where('packages[1]->name', 'laravel/framework')
    ->update(['packages[1]->versions[0]' => '9.0.0']);

For MySQL this compiles to:

UPDATE `owner`
SET `packages` = JSON_SET(`packages`, $"[1]"."versions[0]", '9.0.0')
WHERE json_unquote(json_extract(`packages[1]`, '$."name"')) = 'laravel/framework';

Which would instead append to the above payload with a new nested JSON object key:

{
    "0": {
        "name":"laravel/dusk",
        "versions":["6.17.0", "6.16.0", "6.15.1", "6.15.0"]
    },
    "1": {
        "name":"laravel/framework",
        "versions":["8.54.0", "8.53.1", "8.53.0"]
    },
    "[1]": {
        "versions[0]": "9.0.0"
    }
}

When escaping JSON path segments, avoid enclosing array dereference characters [ and ]:

UPDATE `owner`
SET `framework` = JSON_SET(`framework`, $[1]."versions"[0], '9.0.0')
WHERE json_unquote(json_extract(`packages[1]`, '$."name"')) = 'laravel/framework';

---

Database documentation for JSON path support

Each of Laravel's supported database drivers use the same syntax for JSON path array indices.

• MySQL: https://dev.mysql.com/doc/refman/8.0/en/json.html#json-path-syntax
• MariaDB: https://mariadb.com/kb/en/jsonpath-expressions/
• PostgreSQL: https://www.postgresql.org/docs/12/functions-json.html#FUNCTIONS-SQLJSON-PATH
• SQLite: https://www.sqlite.org/json1.html (scroll down for examples as the prose is bad readin')
• SQL Server: https://docs.microsoft.com/en-us/sql/relational-databases/json/json-path-expressions-sql-server?view=sql-server-ver15

JSON path features supported by database drivers

Database software	Identifier	Object key	Array index	Key asterisk	Array asterisk	Key match double-asterisk
MySQL	[x]	[x]	[x]	[x]	[x]	[x]
MariaDB	[x]	[x]	[x]	[x]	[x]	[x]
PostgreSQL	[x]	[x]	[x]		[x]
SQLite	[x]	[x]	[x]
SQL Server	[x]	[x]	[x]

For the above features, Laravel supports the first three columns that all the database drivers implement. Segment-matching * / ** characters partially supported by some drivers still require Laravel developers to use raw SQL.

9.x upgrade guide

In the low likelihood use case that applications use JSON column payload object key names ending in regex \[.+\], databases will need to be updated to no longer use such characters.

[derekmd]

I guess there's no CI test suite feedback targeting the 9.x master branch until Symfony 6 compatibility gets sorted?

[driesvints]

Thanks for sending this in @derekmd!

Yes, I'm gonna try to get those sorted out today.

[driesvints]

Put this PR in draft for now.

[driesvints]

@derekmd you can rebase this now : )

[taylorotwell]

Thanks!

[driesvints]

Thanks @derekmd! 👏

[CraigMonva]

Is there a limitation on including this in the next laravel 8 minor? Is it backwords breaking?

Always running into issues writing tests around this issue.

January 25th, 2022 for Laravel 9 seems so far off for this feature :(

[derekmd]

@CraigMonva: It's possible for apps to introduce the fix with 8.x by extending the Grammar class for your DB driver and configuring it using Connection::setQueryGrammar():

app/Database/JsonPathFixGrammar.php

namespace App\Database;

// extend the driver-appropriate grammar for your DB connection, like MySqlGrammar
use Illuminate\Database\Query\Grammars\MySqlGrammar;
use Illuminate\Support\Str;

class JsonPathFixGrammar extends MySqlGrammar
{
    protected function wrapJsonPath($value, $delimiter = '->')
    {
        $value = preg_replace("/([\\\\]+)?\\'/", "''", $value);

        $jsonPath = collect(explode($delimiter, $value))
            ->map(function ($segment) {
                return $this->wrapJsonPathSegment($segment);
            })
            ->join('.');

        return "'$".(Str::startsWith($jsonPath, '[') ? '' : '.').$jsonPath."'";
    }

    protected function wrapJsonPathSegment($segment)
    {
        if (preg_match('/(\[[^\]]+\])+$/', $segment, $parts)) {
            $key = Str::beforeLast($segment, $parts[0]);

            if (! empty($key)) {
                return '"'.$key.'"'.$parts[0];
            }

            return $parts[0];
        }

        return '"'.$segment.'"';
    }
}

app/Providers/AppServiceProvider.php

namespace App\Providers;

use App\Database\JsonPathFixGrammar;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\ServiceProvider;

class AppServiceProvider extends ServiceProvider
{
    public function boot()
    {
        DB::connection()->setQueryGrammar(new JsonPathFixGrammar);
    }
}

[CraigMonva]

@derekmd

Hi Derek.

Upgrading to L9 which should include this? Am I miss understanding what this change was meant to fix?

My tests pass without asserting

'preferences->energy->channel->0' => 'email',
'preferences->energy->channel->1' => 'sms',

[CraigMonva]

@derekmd

Jesus christ nevermind. Thought Id been given a laravel shift branch with composer update already run on it. Aparently not.