Execute script

[fantix]

Depends on #3814, edgedb/edgedb-python#310

• Add tests
• Return result of the last command in a script
• Always use stmt_mode = "all" in EdgeConnection._compile() and check in server.compiler
• Fix tests
• Support constant extraction on multiple-statement queries
• Drop remaining SimpleQuery

Sample scenarios of EdgeDB Execute generating Postgres messages using pseudo-code:

Single-command EdgeQL

EdgeDB Protocol:

Execute(
    insert Hero { name := 'Daisy Johnson' }
)

Generated Postgres messages:

Parse("INSERT INTO heros (name) VALUES ($1)")
Bind('Daisy Johnson')
Execute()
Sync()

On protocol-level, transaction control commands like START TRANSACTION, COMMIT, ROLLBACK [...] are allowed.

Multi-command EdgeQL

EdgeDB Protocol:

Execute(
    insert Hero { name := 'Daisy Johnson' };
    insert Hero { name := 'Phillip J. Coulson' };
)

Generated Postgres messages:

Parse("INSERT INTO heros (name) VALUES ($1)")
Bind('Daisy Johnson')
Execute()
Parse("INSERT INTO heros (name) VALUES ($1)")  # Maybe omitted if prepared statement rules apply
Bind('Phillip J. Coulson')
Execute()
Sync()

There is only one Sync per EdgeDB Execute, so it is either an atomic implicit transaction in Postgres, or a part of the outer existing transaction.

On protocol-level, transaction control commands are NOT allowed.

Flush messages may be added if SET GLOBAL is in the script, but that won't affect the atomicity. Only the values generated from the last command will be returned.

Migration Commands

START MIGRATION ..., COMMIT MIGRATION and ABORT MIGRATION are allowed in scripts, but only if all START MIGRATION ... have a matching COMMIT MIGRATION or ABORT MIGRATION. Implementation-wise, migration commands in scripts will never open or close a database transaction or savepoint, they are always a part of the outer implicit or explicit transaction.

In addition, there will be a new capability MIGRATION_TX, gating the ability of the migration commands to actually open or close a database transaction. The client bindings should always turn this capability off, so that the following practice is forbidden:

client.execute("START MIGRATION TO ...")
...
client.execute("COMMIT MIGRATION")

Instead, the client could either run migration in a script:

client.execute("""
    START MIGRATION TO ...;
    ...
    COMMIT MIGRATION;
""")

Or use an explicit transaction to wrap the separate migration commands:

for tx in client.transaction():
    with tx:
        tx.execute("START MIGRATION TO ...")
        ...
        tx.execute("COMMIT MIGRATION")
        # Or just use a script again:
        tx.execute("""
            START MIGRATION TO ...;
            ...
            COMMIT MIGRATION;
        """)

Client Bindings

Both query() and execute() will use the same Parse/Execute EdgeDB protocol messages, both accepting single-command and multi-command EdgeQL. The difference is, execute() will generate Parse/Execute messages with output_format=NULL (previously known as io_format), while query() will use output_format=BINARY (or JSON for the query_*_json() family).

Rejected Ideas

Transactional Script

EdgeDB Protocol:

Execute(
    start transaction;
    insert Hero { name := 'Daisy Johnson' };
    commit;
    start transaction;
    insert Hero { name := 'Phillip J. Coulson' };
    rollback;
)

Generated Postgres messages:

Parse("START TRANSACTION")
Bind()
Execute()
Parse("INSERT INTO heros (name) VALUES ($1)")
Bind('Daisy Johnson')
Execute()
Parse("COMMIT")
Bind()
Execute()
Sync()

# If we're in error state: (or always ROLLBACK after COMMIT to save round-trip time)
Parse("ROLLBACK")
Bind()
Execute()
Sync()

Parse("START TRANSACTION")
Bind()
Execute()
Parse("INSERT INTO heros (name) VALUES ($1)")  # Maybe omitted if prepared statement rules apply
Bind('Phillip J. Coulson')
Execute()
Sync()  # make sure the following ROLLBACK works, clearing `ignore_till_sync` pg state
Parse("ROLLBACK")
Bind()
Execute()
Sync()

Returning values is NOT allowed.

Illegal Transactional Script

Statements outside transactions are not allowed:

Execute(
    start transaction;
    insert Hero { name := 'Daisy Johnson' };
    commit;
    insert Hero { name := 'Phillip J. Coulson' };
)

Mismatching transaction boundaries are not allowed:

Execute(
    insert Hero { name := 'Daisy Johnson' };
    commit;
    start transaction;
    insert Hero { name := 'Phillip J. Coulson' };
    rollback;
)

[elprans]

No obvious issues jumped out at me, so let's merge this and do post-merge review and testing.

[elprans]

Drop this? We shouldn't be supporting 0.11 and below anymore.

[fantix]

I think this is for restoring pre-0.12 dumps - I'll complete the comment here.

[1st1]

Let's merge this.