sql: introduce EXPLAIN BUNDLE #45735

RaduBerinde · 2020-03-05T00:23:32Z

This is an initial version of EXPLAIN BUNDLE. Similar to EXPLAIN ANALYZE, it
runs the statement but throws away all resulting rows and instead returns a URL
to a zip bundle (which is served via a special HTTP endpoint); example:

root@127.249.95.44:37247/movr> EXPLAIN BUNDLE SELECT * FROM promo_codes;
                                  text
------------------------------------------------------------------------
  Download the bundle from:
    http://127.249.95.44:35445/_admin/v1/stmtbundle/535092672054657025
  or from the Admin UI (Advanced Debug -> Statement Diagnostics).
(1 row)

The bundle currently contains the statement and EXPLAIN, EXPLAIN (OPT)
plans. The plan is to add (in future changes) the OPT, ENV data, table stats,
EXPLAIN ANALYZE diagram, and traces.

This is not yet integrated with Andrei's query diagnostics work other than using
the same system.table. This will be addressed in future changes; the user will
be able to request a bundle+trace from the UI. Conversely, an EXPLAIN BUNDLE
will also fill out the trace field in the system table.

Note that the AdminUI will have an interface to list and retrieve collected
bundles.

Release note (sql change): EXPLAIN BUNDLE can be used to run a query and collect
execution information in a support bundle, which can be downloaded via the
AdminUI.

cockroach-teamcity · 2020-03-05T00:23:39Z

This change is

RaduBerinde · 2020-03-05T00:24:38Z

@dhartunian @nathanstilwell can you take a look at the http endpoint I set up in admin,go to double check I'm not doing something stupid security-wise?

andreimatei

LGTM

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @andreimatei, @dhartunian, @jordanlewis, @nathanstilwell, and @RaduBerinde)

pkg/server/admin.go, line 1528 at r1 (raw file):

}

func (s *adminServer) writeStatementBundle(ctx context.Context, id int64, w http.ResponseWriter) {

the "write" in the name seems confusing to me. I understand that we're technically writing something to a writer, but I'd name it more like getStatementBundle.

pkg/server/admin.go, line 1575 at r1 (raw file):

		fmt.Sprintf("attachment; filename=stmt-bundle-%d.zip", id),
	)
	//w.Header().Set("Content-Type", req.Header.Get("Content-Type"))

sup with this?

pkg/server/admin.go, line 1578 at r1 (raw file):

	if _, err := io.Copy(w, &bundle); err != nil {
		// Too late to return an error now..

well, if this fails, it fails because there's no longer anybody to return the error to, right?

pkg/sql/conn_executor_exec.go, line 352 at r1 (raw file):

	case *tree.ExplainBundle:
		stmt.Statement.AST = s.Statement
		// XXX .SQL?

xxx

pkg/sql/conn_executor_exec.go, line 353 at r1 (raw file):

		stmt.Statement.AST = s.Statement
		// XXX .SQL?
		discardRows = true

put a comment here emphasizing that we don't return the rows for the inner query, but instead return a URL. And put a TODO to capture some rows and include them; I think being able to use this functionality through the admin ui to figure out what some requests are returning would be very cool.

pkg/sql/conn_executor_exec.go, line 491 at r1 (raw file):

	}
	if explainBundle {
		if err := p.generateExplainBundle(ctx, res); err != nil {

make a note pls that we're not getting here in case of query execution errors

pkg/sql/explain_bundle.go, line 30 at r1 (raw file):

// generateExplainBundle creates the bundle and returns the result for an
// EXPLAIN BUNDLE statement. The result is text containing a URL for the bundle.
func (p *planner) generateExplainBundle(ctx context.Context, res RestrictedCommandResult) error {

I looked briefly to see who integral the planner is to the whole thing, and I lost track.
To the greatest extent possible, I'd move away from putting more logic in the planner. I think it's fine for the planner to collect information (particularly planning information), but I'd try to pass that information directly into this function.

pkg/sql/explain_bundle.go, line 73 at r1 (raw file):

	err = db.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
		// Insert the bundle into system.statement_bundle_chunks.

while there's a single chunk, I'd modify the reading code to also deal with a single chunk

pkg/sql/explain_common.go, line 15 at r1 (raw file):

import "github.com/cockroachdb/cockroach/pkg/sql/opt/memo"

func (p *planner) FormatOptPlan(flags memo.ExprFmtFlags) string {

add a comment pls

RaduBerinde

TFTR!

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @andreimatei, @dhartunian, @jordanlewis, and @nathanstilwell)

pkg/server/admin.go, line 1528 at r1 (raw file):