Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support Questions status variable #2422

Merged
merged 6 commits into from
Mar 29, 2024
Merged

Support Questions status variable #2422

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
08cadcb
implement status variable questions
Mar 29, 2024
98185f3
adding tests
Mar 29, 2024
f93faa3
[ga-format-pr] Run ./format_repo.sh to fix formatting
jycor Mar 29, 2024
788046b
need to init
Mar 29, 2024
a9df5c5
Merge branch 'james/questions' of https://github.com/dolthub/go-mysql…
Mar 29, 2024
7d0a0fa
feedback
Mar 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
17 changes: 16 additions & 1 deletion server/handler.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Copyright 2020-2021 Dolthub, Inc.
// Copyright 2020-2024 Dolthub, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -335,6 +335,16 @@ func (h *Handler) ComBinlogDumpGTID(c *mysql.Conn, logFile string, logPos uint64

var queryLoggingRegex = regexp.MustCompile(`[\r\n\t ]+`)

func incrementStatusVariableQuestions(ctx *sql.Context) {
// ignore all errors relating to status updates
if _, globalQuesVal, ok := sql.StatusVariables.GetGlobal("Questions"); ok {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good note about ignoring errors. That seems appropriate here. It might be helpful to add a similar comment mentioning that there is a race condition here, but we are consciously making the tradeoff to have a small risk of race condition and losing a question count or two instead of taking on the extra expense of a mutex. Alternatively... I saw your comment below about considering spinning up a go routine. If we do want to add locking here, then I think you're right it would make sense to spin up a goroutine at that point.

I mention that, since I think someone looking at this code in the future might wonder why there is no locking, so it's nice to encode that decision in a comment.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw... any more context on how you saw the count being incremented twice for errors? I'm not seeing the same behavior. I do see an increment of 2, but I think one is for the error statement and one is for SHOW STATUS:

mysql> show status like 'Questions';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 19    |
+---------------+-------+
1 row in set (0.00 sec)

mysql> show status like 'Questions';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 20    |
+---------------+-------+
1 row in set (0.00 sec)

mysql> asdf;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'asdf' at line 1
mysql> show status like 'Questions';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 22    |
+---------------+-------+
1 row in set (0.00 sec)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what I see from MySQL:

mysql> show status like "questions";
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 41    |
+---------------+-------+
1 row in set (0.0010 sec)

mysql> asdf;
ERROR: 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'asdf' at line 1

mysql> show status like "questions";
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 44    |
+---------------+-------+
1 row in set (0.0012 sec)

I think it might be running a query under the hood to log errors? or... it's some windows quirk.
I'll just have it increment by one now, and can just update it whenever I figure out why errors are counted twice on my machine.

sql.StatusVariables.SetGlobal("Questions", globalQuesVal.(int64)+1)
}
if sessQuesVal, err2 := ctx.Session.GetStatusVariable(ctx, "Questions"); err2 == nil {
ctx.Session.SetStatusVariable(ctx, "Questions", sessQuesVal.(int64)+1)
}
}

func (h *Handler) doQuery(
c *mysql.Conn,
query string,
Expand Down Expand Up @@ -388,6 +398,11 @@ func (h *Handler) doQuery(

ctx.GetLogger().Tracef("beginning execution")

// TODO: a goroutine might be worth it here, as read/writing status variables go through a mutex
defer func() {
incrementStatusVariableQuestions(ctx)
}()

oCtx := ctx
eg, ctx := ctx.NewErrgroup()

Expand Down
161 changes: 160 additions & 1 deletion server/handler_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Copyright 2020-2021 Dolthub, Inc.
// Copyright 2020-2024 Dolthub, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -1199,3 +1199,162 @@ func newConn(id uint32) *mysql.Conn {
Conn: new(mockConn),
}
}

func dummyCb(_ *sqltypes.Result, _ bool) error {
return nil
}

func TestStatusVariableQuestions(t *testing.T) {
variables.InitStatusVariables()

e, pro := setupMemDB(require.New(t))
dbFunc := pro.Database
handler := &Handler{
e: e,
sm: NewSessionManager(
testSessionBuilder(pro),
sql.NoopTracer,
dbFunc,
sql.NewMemoryManager(nil),
sqle.NewProcessList(),
"foo",
),
readTimeout: time.Second,
}

conn1 := newConn(1)
handler.NewConnection(conn1)
err := handler.ComInitDB(conn1, "test")
require.NoError(t, err)
sess1 := handler.sm.sessions[1]

_, globalVal, ok := sql.StatusVariables.GetGlobal("Questions")
require.True(t, ok)
require.Equal(t, int64(0), globalVal)

sessVal, err := sess1.GetStatusVariable(nil, "Questions")
require.NoError(t, err)
require.Equal(t, int64(0), sessVal)

// Call ComQuery 5 times
for i := 0; i < 5; i++ {
err = handler.ComQuery(conn1, "SELECT 1", dummyCb)
require.NoError(t, err)
}

_, globalVal, ok = sql.StatusVariables.GetGlobal("Questions")
require.True(t, ok)
require.Equal(t, int64(5), globalVal)

sessVal, err = sess1.GetStatusVariable(nil, "Questions")
require.NoError(t, err)
require.Equal(t, int64(5), sessVal)

conn2 := newConn(2)
handler.NewConnection(conn2)
err = handler.ComInitDB(conn2, "test")
require.NoError(t, err)
sess2 := handler.sm.sessions[2]

// Get 5 syntax errors
for i := 0; i < 5; i++ {
err = handler.ComQuery(conn2, "syntax error", dummyCb)
require.Error(t, err)
}

_, globalVal, ok = sql.StatusVariables.GetGlobal("Questions")
require.True(t, ok)
require.Equal(t, int64(10), globalVal)

sessVal, err = sess1.GetStatusVariable(nil, "Questions")
require.NoError(t, err)
require.Equal(t, int64(5), sessVal)

// Errors also increment Questions (it's 2x on Windows for some reason)
sessVal, err = sess2.GetStatusVariable(nil, "Questions")
require.NoError(t, err)
require.Equal(t, int64(5), sessVal)

conn3 := newConn(3)
handler.NewConnection(conn3)
err = handler.ComInitDB(conn3, "test")
require.NoError(t, err)
sess3 := handler.sm.sessions[3]

err = handler.ComQuery(conn3, "create procedure p() begin select 1; select 2; select 3; end", dummyCb)
require.NoError(t, err)

_, globalVal, ok = sql.StatusVariables.GetGlobal("Questions")
require.True(t, ok)
require.Equal(t, int64(11), globalVal)

sessVal, err = sess3.GetStatusVariable(nil, "Questions")
require.NoError(t, err)
require.Equal(t, int64(1), sessVal)

// Calling stored procedure with multiple queries only increment Questions once.
err = handler.ComQuery(conn3, "call p()", dummyCb)
require.NoError(t, err)

_, globalVal, ok = sql.StatusVariables.GetGlobal("Questions")
require.True(t, ok)
require.Equal(t, int64(12), globalVal)

sessVal, err = sess1.GetStatusVariable(nil, "Questions")
require.NoError(t, err)
require.Equal(t, int64(5), sessVal)

sessVal, err = sess2.GetStatusVariable(nil, "Questions")
require.NoError(t, err)
require.Equal(t, int64(5), sessVal)

sessVal, err = sess3.GetStatusVariable(nil, "Questions")
require.NoError(t, err)
require.Equal(t, int64(2), sessVal)

conn4 := newConn(4)
handler.NewConnection(conn4)
err = handler.ComInitDB(conn4, "test")
require.NoError(t, err)
sess4 := handler.sm.sessions[4]

// TODO: implement and test that ComPing does not increment Questions
// TODO: implement and test that ComStatistics does not increment Questions
// TODO: implement and test that ComStmtClose does not increment Questions
// TODO: implement and test that ComStmtReset does not increment Questions

// Prepare does not increment Questions
prepare := &mysql.PrepareData{
StatementID: 0,
PrepareStmt: "select ?",
ParamsCount: 0,
ParamsType: nil,
ColumnNames: nil,
BindVars: map[string]*query.BindVariable{
"v1": {Type: query.Type_INT8, Value: []byte("5")},
},
}

_, err = handler.ComPrepare(conn4, prepare.PrepareStmt, samplePrepareData)
require.NoError(t, err)

_, globalVal, ok = sql.StatusVariables.GetGlobal("Questions")
require.True(t, ok)
require.Equal(t, int64(12), globalVal)

sessVal, err = sess4.GetStatusVariable(nil, "Questions")
require.NoError(t, err)
require.Equal(t, int64(0), sessVal)

// Execute does increment Questions
err = handler.ComStmtExecute(conn4, prepare, func(*sqltypes.Result) error { return nil })
require.NoError(t, err)

_, globalVal, ok = sql.StatusVariables.GetGlobal("Questions")
require.True(t, ok)
require.Equal(t, int64(13), globalVal)

sessVal, err = sess4.GetStatusVariable(nil, "Questions")
require.NoError(t, err)
require.Equal(t, int64(1), sessVal)
}
5 changes: 4 additions & 1 deletion sql/variables/status_variables.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,10 @@ func (g *globalStatusVariables) NewSessionMap() map[string]sql.StatusVarValue {
if v.Var.GetScope() == sql.StatusVariableScope_Global {
continue
}
sessionMap[k] = v
sessionMap[k] = sql.StatusVarValue{
Var: v.Var,
Val: v.Var.GetDefault(),
}
}
return sessionMap
}
Expand Down
4 changes: 2 additions & 2 deletions sql/variables/status_variables_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,11 +101,11 @@ func TestStatusVariables(t *testing.T) {
require.True(ok)
require.Equal(int64(200), globalVal)

// New Session preserves values with Both scope
// New Session does not preserve values with Both scope
newSess := sql.NewBaseSessionWithClientServer("foo", sql.Client{Address: "baz2", User: "bar2"}, 2)
sessVal, err = newSess.GetStatusVariable(ctx, "Bytes_received")
require.NoError(err)
require.Equal(int64(200), sessVal)
require.Equal(int64(0), sessVal)

// New Session does not preserve values with Session scope
sessVal, err = newSess.GetStatusVariable(ctx, "Compression")
Expand Down