Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow host in dsn and use statement based transactions #10

Merged
merged 13 commits into from
Aug 2, 2021

Conversation

olavloite
Copy link
Collaborator

@olavloite olavloite commented Jul 21, 2021

  • Allows adding a host name to the dsn. This makes the driver testable with a mock server.
  • Changes the transaction handling from using a transaction function to using statement based transactions.
  • Moves the integration tests to a single file, separating them from the unit tests. Skips integration tests in -short mode.
  • Adds a parser to remove comments from sql strings. This can be used to reliably determine the query parameters in a sql string, instead of the current regular expression that will match any string starting with a @.
  • Adds a GitHub Actions configuration for automatically running all unit tests during presubmits.

Fixes #11
Fixes #12

@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Jul 21, 2021

---

gorm cannot use the driver as it-is but @rakyll has been working on a dialect.
She doesn't have bandwidth to ship a fully featured dialect right now but contact
her if you would like to contribute.

---

`error = <use T(nil), not nil>`: Use a typed nil, instead of just nil.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is no longer a problem. Untyped nil parameters are now supported.

driver.go Outdated
@@ -33,67 +36,151 @@ import (
)

const userAgent = "go-sql-driver-spanner/0.1"
const dsnRegExpString = "((?P<HOSTGROUP>[\\w.-]+(?:\\.[\\w\\.-]+)*[\\w\\-\\._~:/?#\\[\\]@!\\$&'\\(\\)\\*\\+,;=.]+)/)?projects/(?P<PROJECTGROUP>(([a-z]|[-.:]|[0-9])+|(DEFAULT_PROJECT_ID)))(/instances/(?P<INSTANCEGROUP>([a-z]|[-]|[0-9])+)(/databases/(?P<DATABASEGROUP>([a-z]|[-]|[_]|[0-9])+))?)?(([\\?|;])(?P<PARAMSGROUP>.*))?"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Support setting a host name in the connection string. The format of the connection string is the same as for Java / JDBC, except it is not prefixed by [jdbc:]cloudspanner:.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we parse this regex during the compile time? A ref: https://github.com/googleapis/google-cloud-go/blob/f08c73a75e2d2a8b9a0b184179346cb97c82e9e5/spanner/client.go#L56

Also, can we give an example in the comment? It is difficult to see what dsn format should be.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I had forgotten about that possibility. I've also added some additional documentation and an example.

sql.Register("spanner", &Driver{})
}

// Driver represents a Google Cloud Spanner database/sql driver.
type Driver struct {
// Config represents the optional advanced configuration to be used
// by the Google Cloud Spanner client.
Config spanner.ClientConfig
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The driver should not contain any client configuration, that should be in the connector.

project string
instance string
database string
params map[string]string
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

params is a map that contains configuration like usePlainText=true, readonly=true, etc.

}

func openDriverConn(ctx context.Context, d *Driver, name string) (driver.Conn, error) {
if d.Config.NumChannels == 0 {
d.Config.NumChannels = 1 // TODO(jbd): Explain database/sql has a high-level management.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we are using the client library, the management of this should be kept in the client library, and not set to a reduced value here.

}

func (c *conn) PrepareContext(ctx context.Context, query string) (driver.Stmt, error) {
// TODO(jbd): Mention emails need to be escaped.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer needed, as the new parser is able to detect string literals and comments that may contain email addresses and other strings that could otherwise be seen as parameter names.

}
if err != nil {
return nil, err
}
return &result{rowsAffected: rowsAffected}, nil
}

func isDdl(query string) (bool, error) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to statement_parser.go

"testing"
)

func TestRowsAtomicTypes(t *testing.T) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Integration tests have been moved to integration_test.go and are now skipped if -short has been specified.

@olavloite olavloite marked this pull request as ready for review July 27, 2021 11:24
driver.go Outdated
return &connector{
driver: d,
connectorConfig: connectorConfig,
config: config,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we have a better name for config? We have connectorConfig and config in this struct. It is unclear what config is for.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, changed to spannerClientConfig.

driver.go Outdated
if err != nil {
return nil, err
}
return &conn{client: client, adminClient: adminClient, name: name}, nil
return &conn{client: client, adminClient: adminClient, database: databaseName}, nil
}

func createAdminClient(ctx context.Context) (adminClient *adminapi.DatabaseAdminClient, err error) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this used any where?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, not anymore, so removed.

driver.go Outdated
func (c *conn) Close() error {
c.client.Close()
return nil
}

func (c *conn) Begin() (driver.Tx, error) {
panic("Using BeginTx instead")
return nil, fmt.Errorf("use BeginTx instead")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we call BeginTx without a driver.TxOptions? E.g.,

return c.BeginTx(context.Background(), driver.TxOptions{})

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

stmt.go Show resolved Hide resolved
stmt.go Show resolved Hide resolved
stmt.go Show resolved Hide resolved
Copy link
Collaborator

@hengfengli hengfengli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@olavloite olavloite merged commit 0528e13 into googleapis:main Aug 2, 2021
@olavloite olavloite deleted the dsn branch August 2, 2021 04:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Parse SQL statements to safely determine query parameters Use StatementBasedTransactions
2 participants