Skip to content

Commit

Permalink
feat: Support for Directed Reads (#1966)
Browse files Browse the repository at this point in the history
  • Loading branch information
surbhigarg92 committed Jan 8, 2024
1 parent 6c0eef1 commit c0a4363
Show file tree
Hide file tree
Showing 17 changed files with 617 additions and 18 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Expand Up @@ -2,6 +2,8 @@
**/node_modules
/.coverage
/coverage
/.idea
/.vscode
/.nyc_output
/docs/
/out/
Expand Down
1 change: 1 addition & 0 deletions README.md
Expand Up @@ -112,6 +112,7 @@ Samples are in the [`samples/`](https://github.com/googleapis/nodejs-spanner/tre
| Updates the default leader of an existing database | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/database-update-default-leader.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/database-update-default-leader.js,samples/README.md) |
| Updates a Cloud Spanner Database. | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/database-update.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/database-update.js,samples/README.md) |
| Datatypes | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/datatypes.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/datatypes.js,samples/README.md) |
| Runs an execute sql request with directed read options | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/directed-reads.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/directed-reads.js,samples/README.md) |
| Delete using DML returning. | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/dml-returning-delete.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/dml-returning-delete.js,samples/README.md) |
| Insert using DML returning. | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/dml-returning-insert.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/dml-returning-insert.js,samples/README.md) |
| Update using DML returning. | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/dml-returning-update.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/dml-returning-update.js,samples/README.md) |
Expand Down
2 changes: 1 addition & 1 deletion protos/protos.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion protos/protos.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 18 additions & 0 deletions samples/README.md
Expand Up @@ -37,6 +37,7 @@ and automatic, synchronous replication for high availability.
* [Updates the default leader of an existing database](#updates-the-default-leader-of-an-existing-database)
* [Updates a Cloud Spanner Database.](#updates-a-cloud-spanner-database.)
* [Datatypes](#datatypes)
* [Runs an execute sql request with directed read options](#runs-an-execute-sql-request-with-directed-read-options)
* [Delete using DML returning.](#delete-using-dml-returning.)
* [Insert using DML returning.](#insert-using-dml-returning.)
* [Update using DML returning.](#update-using-dml-returning.)
Expand Down Expand Up @@ -517,6 +518,23 @@ __Usage:__



### Runs an execute sql request with directed read options

View the [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/directed-reads.js).

[![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/directed-reads.js,samples/README.md)

__Usage:__


`node directed-reads.js <INSTANCE_ID> <DATABASE_ID> <PROJECT_ID>`


-----




### Delete using DML returning.

View the [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/dml-returning-delete.js).
Expand Down
117 changes: 117 additions & 0 deletions samples/directed-reads.js
@@ -0,0 +1,117 @@
// Copyright 2024 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// sample-metadata:
// title: Runs an execute sql request with directed read options
// usage: node directed-reads.js <INSTANCE_ID> <DATABASE_ID> <PROJECT_ID>

'use strict';

function main(
instanceId = 'my-instance',
databaseId = 'my-database',
projectId = 'my-project-id'
) {
// [START spanner_directed_read]
// Imports the Google Cloud Spanner client library
const {Spanner, protos} = require('@google-cloud/spanner');

// Only one of excludeReplicas or includeReplicas can be set
// Each accepts a list of replicaSelections which contains location and type
// * `location` - The location must be one of the regions within the
// multi-region configuration of your database.
// * `type` - The type of the replica
// Some examples of using replicaSelectors are:
// * `location:us-east1` --> The "us-east1" replica(s) of any available type
// will be used to process the request.
// * `type:READ_ONLY` --> The "READ_ONLY" type replica(s) in nearest
//. available location will be used to process the
// request.
// * `location:us-east1 type:READ_ONLY` --> The "READ_ONLY" type replica(s)
// in location "us-east1" will be used to process
// the request.
// includeReplicas also contains an option for autoFailover which when set
// Spanner will not route requests to a replica outside the
// includeReplicas list when all the specified replicas are unavailable
// or unhealthy. The default value is `false`
const directedReadOptionsForClient = {
excludeReplicas: {
replicaSelections: [
{
location: 'us-east4',
},
],
},
};

// Instantiates a client with directedReadOptions
const spanner = new Spanner({
projectId: projectId,
directedReadOptions: directedReadOptionsForClient,
});

async function spannerDirectedReads() {
// Gets a reference to a Cloud Spanner instance and backup
const instance = spanner.instance(instanceId);
const database = instance.database(databaseId);
const directedReadOptionsForRequest = {
includeReplicas: {
replicaSelections: [
{
type: protos.google.spanner.v1.DirectedReadOptions.ReplicaSelection
.Type.READ_ONLY,
},
],
autoFailoverDisabled: true,
},
};

await database.getSnapshot(async (err, transaction) => {
if (err) {
console.error(err);
return;
}
try {
// Read rows while passing directedReadOptions directly to the query.
// These will override the options passed at Client level.
const [rows] = await transaction.run({
sql: 'SELECT SingerId, AlbumId, AlbumTitle FROM Albums',
directedReadOptions: directedReadOptionsForRequest,
});
rows.forEach(row => {
const json = row.toJSON();
console.log(
`SingerId: ${json.SingerId}, AlbumId: ${json.AlbumId}, AlbumTitle: ${json.AlbumTitle}`
);
});
console.log(
'Successfully executed read-only transaction with directedReadOptions'
);
} catch (err) {
console.error('ERROR:', err);
} finally {
transaction.end();
// Close the database when finished.
await database.close();
}
});
}
spannerDirectedReads();
// [END spanner_directed_read]
}
process.on('unhandledRejection', err => {
console.error(err.message);
process.exitCode = 1;
});
main(...process.argv.slice(2));
20 changes: 20 additions & 0 deletions samples/system-test/spanner.test.js
Expand Up @@ -1985,5 +1985,25 @@ describe('Spanner', () => {
)
);
});

// directed_read_options
it('should run read-only transaction with directed read options set', async () => {
const output = execSync(
`node directed-reads.js ${INSTANCE_ID} ${DATABASE_ID} ${PROJECT_ID}`
);
console.log(output);
assert.match(
output,
new RegExp(
'SingerId: 2, AlbumId: 2, AlbumTitle: Forever Hold your Peace'
)
);
assert.match(
output,
new RegExp(
'Successfully executed read-only transaction with directedReadOptions'
)
);
});
});
});
2 changes: 1 addition & 1 deletion src/batch-transaction.ts
Expand Up @@ -20,7 +20,7 @@ import * as extend from 'extend';
import * as is from 'is';
import {Snapshot} from './transaction';
import {google} from '../protos/protos';
import {Session, Database, Spanner} from '.';
import {Session, Database} from '.';
import {
CLOUD_RESOURCE_HEADER,
addLeaderAwareRoutingHeader,
Expand Down
12 changes: 12 additions & 0 deletions src/index.ts
Expand Up @@ -110,13 +110,17 @@ export type GetInstanceConfigOperationsCallback = PagedCallback<
* Session pool configuration options.
* @property {boolean} [routeToLeaderEnabled=True] If set to false leader aware routing will be disabled.
* Disabling leader aware routing would route all requests in RW/PDML transactions to any region.
* @property {google.spanner.v1.IDirectedReadOptions} [directedReadOptions] Sets the DirectedReadOptions for all ReadRequests and ExecuteSqlRequests for the Client.
* Indicates which replicas or regions should be used for non-transactional reads or queries.
* DirectedReadOptions won't be set for readWrite transactions"
*/
export interface SpannerOptions extends GrpcClientOptions {
apiEndpoint?: string;
servicePath?: string;
port?: number;
sslCreds?: grpc.ChannelCredentials;
routeToLeaderEnabled?: boolean;
directedReadOptions?: google.spanner.v1.IDirectedReadOptions | null;
}
export interface RequestConfig {
client: string;
Expand Down Expand Up @@ -217,6 +221,7 @@ class Spanner extends GrpcService {
projectFormattedName_: string;
resourceHeader_: {[k: string]: string};
routeToLeaderEnabled = true;
directedReadOptions: google.spanner.v1.IDirectedReadOptions | null;

/**
* Placeholder used to auto populate a column with the commit timestamp.
Expand Down Expand Up @@ -291,6 +296,12 @@ class Spanner extends GrpcService {
},
options || {}
) as {} as SpannerOptions;

const directedReadOptions = options.directedReadOptions
? options.directedReadOptions
: null;
delete options.directedReadOptions;

const emulatorHost = Spanner.getSpannerEmulatorHost();
if (
emulatorHost &&
Expand Down Expand Up @@ -332,6 +343,7 @@ class Spanner extends GrpcService {
this.resourceHeader_ = {
[CLOUD_RESOURCE_HEADER]: this.projectFormattedName_,
};
this.directedReadOptions = directedReadOptions;
}

/** Closes this Spanner client and cleans up all resources used by it. */
Expand Down
2 changes: 2 additions & 0 deletions src/session-pool.ts
Expand Up @@ -1081,7 +1081,9 @@ export class SessionPool extends EventEmitter implements SessionPoolInterface {
* @private
*/
_stopHouseKeeping(): void {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
clearInterval(this._pingHandle as any);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
clearInterval(this._evictHandle as any);
}
}
40 changes: 40 additions & 0 deletions src/transaction.ts
Expand Up @@ -86,6 +86,7 @@ export interface ExecuteSqlRequest extends Statement, RequestOptions {
queryOptions?: IQueryOptions;
requestOptions?: Omit<IRequestOptions, 'transactionTag'>;
dataBoostEnabled?: boolean | null;
directedReadOptions?: google.spanner.v1.IDirectedReadOptions;
}

export interface KeyRange {
Expand All @@ -107,6 +108,7 @@ export interface ReadRequest extends RequestOptions {
partitionToken?: Uint8Array | null;
requestOptions?: Omit<IRequestOptions, 'transactionTag'>;
dataBoostEnabled?: boolean | null;
directedReadOptions?: google.spanner.v1.IDirectedReadOptions;
}

export interface BatchUpdateError extends grpc.ServiceError {
Expand Down Expand Up @@ -457,6 +459,8 @@ export class Snapshot extends EventEmitter {
* PartitionReadRequest message used to create this partition_token.
* @property {google.spanner.v1.RequestOptions} [requestOptions]
* Common options for this request.
* @property {google.spanner.v1.IDirectedReadOptions} [directedReadOptions]
* Indicates which replicas or regions should be used for non-transactional reads or queries.
* @property {object} [gaxOptions]
* Call options. See {@link https://googleapis.dev/nodejs/google-gax/latest/interfaces/CallOptions.html|CallOptions}
* for more details.
Expand Down Expand Up @@ -591,6 +595,10 @@ export class Snapshot extends EventEmitter {
transaction.singleUse = this._options;
}

const directedReadOptions = this._getDirectedReadOptions(
request.directedReadOptions
);

request = Object.assign({}, request);

delete request.gaxOptions;
Expand All @@ -600,6 +608,7 @@ export class Snapshot extends EventEmitter {
delete request.keys;
delete request.ranges;
delete request.requestOptions;
delete request.directedReadOptions;

const reqOpts: spannerClient.spanner.v1.IReadRequest = Object.assign(
request,
Expand All @@ -610,6 +619,7 @@ export class Snapshot extends EventEmitter {
this.requestOptions?.transactionTag ?? undefined,
requestOptions
),
directedReadOptions: directedReadOptions,
transaction,
table,
keySet,
Expand Down Expand Up @@ -993,6 +1003,8 @@ export class Snapshot extends EventEmitter {
* that it is not ready for any more data. Increase this value if you
* experience 'Stream is still not ready to receive data' errors as a
* result of a slow writer in your receiving stream.
* @property {object} [directedReadOptions]
* Indicates which replicas or regions should be used for non-transactional reads or queries.
*/
/**
* Create a readable object stream to receive resulting rows from a SQL
Expand Down Expand Up @@ -1068,6 +1080,10 @@ export class Snapshot extends EventEmitter {
query;
let reqOpts;

const directedReadOptions = this._getDirectedReadOptions(
query.directedReadOptions
);

const sanitizeRequest = () => {
query = query as ExecuteSqlRequest;
const {params, paramTypes} = Snapshot.encodeParams(query);
Expand All @@ -1085,6 +1101,7 @@ export class Snapshot extends EventEmitter {
delete query.maxResumeRetries;
delete query.requestOptions;
delete query.types;
delete query.directedReadOptions;

reqOpts = Object.assign(query, {
session: this.session.formattedName_!,
Expand All @@ -1094,6 +1111,7 @@ export class Snapshot extends EventEmitter {
this.requestOptions?.transactionTag ?? undefined,
requestOptions
),
directedReadOptions: directedReadOptions,
transaction,
params,
paramTypes,
Expand Down Expand Up @@ -1288,6 +1306,28 @@ export class Snapshot extends EventEmitter {
return {params, paramTypes};
}

/**
* Get directed read options
* @private
* @param {google.spanner.v1.IDirectedReadOptions} directedReadOptions Request directedReadOptions object.
*/
protected _getDirectedReadOptions(
directedReadOptions:
| google.spanner.v1.IDirectedReadOptions
| null
| undefined
) {
if (
!directedReadOptions &&
this._getSpanner().directedReadOptions &&
this._options.readOnly
) {
return this._getSpanner().directedReadOptions;
}

return directedReadOptions;
}

/**
* Update transaction properties from the response.
*
Expand Down

0 comments on commit c0a4363

Please sign in to comment.