Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Allow the user to set fallback to rest #1203

Merged
merged 30 commits into from
Dec 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
bf9c81f
Allow the user to set fallback to rest
danieljbruce Nov 17, 2023
778f20e
Merge branch 'main' into use-http-fallback
danieljbruce Dec 1, 2023
b6ce5d9
Putting together a test that mocks out gapic
danieljbruce Dec 1, 2023
99ef146
Merge branch 'use-http-fallback' of https://github.com/danieljbruce/n…
danieljbruce Dec 1, 2023
4b8ab09
This harness works properly
danieljbruce Dec 4, 2023
60cbaed
Add client testing module
danieljbruce Dec 4, 2023
d066264
Regroup the code so it is better arranged
danieljbruce Dec 4, 2023
fa5d14b
Remove the datastore mock
danieljbruce Dec 4, 2023
591ce30
Add comments for test helper functions. Add tests for the non rest case.
danieljbruce Dec 4, 2023
fa56366
Clean up source code change
danieljbruce Dec 4, 2023
850781c
Add parameterized test
danieljbruce Dec 4, 2023
09ee75d
Document the class for mocking datastore client
danieljbruce Dec 4, 2023
115b40a
Remove only
danieljbruce Dec 4, 2023
7f4a02a
Remove fallback params type
danieljbruce Dec 4, 2023
92030db
Remove changes to system tests
danieljbruce Dec 4, 2023
a42e8c6
Add header
danieljbruce Dec 4, 2023
e4bf66c
Check for client existence
danieljbruce Dec 4, 2023
6faca17
Change it back
danieljbruce Dec 5, 2023
54a0889
Add more debugger logs
danieljbruce Dec 5, 2023
f2ad045
Add project load logs
danieljbruce Dec 5, 2023
b532c8d
console error
danieljbruce Dec 5, 2023
2e5700a
Add comment for mocking out project id
danieljbruce Dec 5, 2023
31151be
linting fix
danieljbruce Dec 5, 2023
b3eda27
Correct title of describe block
danieljbruce Dec 5, 2023
3f97583
Change rest parameter in test to fallback
danieljbruce Dec 6, 2023
09233b5
Create a simple system test for using rest
danieljbruce Dec 6, 2023
f009f7b
Should check to see if entity is undefined
danieljbruce Dec 6, 2023
c9a7153
Merge branch 'main' into use-http-fallback
danieljbruce Dec 6, 2023
b5723a0
Specify fallback type instead of string | undefin
danieljbruce Dec 6, 2023
afc8326
Merge branch 'use-http-fallback' of https://github.com/danieljbruce/n…
danieljbruce Dec 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@ export interface LongRunningCallback {
): void;
}
export type LongRunningResponse = [Operation, google.longrunning.IOperation];
export type Fallback = boolean | 'rest' | 'proto';

export interface ExportEntitiesConfig
extends Omit<google.datastore.admin.v1.IExportEntitiesRequest, 'projectId'> {
Expand Down Expand Up @@ -1912,6 +1913,7 @@ export interface DatastoreOptions extends GoogleAuthOptions {
apiEndpoint?: string;
sslCreds?: ChannelCredentials;
databaseId?: string;
fallback?: Fallback;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is already commented somewhere else. Do you think we can provide an example how to initiate client library with fallback options?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a simple system test for this.

}

export interface KeyToLegacyUrlSafeCallback {
Expand Down
10 changes: 10 additions & 0 deletions system-test/datastore.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2080,6 +2080,16 @@ async.each(
assert.strictEqual(entities.length, 0);
});
});
describe('using the rest fallback parameter', () => {
it('should make a get call using rest instead of grpc', async () => {
const customDatastore = new Datastore({
fallback: 'rest',
});
const postKey = datastore.key(['Post', 'post1']);
const [entity] = await customDatastore.get(postKey);
assert.strictEqual(entity, undefined);
});
});
});
}
);
165 changes: 165 additions & 0 deletions test/gapic-mocks/client-initialization-testing.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

import {beforeEach, describe, it} from 'mocha';
import {
Datastore,
DatastoreClient,
Fallback,
DatastoreRequest,
DatastoreOptions,
} from '../../src';
import * as assert from 'assert';
import * as proxyquire from 'proxyquire';
import {Callback, CallOptions} from 'google-gax';
import * as protos from '../../protos/protos';
import * as ds from '../../src';
import * as mocha from 'mocha';

// eslint-disable-next-line @typescript-eslint/no-explicit-any
type Any = any;

const clientName = 'DatastoreClient';
const async = require('async');

/**
* This class mocks out the lookup function so that for tests in this file
* the lookup function just sends data back instead of making a call to the
* server. The class also saves the rest parameter in the constructor so that
* it can be read later for correctness.
*
*/
class FakeDatastoreClient extends DatastoreClient {
restParameter: Fallback;
constructor(...args: any[]) {
super();
this.restParameter = args[0].fallback;
}
lookup(
request?: protos.google.datastore.v1.ILookupRequest,
optionsOrCallback?:
| CallOptions
| Callback<
protos.google.datastore.v1.ILookupResponse,
protos.google.datastore.v1.ILookupRequest | null | undefined,
{} | null | undefined
>,
callback?: Callback<
protos.google.datastore.v1.ILookupResponse,
protos.google.datastore.v1.ILookupRequest | null | undefined,
{} | null | undefined
>
): Promise<
[
protos.google.datastore.v1.ILookupResponse,
protos.google.datastore.v1.ILookupRequest | undefined,
{} | undefined,
]
> {
if (callback) {
callback(null, {});
}
return new Promise((resolve, reject) => {
resolve([{}, {}, {}]);
});
}
}

describe('Client Initialization Testing', () => {
describe('Request', () => {
let Request: typeof ds.DatastoreRequest;
let request: Any;

/**
* This function is called by a test to ensure that the rest parameter
* gets passed to the Gapic data client's constructor
*
* @param {DatastoreRequest} [request] The request object whose data client
* the test makes comparisons with.
* @param {string | undefined} [expectedFallback] The value that the test
* expects the rest parameter of the data client to be equal to.
* @param {mocha.Done} [done] The done function used for indicating
* that the test is complete or that there is an error in the mocha test
* environment.
*
*/
function compareRequest(
request: DatastoreRequest,
expectedFallback: Fallback,
done: mocha.Done
) {
try {
const client = request.datastore.clients_.get(clientName);
assert(client);
assert.strictEqual(client.restParameter, expectedFallback);
done();
} catch (err: unknown) {
done(err);
}
}
async.each(
[
{
options: {fallback: 'rest' as Fallback},
expectedFallback: 'rest',
description: 'when specifying rest as a fallback parameter',
},
{
options: {},
expectedFallback: undefined,
description: 'when specifying no fallback parameter',
},
],
(testParameters: {
options: DatastoreOptions;
expectedFallback: Fallback;
description: string;
}) => {
describe(testParameters.description, () => {
beforeEach(() => {
Request = proxyquire('../../src/request', {
'./v1': {
DatastoreClient: FakeDatastoreClient,
},
}).DatastoreRequest;
request = new Request();
request.datastore = new Datastore(testParameters.options);
// The CI environment can't fetch project id so the function that
// fetches the project id needs to be mocked out.
request.datastore.auth.getProjectId = (
callback: (err: any, projectId: string) => void
) => {
callback(null, 'some-project-id');
};
});
it('should set the rest parameter in the data client when calling prepareGaxRequest_', done => {
// This request does lazy initialization of the gapic layer Datastore client.
request.prepareGaxRequest_(
{client: clientName, method: 'lookup'},
() => {
compareRequest(request, testParameters.expectedFallback, done);
}
);
});
it('should set the rest parameter in the data client when calling request_', done => {
// This request does lazy initialization of the gapic layer Datastore client.
request.request_({client: clientName, method: 'lookup'}, () => {
compareRequest(request, testParameters.expectedFallback, done);
});
});
});
}
);
});
});