Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix!: don't unwrap int, double by default, add a warning when a number is autoconverted to double or int #773

Closed
wants to merge 18 commits into from

Conversation

crwilcox
Copy link
Contributor

@crwilcox crwilcox commented Dec 8, 2020

BREAKING CHANGE: Datastore doubles and ints are no longer automatically decoded to Javascript Numbers
BREAKING CHANGE: a warning is emitted whenever autoconversion from a Javascript Number to a Datastore Double or Int is used.

As the autoconversion from number to int/double can result in inconsistent db representations, this warns to not use this and that there is some danger to doing so.

Take the double value 4.0. While in Node.js this is consider to be an int, a user specifying 4.0 likely intends for that to be a Datastore Double. The automatic conversion can result in mixed type columns which can make querying entities and their ordering odd, as at this time Datastore entities are ordered by type, then value.

Fixes #759, #774, #760 🦕

@crwilcox crwilcox requested a review from a team as a code owner December 8, 2020 18:56
@product-auto-label product-auto-label bot added the api: datastore Issues related to the googleapis/nodejs-datastore API. label Dec 8, 2020
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Dec 8, 2020
src/entity.ts Outdated Show resolved Hide resolved
if (Number.isInteger(value)) {
if (!Number.isSafeInteger(value)) {
process.emitWarning(integerOutOfBoundsWarning);
} else {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the else here ?
I don't think the branches are exclusive.

Also maybe move process.emitWarning(typeCastWarning); right after l. 594 as soon as we know that the type is a number ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are exclusive. They both recommend the same outcome, but I think having two separate warnings is a bit confusing. This one warning is it could be dangerous, the other is destructive. I had what you recommended earlier but it is very very noisy :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: added 'debouncing' you will only get a single typecast warning for the entity. that way it doesn't flood the warning feed.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be reasonable to augment the out of bounds warning to mention the type issue as well, since that'll still be an issue if they fix their number size issue.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're debouncing the typescast warning any ways, I'd consider just dropping the if and printing both.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think printing both could get noisy as Chris said. I ended up going with @feywind's suggestion here.

src/entity.ts Outdated Show resolved Hide resolved
@vicb
Copy link
Contributor

vicb commented Dec 8, 2020

@crwilcox I have added a few inline comments.

A few questions:

  1. Could you please confirm that this code is called from both query and save ?
  2. What about an option to silence the "Number" warning ? The rationale would be that the warning will start trigger as soon as users save (and query ?) an integer value with the updated code. I think giving them an option to silence the warning while they update their code would be nice ?

@crwilcox
Copy link
Contributor Author

crwilcox commented Dec 8, 2020

  • Could you please confirm that this code is called from both query and save ?

Correct.

What about an option to silence the "Number" warning ? The rationale would be that the warning will start trigger as soon as users save (and query ?) an integer value with the updated code. I think giving them an option to silence the warning while they update their code would be nice ?

I don't think I am in favor of allowing silencing a warning. The warning exists because this is unsafe. This is already the softer approach to discontinuing this altogether (see the first commit).

@crwilcox crwilcox requested a review from bcoe December 8, 2020 20:22
@vicb
Copy link
Contributor

vicb commented Dec 8, 2020

I don't think I am in favor of allowing silencing a warning. The warning exists because this is unsafe. This is already the softer approach to discontinuing this altogether (see the first commit).

Datastore.UNSAFE_CODE_MUST_BE_UPDATED_silence_number_warning = true ?

src/entity.ts Outdated Show resolved Hide resolved
src/entity.ts Show resolved Hide resolved
test/entity.ts Outdated Show resolved Hide resolved
@vicb
Copy link
Contributor

vicb commented Dec 8, 2020

I was wondering about the meaning of "!" in the commit message type.

Just looked that up. "!" is for breaking change.

I don't think it applies here ?

@codecov
Copy link

codecov bot commented Dec 8, 2020

Codecov Report

Merging #773 (beab398) into master (cf31ede) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #773   +/-   ##
=======================================
  Coverage   98.92%   98.93%           
=======================================
  Files          11       11           
  Lines        7921     7945   +24     
  Branches      471      473    +2     
=======================================
+ Hits         7836     7860   +24     
  Misses         83       83           
  Partials        2        2           
Impacted Files Coverage Δ
src/entity.ts 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cf31ede...b28df5f. Read the comment docs.

@vicb
Copy link
Contributor

vicb commented Dec 8, 2020

What about:

  1. Updating Add warning when inference used for Number types to Datastore types #759 with a rationale about why it has been re-opened ?
  2. Not merging this PR until there is migration path for old DB as discussed in Don't unwrap automatically for Number to Datastore Types #760 ?
  3. Not merging this PR until there is some docs detailing the issue ?

More context about 2: if users fix their code now, they will no get any warning any more but they DB might still be in a "bad state" until they fix it using a solution to be provided.

@vicb
Copy link
Contributor

vicb commented Dec 9, 2020

Thinking more about this PR, I really think it would be premature to merge the code as is.

  1. Inconsistent API

Imagine the case:

// Retrieve an entity with *any* of the ds methods.
const [entity] = await ds.get(id);

// [...]

// This would warn because the numbers are not wrapped.
await ds.save(entity);
  1. No migration path

  2. No doc explaining the issue

A more logical way to move forward would be:

  1. Document the issue,
  2. Propose/Discuss a few different solutions,
  3. Implement whatever seems a solid and future proof solution.

To start with may be add some comments in the PR description about when this is intended to be merged ?

@crwilcox crwilcox changed the title fix!:add a warning when a number is autoconverted to double or int 8a5aba8 fix!:add a warning when a number is autoconverted to double or int Dec 9, 2020
@crwilcox crwilcox closed this Dec 9, 2020
@crwilcox crwilcox reopened this Dec 9, 2020
src/entity.ts Outdated Show resolved Hide resolved
@crwilcox
Copy link
Contributor Author

1d80aea is further breaking, users who weren't specifying that we wrapNumbers, will now have it happen as the default was previously false. This change will break a great deal of tests, which I have not addressed yet.

This would fix #774 and allow us to improve the behavior of the above system test so that double/int values weren't so easily converted behind-the-scenes.

@crwilcox
Copy link
Contributor Author

crwilcox commented Dec 11, 2020

Here is a modified version of that system test (committing after) with the behavior if we reverse wrapNumbers defaults

    it('should save/get an int-able double value via Datastore.', async () => {
      const postKey = datastore.key('Team');
      const points = Datastore.double(2);
      await datastore.save({ key: postKey, data: { points } });
      let [entity] = await datastore.get(postKey, { wrapNumbers: true });
      // Expect content is stored in datastore as a double with wrapping to 
      // return a wrapped double
      assert.strictEqual(entity.points.type, 'DatastoreDouble');
      assert.strictEqual(entity.points.value, 2);

      [entity] = await datastore.get(postKey, { wrapNumbers: false });
      // Verify that when requested with wrapNumbers false, we get a plain
      // javascript Number 2.
      assert.strictEqual(entity.points, 2);

      [entity] = await datastore.get(postKey);
      // Expect without any options, a wrapped double to be returned.
      assert.strictEqual(entity.points.type, 'DatastoreDouble');
      assert.strictEqual(entity.points.value, 2);

      // Save the data again, reget, ensuring that along the way it isn't 
      // somehow changed to another numeric type.
      await datastore.save(entity);
      [entity] = await datastore.get(postKey);
      // expect as we saved, that this property is still a DatastoreDouble.
      assert.strictEqual(entity.points.type, 'DatastoreDouble');
      assert.strictEqual(entity.points.value, 2);

      await datastore.delete(postKey);
    });

@vicb
Copy link
Contributor

vicb commented Dec 11, 2020

Thanks for clarifying Christopher.

Regarding this PR and addressing the issue of autoconversion causing friction, it seems to move this forward unwrapping of datastore types to numbers need to be have a reversed default. Currently we support specifying of wrapNumbers as part of ints, but by default we unbox them into Numbers. I think leaving support to do that is fine, but we likely want to opt to not unbox. Datastore.int and Datastore.double have the Number interface anyway, so there should be minimal disruption.

Again I think that creating a document to discuss about the different solutions with their pros and cons, the impact for current users, and the impact for Google would help a lot here. And also I'd like to see in the document the option to fix this server side along with other solutions that all seem to require more changes by the customers.

@stephenplusplus
Copy link
Contributor

To match Int, should the custom Double class extend Number as well? Or in any case, a quick helper for some situations would be to add a valueOf() method to the custom Double class prototype, so that this would work:

assert.strictEqual(entity.points.type, 'DatastoreDouble');
assert.strictEqual(entity.points.value, 2);
+ assert.strictEqual(entity.points + 1, 3);

@crwilcox
Copy link
Contributor Author

crwilcox commented Dec 11, 2020

@stephenplusplus 100%. There is some cleanup baggage if we go with this, though I didn't want to sink time into it until we are mostly sure the 'feel' of this would be right. I do think the source of our problems are in the automatic decoding of things as that then results in loss of type metadata for later operations. Locally added a check to make sure this happens :)

@vicb Alterations to the ordering of objects in datastore is out of scope of the library. I also don't currently know of a plan to interleave double and ints. It may be worth opening an issue specifically about that in relation to the Datastore service.

Given that, I am focusing on the client library. In the near term, changes to the client library are likely more tractable to improving the end-user experience. Given that the number interface can be maintained, but type metadata as well if we don't auto unwrap, we can make this work much better I think :)

@crwilcox
Copy link
Contributor Author

@stephenplusplus I extended Double. Tests will very much need fixing if we go this route, though the user interface should be there.

@crwilcox crwilcox changed the title fix!: add a warning when a number is autoconverted to double or int fix!: don't unwrap int, double by default, add a warning when a number is autoconverted to double or int Dec 11, 2020
@crwilcox
Copy link
Contributor Author

crwilcox commented Dec 12, 2020

Tests have been updated to match behavior. Samples will still need updating from here.

type: string;
value: number;
constructor(value: number) {
constructor(value: number | string | ValueProto) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are Double / Int included in this type ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They ought to be, via the Number type. Though that isn't really a use path. Support for that could be added, but as you might detect a trend, this is a mirroring of the existing style we went with for Ints.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a case where we'd want to construct a Number from another Number? If not, I'd start with accepting a string, number, and potentially ValueProto (as we are).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like there is never a common case where we would want to construct a Double from a Double / Int. Constructing a Double from a Double is just creating a copy and making a Double from an Int could encourage users to misuse the API by inadvertently changing the column type.

type: string;
value: number;
constructor(value: number) {
constructor(value: number | string | ValueProto) {
super(typeof value === 'object' ? value.doubleValue : value);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the use case of constructing from a proto ?
If it needs to be supported int would probably make sense too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Int has the same support from proto, and it is part of the entityFromEntityProto path. This code mirrors the behavior of our Int class.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering the same thing, the fact that we allow this in the Int type seems like good motivation 👍 Let's just make sure we have a test for it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a bunch more tests to Double here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests and all other changes will be in a separate PR.

/**
* @name Double#type
* @type {string}
*/
this.type = 'DatastoreDouble';

this._entityPropertyName =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need to keep the property name ?
If yes it would be good to comment why.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, I am not sure what this is for, but I mirrored behavior in the Int implementation. Maybe @stephenplusplus recalls?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're not 100% what it's here for, I'd be tempted to drop it (let's chat with @stephenplusplus offline).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't checked the use cases but it seems dangerous

if an entity has the form {oldValue: Int, newValue: Int} and you do (with wrapped Number):

const [entity] = await ds.get(...);

entity.oldValue = entity.newValue;
entity.newValue = ds.Int(123);

await ds.save(entity);

I can only imagine that keeping the entity property name would cause problems as oldValue would have a reference to "newValue".

I am not sure if it is actually a problem in the current code but at least it makes it easy to shoot yourself in the foot.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll remove this. The reason we have this for Int is because we use it when using the valueOf() function.

/**
* @name Double#value
* @type {number}
*/
this.value = value;
this.value =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do need a value member ?
It seems like it could be dropped as well as valueOf which are both already in the super class.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly? This started as a string, but realized I could store this as a number. We store it as a string for Int mostly due to Int range differences.

Datastore Doubles have 64-bit double precision, IEEE 754, as do Node.js Numbers. So I was able to change this type and simplify a bit.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

feels kind of like this should be private.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know about that. If we make it private then that would be a breaking change. Consider the case where users are using .value in various places for these types. In the Int class this is private.


// eslint-disable-next-line @typescript-eslint/no-explicit-any
valueOf(): any {
return Number(this.value);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Number is redundant with l 83.
But is seems like valueOf should be dropped altogether as you extend Number ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Playing around with extending Number, there isn't access to a this.value unless we store the value ourselves:

class F extends Number {
  constructor(value) {
    super(value)
  }
  blerg() {
    console.info(this.value)
  }
}

const f = new F(99);
console.info(f.valueOf())
f.blerg()

outputs:

99
undefined

Part of me wonders if we should avoid extending the base Number type, and instead create a DatastoreNumber type, since Number implies a a larger interface than we potentially want to support out of the gate?

Edit: I see we're already extending Number in Int, I'd stick with this approach.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

class F extends Number {
  constructor(value) {
    super(value);
  }
  blerg() {
    console.info(Number(this));
  }
}

const f = new F(99);
f.blerg(); // outputs 99 as expected

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to know.

}

toJSON(): any {
return {type: this.type, value: this.value};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems quite dangerous if the goal of Double is to fake a Number ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you expand? Sorry, I don't quite understand/follow. Also, class Int has a similar toJson.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am really not sure to get the intent of the change right but my understanding of the current state is that you want to return Ints and Doubles that users would use as if they were Numbers.

If I get this right then Ints and Doubles should behave like Numbers as much a possible.

However with this code:

  • JSON.stringify(1.23) == "1.23"
  • but JSON.stringify(Double(1.23)) == '{"type": ..., "value": ...}'

While not related to toJSON() returning wrapped numbers (Int / Double) could be problematic:

const [entity] = await ds.get(...);

// entity.price is a Double, i.e. Double(10)

entity.price *= 0.90;

// entity.price is now a number (9, "int")

// warning + DB screwed
await ds.save(entity);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that returning wrapped numbers in Int / Double format is necessary in order to avoid the problem where we change the column type. I see how adding toJson changes the output of JSON.stringify which is technically a breaking change, but since Int currently has toJson implemented and Double / Int are likely being returned interchangably, it is worth the tradeoff to add toJSON here to ensure JSON.stringify will handle each type the same way.

Copy link
Contributor

@vicb vicb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some details on my former brief review.

But at this point I am not sure any more sure of how exactly you are trying to solve the number thing. It might not be applicable ?

type: string;
value: number;
constructor(value: number) {
constructor(value: number | string | ValueProto) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}

toJSON(): any {
return {type: this.type, value: this.value};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am really not sure to get the intent of the change right but my understanding of the current state is that you want to return Ints and Doubles that users would use as if they were Numbers.

If I get this right then Ints and Doubles should behave like Numbers as much a possible.

However with this code:

  • JSON.stringify(1.23) == "1.23"
  • but JSON.stringify(Double(1.23)) == '{"type": ..., "value": ...}'

While not related to toJSON() returning wrapped numbers (Int / Double) could be problematic:

const [entity] = await ds.get(...);

// entity.price is a Double, i.e. Double(10)

entity.price *= 0.90;

// entity.price is now a number (9, "int")

// warning + DB screwed
await ds.save(entity);

Copy link
Contributor

@bcoe bcoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mainly left a few nits, I think modeling after class Int is the right approach.

type: string;
value: number;
constructor(value: number) {
constructor(value: number | string | ValueProto) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a case where we'd want to construct a Number from another Number? If not, I'd start with accepting a string, number, and potentially ValueProto (as we are).

type: string;
value: number;
constructor(value: number) {
constructor(value: number | string | ValueProto) {
super(typeof value === 'object' ? value.doubleValue : value);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering the same thing, the fact that we allow this in the Int type seems like good motivation 👍 Let's just make sure we have a test for it.

/**
* @name Double#type
* @type {string}
*/
this.type = 'DatastoreDouble';

this._entityPropertyName =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're not 100% what it's here for, I'd be tempted to drop it (let's chat with @stephenplusplus offline).


// eslint-disable-next-line @typescript-eslint/no-explicit-any
valueOf(): any {
return Number(this.value);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Playing around with extending Number, there isn't access to a this.value unless we store the value ourselves:

class F extends Number {
  constructor(value) {
    super(value)
  }
  blerg() {
    console.info(this.value)
  }
}

const f = new F(99);
console.info(f.valueOf())
f.blerg()

outputs:

99
undefined

Part of me wonders if we should avoid extending the base Number type, and instead create a DatastoreNumber type, since Number implies a a larger interface than we potentially want to support out of the gate?

Edit: I see we're already extending Number in Int, I'd stick with this approach.

if (Number.isInteger(value)) {
if (!Number.isSafeInteger(value)) {
process.emitWarning(integerOutOfBoundsWarning);
} else {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're debouncing the typescast warning any ways, I'd consider just dropping the if and printing both.

@@ -19,6 +19,7 @@ import * as sinon from 'sinon';
import {Datastore} from '../src';
import {Entity} from '../src/entity';
import {IntegerTypeCastOptions} from '../src/query';
import {decode} from 'punycode';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this used anywhere, I think it might just be vscode adding something?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not used. Will delete.

/**
* @name Double#value
* @type {number}
*/
this.value = value;
this.value =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

feels kind of like this should be private.

const postKey = datastore.key('Team');
const points = Datastore.double(2);
await datastore.save({key: postKey, data: {points}});
let [entity] = await datastore.get(postKey, {wrapNumbers: true});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a TODO, I wish we had unit tests for the Int and Double types, vs., only integration tests.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a good unit test corresponding to this one would be to mock out the get and the save and ensure that getting a Double/Int leads to saving the same type.

@bcoe bcoe added the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 2, 2021
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 2, 2021
@meredithslota
Copy link
Contributor

Coming back to this — I see in #773 (comment) a request to create a document describing potential solutions (including server-side solutions) and their pros/cons. Did such a document ever get authored? It feels like perhaps we aren't yet aligned on the path forward yet, but I can't tell if there was a decision reached outside of this thread.

@crwilcox
Copy link
Contributor Author

There isn't much in the way of server side to be done as datastore sorts by name and type, and javascript doesn't distinguish between the different numerical types. The path forward almost certainly will break things for users. I didn't reach out to the datastore backend team about this as I don't believe there is an issue with the way the backend operates.

Copy link
Contributor

@danieljbruce danieljbruce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding comments now I plan to follow up on.

@@ -19,6 +19,7 @@ import * as sinon from 'sinon';
import {Datastore} from '../src';
import {Entity} from '../src/entity';
import {IntegerTypeCastOptions} from '../src/query';
import {decode} from 'punycode';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not used. Will delete.

const postKey = datastore.key('Team');
const points = Datastore.double(2);
await datastore.save({key: postKey, data: {points}});
let [entity] = await datastore.get(postKey, {wrapNumbers: true});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a good unit test corresponding to this one would be to mock out the get and the save and ensure that getting a Double/Int leads to saving the same type.

Copy link
Contributor

@danieljbruce danieljbruce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a bunch of comments for more suggestions.

type: string;
value: number;
constructor(value: number) {
constructor(value: number | string | ValueProto) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like there is never a common case where we would want to construct a Double from a Double / Int. Constructing a Double from a Double is just creating a copy and making a Double from an Int could encourage users to misuse the API by inadvertently changing the column type.

type: string;
value: number;
constructor(value: number) {
constructor(value: number | string | ValueProto) {
super(typeof value === 'object' ? value.doubleValue : value);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a bunch more tests to Double here.

type: string;
value: number;
constructor(value: number) {
constructor(value: number | string | ValueProto) {
super(typeof value === 'object' ? value.doubleValue : value);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests and all other changes will be in a separate PR.

/**
* @name Double#type
* @type {string}
*/
this.type = 'DatastoreDouble';

this._entityPropertyName =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll remove this. The reason we have this for Int is because we use it when using the valueOf() function.

/**
* @name Double#value
* @type {number}
*/
this.value = value;
this.value =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know about that. If we make it private then that would be a breaking change. Consider the case where users are using .value in various places for these types. In the Int class this is private.


// eslint-disable-next-line @typescript-eslint/no-explicit-any
valueOf(): any {
return Number(this.value);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to know.

}

toJSON(): any {
return {type: this.type, value: this.value};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that returning wrapped numbers in Int / Double format is necessary in order to avoid the problem where we change the column type. I see how adding toJson changes the output of JSON.stringify which is technically a breaking change, but since Int currently has toJson implemented and Double / Int are likely being returned interchangably, it is worth the tradeoff to add toJSON here to ensure JSON.stringify will handle each type the same way.

if (Number.isInteger(value)) {
if (!Number.isSafeInteger(value)) {
process.emitWarning(integerOutOfBoundsWarning);
} else {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think printing both could get noisy as Chris said. I ended up going with @feywind's suggestion here.

@danieljbruce
Copy link
Contributor

A continuation of the work done on this PR is contained in https://github.com/googleapis/nodejs-datastore/pull/941/files. Closing for now, but feel free to leave any comments for on the new PR for next steps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: datastore Issues related to the googleapis/nodejs-datastore API. cla: yes This human has signed the Contributor License Agreement. do not merge Indicates a pull request not ready for merge, due to either quality or timing.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add warning when inference used for Number types to Datastore types
8 participants