Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skipping floats that cannot be marshalled #5163

Merged
merged 12 commits into from Apr 14, 2020
Merged

Skipping floats that cannot be marshalled #5163

merged 12 commits into from Apr 14, 2020

Conversation

gja
Copy link
Contributor

@gja gja commented Apr 10, 2020

Explicitly checking for +Inf / -Inf / NaN when serializing floating points. This is the same thing go does internally.

Open Questions:

  • I explictly cast to a float64. Should I be checking for float32?
  • In an earlier version of this PR, I used to use json.Mashall(v.Value), which worked fine. However, the serializer spat out more accurate numbers than expected, so tests failed. Is this incorrect? @manishrjain

This change is Reviewable

Docs Preview: Dgraph Preview

@gja gja requested a review from pawanrawal as a code owner April 10, 2020 11:00
@gja gja changed the title Skipping floats that cannot be marshalled [DGRAPH-1186] Skipping floats that cannot be marshalled Apr 10, 2020
@gja gja requested a review from manishrjain April 10, 2020 11:03
@gja gja marked this pull request as draft April 10, 2020 11:45
Copy link
Contributor Author

@gja gja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 1 files reviewed, 1 unresolved discussion (waiting on @manishrjain and @pawanrawal)


query/outputnode.go, line 257 at r1 (raw file):

		// +Inf, -Inf and NaN are not representable in JSON.
		// Please see https://golang.org/src/encoding/json/encode.go?s=6458:6501#L573
		if math.IsInf(v.Value.(float64), 0) || math.IsNaN(v.Value.(float64)) {

Am I casting this the wrong way? This is how golang casts internally https://golang.org/src/reflect/value.go?s=27933:27963#L890

Copy link
Contributor

@manishrjain manishrjain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good from my end. Get it approved by @pawanrawal .

Reviewed 1 of 1 files at r1.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @gja and @pawanrawal)


query/outputnode.go, line 257 at r1 (raw file):

Previously, gja (Tejas Dinkar) wrote…

Am I casting this the wrong way? This is how golang casts internally https://golang.org/src/reflect/value.go?s=27933:27963#L890

Add some tests, approach looks good.

Copy link
Contributor

@manishrjain manishrjain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, can you remove the JIRA issue number from the top line. It's not something that our open source users can track. You can mention it in the description of the PR.

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @gja and @pawanrawal)

Copy link
Contributor

@martinmr martinmr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One comment but otherwise it :lgtm:

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @gja and @pawanrawal)


query/outputnode.go, line 257 at r1 (raw file):

v.Value.(float64)

Doing it this way, could trigger a panic if for some reason the value says it's type is a float but the actual value is not one. Not super likely I guess but better to be proactive.

The fix is easy, see https://tour.golang.org/methods/15.

@@ -337,77 +343,55 @@ func (fj *fastJsonNode) encode(out *bytes.Buffer) error {
a.order = i
}

// Scalar Value
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please hide whitespace when going through this change.

@gja gja marked this pull request as ready for review April 13, 2020 10:51
@gja gja requested a review from a team as a code owner April 13, 2020 10:51
query/query.go Outdated
@@ -216,6 +216,26 @@ type Function struct {

// SubGraph is the way to represent data. It contains both the request parameters and the response.
// Once generated, this can then be encoded to other client convenient formats, like GraphQL / JSON.
// SubGraphs are recursively nested. Each SubGraph contain the following:
// * SrcUIDS: A list of UIDs that were processed by this query. If this subgraph is a child graph, then the

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line is 107 characters (from lll)

// DestUIDs of the parent must match the SrcUIDs of the childe.
// * DestUIDs: A list of UIDs for which there can be output found in the Children field
// * Children: A list of child results for this query
// * valueMatrix: A list of values, against a single attribute, such as name (for a scalar subgraph).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line is 101 characters (from lll)

// * valueMatrix: A list of values, against a single attribute, such as name (for a scalar subgraph).
// This must be the same length as the SrcUIDs
// * uidMatrix: A list of outgoing edges. This must be same length as the SrcUIDs list.
// Example, say we are creating a SubGraph for a query "users", which returns one user with name 'Foo', you may get

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line is 115 characters (from lll)

@gja gja changed the title [DGRAPH-1186] Skipping floats that cannot be marshalled Skipping floats that cannot be marshalled Apr 13, 2020
Copy link
Contributor

@pawanrawal pawanrawal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: nice change

Reviewed 4 of 4 files at r2.
Reviewable status: all files reviewed, 8 unresolved discussions (waiting on @gja)


query/query.go, line 220 at r2 (raw file):

// Once generated, this can then be encoded to other client convenient formats, like GraphQL / JSON.
// SubGraphs are recursively nested. Each SubGraph contain the following:
// * SrcUIDS: A list of UIDs that were processed by this query. If this subgraph is a child graph, then the

UIDs that were given as input to this query


query/query.go, line 221 at r2 (raw file):

// SubGraphs are recursively nested. Each SubGraph contain the following:
// * SrcUIDS: A list of UIDs that were processed by this query. If this subgraph is a child graph, then the
//            DestUIDs of the parent must match the SrcUIDs of the childe.

children

Copy link
Contributor

@ashish-goswami ashish-goswami left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 8 unresolved discussions (waiting on @gja and @martinmr)


query/outputnode.go, line 257 at r1 (raw file):

Previously, martinmr (Martin Martinez Rivera) wrote…
v.Value.(float64)

Doing it this way, could trigger a panic if for some reason the value says it's type is a float but the actual value is not one. Not super likely I guess but better to be proactive.

The fix is easy, see https://tour.golang.org/methods/15.

Yes, I agree here. Other way to achieve this would be a type switch as we have done for types.IntID.

@@ -216,6 +216,26 @@ type Function struct {

// SubGraph is the way to represent data. It contains both the request parameters and the response.
// Once generated, this can then be encoded to other client convenient formats, like GraphQL / JSON.
// SubGraphs are recursively nested. Each SubGraph contain the following:
// * SrcUIDS: A list of UIDs that were sent to this query. If this subgraph is a child graph, then the

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line is 102 characters (from lll)

@gja gja merged commit d4cdf5e into master Apr 14, 2020
@gja gja deleted the tejas/skip-floats branch April 14, 2020 09:27
ashish-goswami pushed a commit that referenced this pull request Apr 22, 2020
dna2github pushed a commit to dna2fork/dgraph that referenced this pull request Jul 18, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants