Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unified date json format for gremlin api and rest api #1014

Merged
merged 6 commits into from
Jun 9, 2020
Merged

Unified date json format for gremlin api and rest api #1014

merged 6 commits into from
Jun 9, 2020

Conversation

javeme
Copy link
Contributor

@javeme javeme commented Jun 2, 2020

fix #1011
Change-Id: Ic47550d22a0b8a857f140c7df1b8352e7d86303a

@codecov
Copy link

codecov bot commented Jun 3, 2020 •
edited

Codecov Report

Merging #1014 into master will decrease coverage by 1.18%.
The diff coverage is 93.33%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #1014      +/-   ##
============================================
- Coverage     69.15%   67.97%   -1.19%     
+ Complexity     5361     5289      -72     
============================================
  Files           328      328              
  Lines         26215    26245      +30     
  Branches       3731     3739       +8     
============================================
- Hits          18129    17840     -289     
- Misses         6327     6660     +333     
+ Partials       1759     1745      -14
Impacted Files Coverage Δ Complexity Δ
...egraph/traversal/optimize/HugeScriptTraversal.java 75.60% <ø> (+3.51%) 4.00 <0.00> (-1.00) ⬆️
...c/main/java/com/baidu/hugegraph/util/JsonUtil.java 70.00% <50.00%> (-7.05%) 12.00 <0.00> (-1.00)
...ava/com/baidu/hugegraph/io/HugeGraphSONModule.java 81.81% <100.00%> (+0.60%) 11.00 <1.00> (+1.00)
...gegraph/backend/store/rocksdb/RocksDBFeatures.java 10.00% <0.00%> (-75.00%) 2.00% <0.00%> (-15.00%)
...egraph/backend/store/mysql/MysqlStoreProvider.java 33.33% <0.00%> (-66.67%) 2.00% <0.00%> (-4.00%)
.../hugegraph/backend/store/rocksdb/RocksDBTable.java 16.15% <0.00%> (-56.16%) 10.00% <0.00%> (-27.00%)
...om/baidu/hugegraph/backend/store/BackendEntry.java 34.28% <0.00%> (-40.00%) 1.00% <0.00%> (ø%)
...aidu/hugegraph/backend/store/mysql/MysqlStore.java 60.91% <0.00%> (-17.77%) 42.00% <0.00%> (-1.00%)
...idu/hugegraph/backend/store/mysql/MysqlTables.java 76.34% <0.00%> (-13.27%) 1.00% <0.00%> (ø%)
...u/hugegraph/backend/store/mysql/MysqlSessions.java 56.16% <0.00%> (-10.96%) 18.00% <0.00%> (-9.00%)
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 073b070...e681772. Read the comment docs.

@javeme
Unified date json format for gremlin api and rest api
b5aba1d 
Change-Id: Ic47550d22a0b8a857f140c7df1b8352e7d86303a
@javeme
remove JsonUtil.DateSerializer
9392dad 
Change-Id: I1c27e74cedadee91e8ead97b094c65926806e175
@javeme
fix: missing separators with writeRaw
5f74c52 
Change-Id: I889d40e04b06a4ac8f68b312569a39b8d185f0f0
ref: FasterXML/jackson-databind#210
@javeme
serialize json of date as string
71cb932 
Change-Id: I3ba4ed2dfc65fa7f002f48b166f9320d29304314
Linary
Linary requested changes Jun 5, 2020
View reviewed changes
hugegraph-core/src/main/java/com/baidu/hugegraph/io/HugeGraphSONModule.java

module.addSerializer(File.class, new FileSerializer());

module.addSerializer(Date.class,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a comment for true and false

@javeme
Copy link
Contributor Author

javeme commented Jun 5, 2020 


2020-06-04 10:14:50 29511 [task-db-worker-1] [ERROR] com.baidu.hugegraph.backend.Transaction [] - Failed to parse entry: TableBackendEntry{Row{type=VERTEX, id=null, columns={ID=L6T, LABEL=-47, PROPERTIES={"-34":"remove_schema","-35":"VERTEX_LABEL:227:language","-36":"com.baidu.hugegraph.job.schema.VertexLabelRemoveCallable","-39":4,"-40":0,"-41":"2020-06-04 10:14:50.707","-42":"2020-06-04 10:14:50.707","-43":0}, EXPIRED_TIME=0}}, sub-rows: []}
com.baidu.hugegraph.HugeException: Can't read json: Cannot deserialize value of type `java.util.Date` from String "2020-06-04 10:14:50.707": not a valid representation (error: Failed to parse Date value '2020-06-04 10:14:50.707': Cannot parse date "2020-06-04 10:14:50.707": while it seems to fit format 'yyyy-MM-dd'T'HH:mm:ss.SSSZ', parsing fails (leniency? null))
 at [Source: (String)""2020-06-04 10:14:50.707""; line: 1, column: 1]
	at com.baidu.hugegraph.util.JsonUtil.fromJson(JsonUtil.java:74) ~[classes/:?]
	at com.baidu.hugegraph.backend.serializer.TableSerializer.readProperty(TableSerializer.java:124) ~[classes/:?]
	at com.baidu.hugegraph.backend.serializer.TableSerializer.parseProperty(TableSerializer.java:99) ~[classes/:?]
	at com.baidu.hugegraph.backend.store.mysql.MysqlSerializer.parseProperties(MysqlSerializer.java:142) ~[classes/:?]
	at com.baidu.hugegraph.backend.serializer.TableSerializer.readVertex(TableSerializer.java:264) ~[classes/:?]
	at com.baidu.hugegraph.backend.tx.GraphTransaction.parseEntry(GraphTransaction.java:1717) ~[classes/:?]
	at com.baidu.hugegraph.iterator.MapperIterator.fetch(MapperIterator.java:44) ~[hugegraph-common-1.7.5.jar:1.7.5.0]
	at com.baidu.hugegraph.iterator.WrappedIterator.hasNext(WrappedIterator.java:40) ~[hugegraph-common-1.7.5.jar:1.7.5.0]
	at com.baidu.hugegraph.iterator.FilterIterator.fetch(FilterIterator.java:42) ~[hugegraph-common-1.7.5.jar:1.7.5.0]
	at com.baidu.hugegraph.iterator.WrappedIterator.hasNext(WrappedIterator.java:40) ~[hugegraph-common-1.7.5.jar:1.7.5.0]
	at com.baidu.hugegraph.backend.query.QueryResults.keepInputOrderIfNeeded(QueryResults.java:101) ~[classes/:?]
	at com.baidu.hugegraph.backend.tx.GraphTransaction.queryVerticesFromBackend(GraphTransaction.java:772) ~[classes/:?]
	at com.baidu.hugegraph.backend.tx.GraphTransaction.queryVertices(GraphTransaction.java:685) ~[classes/:?]
	at com.baidu.hugegraph.backend.tx.GraphTransaction.queryVertices(GraphTransaction.java:649) ~[classes/:?]
	at com.baidu.hugegraph.task.StandardTaskScheduler$TaskTransaction.deleteIndex(StandardTaskScheduler.java:516) ~[classes/:?]
	at com.baidu.hugegraph.task.StandardTaskScheduler.lambda$save$2(StandardTaskScheduler.java:259) ~[classes/:?]
	at com.baidu.hugegraph.task.TaskManager$ContextCallable.call(TaskManager.java:195) [classes/:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_151]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_151]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_151]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
Caused by: org.apache.tinkerpop.shaded.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `java.util.Date` from String "2020-06-04 10:14:50.707": not a valid representation (error: Failed to parse Date value '2020-06-04 10:14:50.707': Cannot parse date "2020-06-04 10:14:50.707": while it seems to fit format 'yyyy-MM-dd'T'HH:mm:ss.SSSZ', parsing fails (leniency? null))
 at [Source: (String)""2020-06-04 10:14:50.707""; line: 1, column: 1]
	at org.apache.tinkerpop.shaded.jackson.databind.exc.InvalidFormatException.from(InvalidFormatException.java:67) ~[gremlin-shaded-3.4.3.jar:3.4.3]
	at org.apache.tinkerpop.shaded.jackson.databind.DeserializationContext.weirdStringException(DeserializationContext.java:1549) ~[gremlin-shaded-3.4.3.jar:3.4.3]
	at org.apache.tinkerpop.shaded.jackson.databind.DeserializationContext.handleWeirdStringValue(DeserializationContext.java:911) ~[gremlin-shaded-3.4.3.jar:3.4.3]
	at org.apache.tinkerpop.shaded.jackson.databind.deser.std.StdDeserializer._parseDate(StdDeserializer.java:524) ~[gremlin-shaded-3.4.3.jar:3.4.3]
	at org.apache.tinkerpop.shaded.jackson.databind.deser.std.StdDeserializer._parseDate(StdDeserializer.java:467) ~[gremlin-shaded-3.4.3.jar:3.4.3]
	at org.apache.tinkerpop.shaded.jackson.databind.deser.std.DateDeserializers$DateBasedDeserializer._parseDate(DateDeserializers.java:195) ~[gremlin-shaded-3.4.3.jar:3.4.3]
	at org.apache.tinkerpop.shaded.jackson.databind.deser.std.DateDeserializers$DateDeserializer.deserialize(DateDeserializers.java:285) ~[gremlin-shaded-3.4.3.jar:3.4.3]
	at org.apache.tinkerpop.shaded.jackson.databind.deser.std.DateDeserializers$DateDeserializer.deserialize(DateDeserializers.java:268) ~[gremlin-shaded-3.4.3.jar:3.4.3]
	at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4014) ~[gremlin-shaded-3.4.3.jar:3.4.3]
	at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3005) ~[gremlin-shaded-3.4.3.jar:3.4.3]
	at com.baidu.hugegraph.util.JsonUtil.fromJson(JsonUtil.java:72) ~[classes/:?]
	... 20 more

@javeme
fix date deserialize
65087c8 
Change-Id: Iccd6c9ac79d5d5deed58c457a71049f0dc608b2e
@javeme
fix unit test
e681772 
Change-Id: Ic52f0090818dcbb9e9e3bc7e0edc634f684433ff
@javeme
Copy link
Contributor Author

javeme commented Jun 8, 2020

Cassandra:

[ERROR] Tests run: 667, Failures: 1, Errors: 0, Skipped: 16, Time elapsed: 369.392 s <<< FAILURE! - in com.baidu.hugegraph.core.CoreTestSuite
[ERROR] testAddEdgeWithTtlAndTtlStartTime(com.baidu.hugegraph.core.EdgeCoreTest)  Time elapsed: 2.545 s  <<< FAILURE!
java.lang.AssertionError
	at com.baidu.hugegraph.core.EdgeCoreTest.testAddEdgeWithTtlAndTtlStartTime(EdgeCoreTest.java:648)

@Linary Linary merged commit 269442f into master Jun 9, 2020
@Linary Linary deleted the date-json-serialize branch June 9, 2020 02:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Linary Linary Linary approved these changes

@zhoney zhoney zhoney approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
3 participants
@javeme @Linary @zhoney