New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mapping of large numbers fails when using MongoDB #192

Closed
linderl opened this Issue Aug 24, 2015 · 21 comments

Comments

Projects
None yet
6 participants
@linderl

linderl commented Aug 24, 2015

When using large numbers as id's for entity objects you get this exception:

com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected NUMBER but was BEGIN_OBJECT
    at com.google.gson.Gson.fromJson(Gson.java:822)
    at com.google.gson.Gson.fromJson(Gson.java:875)
    at com.google.gson.Gson$1.deserialize(Gson.java:128)
    at org.javers.core.json.typeadapter.commit.GlobalIdTypeAdapter.parseInstanceId(GlobalIdTypeAdapter.java:67)
    at org.javers.core.json.typeadapter.commit.GlobalIdTypeAdapter.fromJson(GlobalIdTypeAdapter.java:42)
    at org.javers.core.json.typeadapter.commit.GlobalIdTypeAdapter.fromJson(GlobalIdTypeAdapter.java:18)
    at org.javers.core.json.JsonConverterBuilder$2.deserialize(JsonConverterBuilder.java:139)
    at com.google.gson.TreeTypeAdapter.read(TreeTypeAdapter.java:58)
    at com.google.gson.Gson.fromJson(Gson.java:810)
    at com.google.gson.Gson.fromJson(Gson.java:875)
    at com.google.gson.Gson$1.deserialize(Gson.java:128)
    at org.javers.core.json.typeadapter.commit.CdoSnapshotTypeAdapter.fromJson(CdoSnapshotTypeAdapter.java:48)
    at org.javers.core.json.typeadapter.commit.CdoSnapshotTypeAdapter.fromJson(CdoSnapshotTypeAdapter.java:22)
    at org.javers.core.json.JsonConverterBuilder$2.deserialize(JsonConverterBuilder.java:139)
    at com.google.gson.TreeTypeAdapter.read(TreeTypeAdapter.java:58)
    at com.google.gson.Gson.fromJson(Gson.java:810)
    at com.google.gson.Gson.fromJson(Gson.java:775)
    at com.google.gson.Gson.fromJson(Gson.java:724)
    at com.google.gson.Gson.fromJson(Gson.java:696)
    at org.javers.core.json.JsonConverter.fromJson(JsonConverter.java:78)
    at org.javers.repository.mongo.MongoRepository.readFromDBObject(MongoRepository.java:184)
    at org.javers.repository.mongo.MongoRepository.getLatest(MongoRepository.java:241)
    at org.javers.repository.mongo.MongoRepository.getLatest(MongoRepository.java:79)
    at org.javers.repository.api.JaversExtendedRepository.getLatest(JaversExtendedRepository.java:100)
    at org.javers.core.snapshot.GraphShadowFactory.createLatestShadow(GraphShadowFactory.java:32)
    at org.javers.core.snapshot.GraphSnapshotFacade.createLatestShadow(GraphSnapshotFacade.java:22)
    at org.javers.core.commit.CommitFactory.create(CommitFactory.java:76)
    at org.javers.core.JaversCore.commit(JaversCore.java:60)
    at com.nasdaq.xcsd.audit.logger.exploratory.JaVersMongoDBTest.verifyMappingOfLargerId(JaVersMongoDBTest.java:46)

Here is the unit test that triggers the bugg:

public class JaVersMongoDBTest {
    public static final long ID_ONE_BILLION = 1000000000L;
    public static final long ID_ONE_TRILLION = 1000000000L*1000;

    private Javers javers;
    @Before
    public void setup() {
        MongoDatabase mongoDb = new MongoClient( "localhost" ).getDatabase("test");

        MongoRepository mongoRepo = new MongoRepository(mongoDb);
        javers = JaversBuilder.javers().registerJaversRepository(mongoRepo).build();
    }

    @Builder
    public static class MyEntity {
        @Id
        final private Long id;
        final private String name;
    }
    @Test
    // This test passes
    public void verifyMappingOfLargeId() {
        javers.commit("kent", MyEntity.builder().id(ID_ONE_BILLION).name("red").build());
        javers.commit("kent", MyEntity.builder().id(ID_ONE_BILLION).name("blue").build());
    }
    @Test
    // This test fails
    public void verifyMappingOfLargerId() {
        javers.commit("kent", MyEntity.builder().id(ID_ONE_TRILLION).name("red").build());
        javers.commit("kent", MyEntity.builder().id(ID_ONE_TRILLION).name("blue").build());
    }
}
@bartoszwalacik

This comment has been minimized.

Member

bartoszwalacik commented Aug 24, 2015

cant reproduce it
this test passes with JaversMongoRepository

    def "should support long number as Entity Id "(){
        given:
        def javers = javers().build()
        def longId = 1000000000L*1000
        def category = new Category(longId)

        when:
        javers.commit("author",category)

        then:
        javers.getLatestSnapshot(longId, Category).get().globalId.cdoId == longId
    }

Which version of gson and mongo you have in runtime?
javers has:
compile 'com.google.code.gson:gson:2.2.4'
compile 'org.mongodb:mongo-java-driver:3.0.1'

@linderl

This comment has been minimized.

linderl commented Aug 25, 2015

com.google.code.gson:gson:2.3.1
org.mongodb:mongo-java-driver:3.0.1

@bartoszwalacik

This comment has been minimized.

Member

bartoszwalacik commented Aug 25, 2015

still, i cant reproduce it,
please submit your runnable, failing test to github, could be to forked javers repository

@linderl

This comment has been minimized.

linderl commented Aug 25, 2015

I tried with gson 2.2.4 but the test still fails. We use MongoDB 3.0.5 on windows.
Server info:

"buildinfo" : {
    "version" : "3.0.5",
    "gitVersion" : "8bc4ae20708dbb493cb09338d9e7be6698e4a3a3",
    "targetMinOS" : "Windows 7/Windows Server 2008 R2",
    "OpenSSLVersion" : "OpenSSL 1.0.1p-fips 9 Jul 2015",
    "sysInfo" : "windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1') BOOST_LIB_VERSION=1_49",
    "versionArray" : [3, 0, 5, 0],
    "loaderFlags" : "",
    "compilerFlags" : "/TP /nologo /EHsc /W3 /wd4355 /wd4800 /wd4267 /wd4244 /wd4290 /wd4068 /wd4351 /we4099 /we4930 /Z7 /errorReport:none /MD /O2 /Oy- /Gw /Gy /Zc:inline",
    "allocator" : "tcmalloc",
    "javascriptEngine" : "V8",
    "bits" : 64,
    "debug" : false,
    "maxBsonObjectSize" : 16777216
  }
@bartoszwalacik

This comment has been minimized.

Member

bartoszwalacik commented Aug 25, 2015

I need to reproduce the error on my laptop.

I've pushed new test ("should support long number as Entity Id") for this case to master
https://github.com/javers/javers/blob/8d6f26bcfadb4044b9cf6591c0e936798ece661e/javers-core/src/test/groovy/org/javers/core/JaversRepositoryE2ETest.groovy
but it passes for all repositories including Mongo.
So push your test somewhere, I need to run it.

@linderl

This comment has been minimized.

linderl commented Aug 25, 2015

OK. I'll create a fork and push it there.

@bartoszwalacik

This comment has been minimized.

Member

bartoszwalacik commented Aug 25, 2015

right, we need to proof that it isn't local enviroment issue

@linderl

This comment has been minimized.

linderl commented Aug 26, 2015

I have a nasty firewall here at work, so I cannot download the dependencies specified in the build file. I have to try at home instead.

@bartoszwalacik

This comment has been minimized.

Member

bartoszwalacik commented Sep 21, 2015

any progress with reproducing this issue?

@bartoszwalacik

This comment has been minimized.

Member

bartoszwalacik commented Sep 30, 2015

closing due to lack of activity, cant reproduce it

@rudgirello

This comment has been minimized.

rudgirello commented Oct 14, 2015

Observing the same issue with mongoDB 3.0.6 running on Debian. Using mongo drivers 3.0.1 to 3.1.0 and latest javers.

@bartoszwalacik

This comment has been minimized.

Member

bartoszwalacik commented Oct 16, 2015

I need a failing test with reproduction of this issue, could you provide it?
test could be pushed to your forked javers repository

jlsalmon added a commit to jlsalmon/javers that referenced this issue Nov 25, 2015

@jlsalmon

This comment has been minimized.

Contributor

jlsalmon commented Nov 25, 2015

I am also seeing this issue with gson:2.3.1 and mongo-java-driver:3.0.1.

I took the liberty of adding @linderl 's test case: jlsalmon@88fa048

It fails for me too.

@bartoszwalacik

This comment has been minimized.

Member

bartoszwalacik commented Nov 25, 2015

Guys, if you can reproduce it, think about contributing a Pull Request with a fix.

@pszymczyk

This comment has been minimized.

Member

pszymczyk commented Dec 9, 2015

I discovered that mongo persists different json documents:

  • for 1000000000L * 1000 it's:
{"entity":"org.javers.repository.mongo.Issue123$MyEntity","cdoId":{"$numberLong":"1000000000000"}}
  • for ID_ONE_BILLION = 1000000000L it's:
{"entity":"org.javers.repository.mongo.Issue123$MyEntity","cdoId":1000000000}

and when Gson tries to deserialize the first object it expects number but there is:

{"$numberLong":"1000000000000"}

and that's why it finished with

 java.lang.IllegalStateException: Expected NUMBER but was BEGIN_OBJECT

I will try to go deeper and solve this problem :)

@bartoszwalacik

This comment has been minimized.

Member

bartoszwalacik commented Dec 10, 2015

@pawelszymczyk nice, we are waiting for PR 👍

@pszymczyk

This comment has been minimized.

Member

pszymczyk commented Dec 10, 2015

@bartoszwalacik and here it is #266 :)

@bartoszwalacik

This comment has been minimized.

Member

bartoszwalacik commented Dec 11, 2015

Bug is fixed in version 1.4.0-RC2 http://search.maven.org/#artifactdetails|org.javers|javers-core|1.4.0-RC2|jar
Check it out and give us feedback, does it solve your problem?

@arnzel

This comment has been minimized.

arnzel commented Jan 29, 2016

I have a similiar problem with the class "java.util.UUID". The property mostSigBits is like "{"$numberLong":"453454......"}" but Gson expects a number.

@pszymczyk pszymczyk reopened this Jan 29, 2016

@bartoszwalacik

This comment has been minimized.

Member

bartoszwalacik commented Jan 29, 2016

think it would be better to implement JsonConverter for UUID and to store them as
Strings, for example: 067e6162-3b6f-4ae2-a171-2470b63dff00

@arnzel could you create a separate issue for that?

bartoszwalacik added a commit that referenced this issue Jan 30, 2016

#192
Support for java util types: UUID, File, Currency, ...
@bartoszwalacik

This comment has been minimized.

Member

bartoszwalacik commented Feb 2, 2016

fix released in JaVers 1.4.10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment