Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mapping of large numbers fails when using MongoDB #192

Closed
linderl opened this issue Aug 24, 2015 · 21 comments
Closed

Mapping of large numbers fails when using MongoDB #192

linderl opened this issue Aug 24, 2015 · 21 comments
Assignees
Labels

Comments

@linderl
Copy link

@linderl linderl commented Aug 24, 2015

When using large numbers as id's for entity objects you get this exception:

com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected NUMBER but was BEGIN_OBJECT
    at com.google.gson.Gson.fromJson(Gson.java:822)
    at com.google.gson.Gson.fromJson(Gson.java:875)
    at com.google.gson.Gson$1.deserialize(Gson.java:128)
    at org.javers.core.json.typeadapter.commit.GlobalIdTypeAdapter.parseInstanceId(GlobalIdTypeAdapter.java:67)
    at org.javers.core.json.typeadapter.commit.GlobalIdTypeAdapter.fromJson(GlobalIdTypeAdapter.java:42)
    at org.javers.core.json.typeadapter.commit.GlobalIdTypeAdapter.fromJson(GlobalIdTypeAdapter.java:18)
    at org.javers.core.json.JsonConverterBuilder$2.deserialize(JsonConverterBuilder.java:139)
    at com.google.gson.TreeTypeAdapter.read(TreeTypeAdapter.java:58)
    at com.google.gson.Gson.fromJson(Gson.java:810)
    at com.google.gson.Gson.fromJson(Gson.java:875)
    at com.google.gson.Gson$1.deserialize(Gson.java:128)
    at org.javers.core.json.typeadapter.commit.CdoSnapshotTypeAdapter.fromJson(CdoSnapshotTypeAdapter.java:48)
    at org.javers.core.json.typeadapter.commit.CdoSnapshotTypeAdapter.fromJson(CdoSnapshotTypeAdapter.java:22)
    at org.javers.core.json.JsonConverterBuilder$2.deserialize(JsonConverterBuilder.java:139)
    at com.google.gson.TreeTypeAdapter.read(TreeTypeAdapter.java:58)
    at com.google.gson.Gson.fromJson(Gson.java:810)
    at com.google.gson.Gson.fromJson(Gson.java:775)
    at com.google.gson.Gson.fromJson(Gson.java:724)
    at com.google.gson.Gson.fromJson(Gson.java:696)
    at org.javers.core.json.JsonConverter.fromJson(JsonConverter.java:78)
    at org.javers.repository.mongo.MongoRepository.readFromDBObject(MongoRepository.java:184)
    at org.javers.repository.mongo.MongoRepository.getLatest(MongoRepository.java:241)
    at org.javers.repository.mongo.MongoRepository.getLatest(MongoRepository.java:79)
    at org.javers.repository.api.JaversExtendedRepository.getLatest(JaversExtendedRepository.java:100)
    at org.javers.core.snapshot.GraphShadowFactory.createLatestShadow(GraphShadowFactory.java:32)
    at org.javers.core.snapshot.GraphSnapshotFacade.createLatestShadow(GraphSnapshotFacade.java:22)
    at org.javers.core.commit.CommitFactory.create(CommitFactory.java:76)
    at org.javers.core.JaversCore.commit(JaversCore.java:60)
    at com.nasdaq.xcsd.audit.logger.exploratory.JaVersMongoDBTest.verifyMappingOfLargerId(JaVersMongoDBTest.java:46)

Here is the unit test that triggers the bugg:

public class JaVersMongoDBTest {
    public static final long ID_ONE_BILLION = 1000000000L;
    public static final long ID_ONE_TRILLION = 1000000000L*1000;

    private Javers javers;
    @Before
    public void setup() {
        MongoDatabase mongoDb = new MongoClient( "localhost" ).getDatabase("test");

        MongoRepository mongoRepo = new MongoRepository(mongoDb);
        javers = JaversBuilder.javers().registerJaversRepository(mongoRepo).build();
    }

    @Builder
    public static class MyEntity {
        @Id
        final private Long id;
        final private String name;
    }
    @Test
    // This test passes
    public void verifyMappingOfLargeId() {
        javers.commit("kent", MyEntity.builder().id(ID_ONE_BILLION).name("red").build());
        javers.commit("kent", MyEntity.builder().id(ID_ONE_BILLION).name("blue").build());
    }
    @Test
    // This test fails
    public void verifyMappingOfLargerId() {
        javers.commit("kent", MyEntity.builder().id(ID_ONE_TRILLION).name("red").build());
        javers.commit("kent", MyEntity.builder().id(ID_ONE_TRILLION).name("blue").build());
    }
}
@bartoszwalacik
Copy link
Member

@bartoszwalacik bartoszwalacik commented Aug 24, 2015

cant reproduce it
this test passes with JaversMongoRepository

    def "should support long number as Entity Id "(){
        given:
        def javers = javers().build()
        def longId = 1000000000L*1000
        def category = new Category(longId)

        when:
        javers.commit("author",category)

        then:
        javers.getLatestSnapshot(longId, Category).get().globalId.cdoId == longId
    }

Which version of gson and mongo you have in runtime?
javers has:
compile 'com.google.code.gson:gson:2.2.4'
compile 'org.mongodb:mongo-java-driver:3.0.1'

@linderl
Copy link
Author

@linderl linderl commented Aug 25, 2015

com.google.code.gson:gson:2.3.1
org.mongodb:mongo-java-driver:3.0.1

@bartoszwalacik
Copy link
Member

@bartoszwalacik bartoszwalacik commented Aug 25, 2015

still, i cant reproduce it,
please submit your runnable, failing test to github, could be to forked javers repository

@linderl
Copy link
Author

@linderl linderl commented Aug 25, 2015

I tried with gson 2.2.4 but the test still fails. We use MongoDB 3.0.5 on windows.
Server info:

"buildinfo" : {
    "version" : "3.0.5",
    "gitVersion" : "8bc4ae20708dbb493cb09338d9e7be6698e4a3a3",
    "targetMinOS" : "Windows 7/Windows Server 2008 R2",
    "OpenSSLVersion" : "OpenSSL 1.0.1p-fips 9 Jul 2015",
    "sysInfo" : "windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1') BOOST_LIB_VERSION=1_49",
    "versionArray" : [3, 0, 5, 0],
    "loaderFlags" : "",
    "compilerFlags" : "/TP /nologo /EHsc /W3 /wd4355 /wd4800 /wd4267 /wd4244 /wd4290 /wd4068 /wd4351 /we4099 /we4930 /Z7 /errorReport:none /MD /O2 /Oy- /Gw /Gy /Zc:inline",
    "allocator" : "tcmalloc",
    "javascriptEngine" : "V8",
    "bits" : 64,
    "debug" : false,
    "maxBsonObjectSize" : 16777216
  }
@bartoszwalacik
Copy link
Member

@bartoszwalacik bartoszwalacik commented Aug 25, 2015

I need to reproduce the error on my laptop.

I've pushed new test ("should support long number as Entity Id") for this case to master
https://github.com/javers/javers/blob/8d6f26bcfadb4044b9cf6591c0e936798ece661e/javers-core/src/test/groovy/org/javers/core/JaversRepositoryE2ETest.groovy
but it passes for all repositories including Mongo.
So push your test somewhere, I need to run it.

@linderl
Copy link
Author

@linderl linderl commented Aug 25, 2015

OK. I'll create a fork and push it there.

@bartoszwalacik
Copy link
Member

@bartoszwalacik bartoszwalacik commented Aug 25, 2015

right, we need to proof that it isn't local enviroment issue

@linderl
Copy link
Author

@linderl linderl commented Aug 26, 2015

I have a nasty firewall here at work, so I cannot download the dependencies specified in the build file. I have to try at home instead.

@bartoszwalacik
Copy link
Member

@bartoszwalacik bartoszwalacik commented Sep 21, 2015

any progress with reproducing this issue?

@bartoszwalacik
Copy link
Member

@bartoszwalacik bartoszwalacik commented Sep 30, 2015

closing due to lack of activity, cant reproduce it

@rudgirello
Copy link

@rudgirello rudgirello commented Oct 14, 2015

Observing the same issue with mongoDB 3.0.6 running on Debian. Using mongo drivers 3.0.1 to 3.1.0 and latest javers.

@bartoszwalacik
Copy link
Member

@bartoszwalacik bartoszwalacik commented Oct 16, 2015

I need a failing test with reproduction of this issue, could you provide it?
test could be pushed to your forked javers repository

jlsalmon added a commit to jlsalmon/javers that referenced this issue Nov 25, 2015
@jlsalmon
Copy link
Contributor

@jlsalmon jlsalmon commented Nov 25, 2015

I am also seeing this issue with gson:2.3.1 and mongo-java-driver:3.0.1.

I took the liberty of adding @linderl 's test case: jlsalmon@88fa048

It fails for me too.

@bartoszwalacik
Copy link
Member

@bartoszwalacik bartoszwalacik commented Nov 25, 2015

Guys, if you can reproduce it, think about contributing a Pull Request with a fix.

@pszymczyk
Copy link
Member

@pszymczyk pszymczyk commented Dec 9, 2015

I discovered that mongo persists different json documents:

  • for 1000000000L * 1000 it's:
{"entity":"org.javers.repository.mongo.Issue123$MyEntity","cdoId":{"$numberLong":"1000000000000"}}
  • for ID_ONE_BILLION = 1000000000L it's:
{"entity":"org.javers.repository.mongo.Issue123$MyEntity","cdoId":1000000000}

and when Gson tries to deserialize the first object it expects number but there is:

{"$numberLong":"1000000000000"}

and that's why it finished with

 java.lang.IllegalStateException: Expected NUMBER but was BEGIN_OBJECT

I will try to go deeper and solve this problem :)

@bartoszwalacik
Copy link
Member

@bartoszwalacik bartoszwalacik commented Dec 10, 2015

@pawelszymczyk nice, we are waiting for PR 👍

@pszymczyk
Copy link
Member

@pszymczyk pszymczyk commented Dec 10, 2015

@bartoszwalacik and here it is #266 :)

@bartoszwalacik
Copy link
Member

@bartoszwalacik bartoszwalacik commented Dec 11, 2015

Bug is fixed in version 1.4.0-RC2 http://search.maven.org/#artifactdetails|org.javers|javers-core|1.4.0-RC2|jar
Check it out and give us feedback, does it solve your problem?

@arnzel
Copy link

@arnzel arnzel commented Jan 29, 2016

I have a similiar problem with the class "java.util.UUID". The property mostSigBits is like "{"$numberLong":"453454......"}" but Gson expects a number.

@pszymczyk pszymczyk reopened this Jan 29, 2016
@bartoszwalacik
Copy link
Member

@bartoszwalacik bartoszwalacik commented Jan 29, 2016

think it would be better to implement JsonConverter for UUID and to store them as
Strings, for example: 067e6162-3b6f-4ae2-a171-2470b63dff00

@arnzel could you create a separate issue for that?

bartoszwalacik added a commit that referenced this issue Jan 30, 2016
Support for java util types: UUID, File, Currency, ...
@bartoszwalacik
Copy link
Member

@bartoszwalacik bartoszwalacik commented Feb 2, 2016

fix released in JaVers 1.4.10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants