Invalid Json fetched from jv_snapshot.state column oracle sql database

[WojtekPWas]

Clear description of my expectations versus reality

We have got recently an error
com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 3 path $
at com.google.gson.Gson.assertFullConsumption(Gson.java:864) ~[gson-2.8.2.jar!/:na]
at com.google.gson.Gson.fromJson(Gson.java:854) ~[gson-2.8.2.jar!/:na]
at com.google.gson.Gson.fromJson(Gson.java:802) ~[gson-2.8.2.jar!/:na]
at com.google.gson.Gson.fromJson(Gson.java:774) ~[gson-2.8.2.jar!/:na]
at org.javers.core.json.JsonConverter.fromJsonToJsonElement(JsonConverter.java:70) ~[javers-core-3.10.2.jar!/:na]

The reason was in line 22 in org/javers/repository/sql/finders/CdoSnapshotMapper.java (javers-persistence-sql module)
.withSnapshotState(resultSet.getString(SNAPSHOT_STATE))
The method withSnapshotState in CdoSnapshotSerialized class gets string as a parameter. This parameter has a value, which in debugger looks very similar to
https://stackoverflow.com/questions/46198533/java-sql-resultset-getstring-returns-string-with-extra-characters

Contents of column state (of type CLOB) in the table jv_snapshots in the Database (retrieved with SQL Developer tool) looks ok.

Method getString is not the right way to retrieve data from the CLOB column, although it brings correct results in most cases.

The solution was to use getClob method as described in https://stackoverflow.com/questions/19486648/how-to-retrive-the-clob-value-from-oracle-using-java/19486749.

I prepared custom Javers version basen on 3.10.2 and the problem disappeared.

I looked at the current version of Javers 5.14.0. getString(SNAPSHOT_STATE) is still there.
That is why I suggest the issue. Of course I would like to contribute and implement it in the current javers version.

Steps To Reproduce
I have no idea how to produce test case. Problem arise with fetching data from the database. The data is correct. STATE column of problematic row contains many characters, but there were cases with more characters and no error. I suppose the problem has to do with how Oracle Server maintains the data, that data not necessarily occupy continuous space.

Javers' Version
3.10.2
5.14.0 contains invalid method call getString(SNAPSHOT_STATE)

Additional context
The error was reported after 2 years of production use of the system. Table jv_snapshot contains more than 36 milion rows.

In our next project we will use current javers version. I find Javers well designed, easy to use and powerful.

Best regards
Wojtek Wąs

[bartoszwalacik]

Hi Wojtek. Thanks for the good report, please contribute a PR with the fix for the Oracle dialect. Let me know if you need some hints how to start.

[WojtekPWas]

Hi Bartosz,
thanks for the response. I will begin soon. As I understand, I should commit my changes to the new branch named after issue number. I think this fix should apply for all SQL databases and not only for Oracle dialect, but I will look closely at the current version of code.

[bartoszwalacik]

nice, you dont have to create a branch, just fork this repo and create a PR from your fork's master to this master

[WojtekPWas]

I created the PR with two modifications and a small test class. But I could not reproduce my problem. I have tried on the same database and data as before, but this time getString() performed well. Probably it is because of database maintenance. I will try to investigate it farther.

[bartoszwalacik]

342 tests failed :(
https://travis-ci.org/github/javers/javers/builds/762113505

[bartoszwalacik]

you should not change the behavior for all dialects, getString() works perfectly well on most databases. Clob is a contraption of Oracle. second, you have change the read method, what about the write method?

[WojtekPWas]

I have missed integration tests, sorry. Now tests for MySQL and PostgreSQL run on my workstation. I see, that STATE column is of TEXT type for PostgreSQL and MySQL. My solution needs injection of Dialect into CdoSnapshotFinder and SnapshotQuery, which is not very elegant, but I could not find place for additional parametrizing of Servers. You have written "consult the design" in the Gulideline for developers, so I treat my commit as an input for the consultation.

I have not changed anything in write method because it worked fine. In all my error cases the data stored in the database was correct.

Best regards Wojtek

[bartoszwalacik]

fixed in 6.1.0