Add local version field in snapshots #294
Conversation
@@ -26,6 +26,7 @@ | |||
private final CdoSnapshotState state; | |||
private final SnapshotType type; | |||
private final List<String> changed; | |||
private final Long version; |
bartoszwalacik
Jan 9, 2016
Member
Think we dont want version to be null, so maybe primitive long?
Think we dont want version to be null, so maybe primitive long?
milanov
Jan 9, 2016
Author
Contributor
Ok, I'll change that. Do you prefer for me to simply commit this change or to squash it with the previous (I didn't find contribution guidelines)?
Ok, I'll change that. Do you prefer for me to simply commit this change or to squash it with the previous (I didn't find contribution guidelines)?
bartoszwalacik
Jan 9, 2016
Member
True, we need to write contribution guidelines. Just push another commit but wait a while, i will take a deeper look at this PR at the evening
True, we need to write contribution guidelines. Just push another commit but wait a while, i will take a deeper look at this PR at the evening
@@ -41,6 +40,7 @@ class CdoSnapshotTypeAdapterTest extends Specification { | |||
json.globalId.entity == "org.javers.core.model.DummyUser" | |||
json.globalId.cdoId == "kaz" | |||
json.type == "INITIAL" | |||
json.version == 1 |
bartoszwalacik
Jan 9, 2016
Member
I suggest to test it in a different (minimal) way.
So test new version field only in 2 basic tests:
def "should serialize CdoSnapshot to Json"()
and
def "should deserialize CdoSnapshot"()
and not to change other tests.
Deserializer should not break if field is missing (and it will be missing if users ugrade to new JaVers version and will load snapshots created in old JaVers version).
I suggest to test it in a different (minimal) way.
So test new version field only in 2 basic tests:
def "should serialize CdoSnapshot to Json"()
and
def "should deserialize CdoSnapshot"()
and not to change other tests.
Deserializer should not break if field is missing (and it will be missing if users ugrade to new JaVers version and will load snapshots created in old JaVers version).
@@ -39,6 +39,7 @@ | |||
public static final String SNAPSHOT_COMMIT_FK = "commit_fk"; | |||
public static final String SNAPSHOT_GLOBAL_ID_FK = "global_id_fk"; | |||
public static final String SNAPSHOT_TYPE = "type"; | |||
public static final String VERSION = "version"; |
bartoszwalacik
Jan 9, 2016
Member
unfortunately, there is no db schema update in PolyJDBC (the tool we use to manage sql),
we do it manually, see ex JaversSchemaManager.alterCommitIdColumnIfNeeded()
So we need a new method which checks if column not exists and then runs DDL.
It's not easy to write and test.
There is another way we can try. Version field can be moved to CdoSnapshotState, which is serialized to JSON in all repositories. If so, no need to touch repositories code (only core JsonTypeAdapter).
unfortunately, there is no db schema update in PolyJDBC (the tool we use to manage sql),
we do it manually, see ex JaversSchemaManager.alterCommitIdColumnIfNeeded()
So we need a new method which checks if column not exists and then runs DDL.
It's not easy to write and test.
There is another way we can try. Version field can be moved to CdoSnapshotState, which is serialized to JSON in all repositories. If so, no need to touch repositories code (only core JsonTypeAdapter).
milanov
Jan 9, 2016
Author
Contributor
Okay, I went this way but I'm a bit stuck on how to serialize the CdoSnapshotState including the new version field. Currently it is serialized as state: {prop1: val1, ..}
. I can either add the version field inside of the state as another property (but then it could already have a property with the same name which means a conflict) or serialize it as state: {version: X, properties: {prop1: val1, ..}}
, but this wouldn't be backward compatible. Any ideas on this?
Okay, I went this way but I'm a bit stuck on how to serialize the CdoSnapshotState including the new version field. Currently it is serialized as state: {prop1: val1, ..}
. I can either add the version field inside of the state as another property (but then it could already have a property with the same name which means a conflict) or serialize it as state: {version: X, properties: {prop1: val1, ..}}
, but this wouldn't be backward compatible. Any ideas on this?
bartoszwalacik
Jan 10, 2016
Member
You are right, there is no elegant way to serialize version in CdoSnapshotState. Looks like we need to come back to your original solution and write addColumnIfNotExists()
method in JaversSchemaManager
. We need to take a deep dive into PollyJDBC code. It's very lightweight but simple library ...
You are right, there is no elegant way to serialize version in CdoSnapshotState. Looks like we need to come back to your original solution and write addColumnIfNotExists()
method in JaversSchemaManager
. We need to take a deep dive into PollyJDBC code. It's very lightweight but simple library ...
bartoszwalacik
Jan 10, 2016
Member
There are two possible solutions. We can write generic addColumnIfNotExists() method. Better place for this is PollyJDBC, but easier way is to write it in JaVers.
Second solution, would be quick hack, just a series of Ifs with proper DDLs,
like we did in line 57 in JaversSchemaManager
There are two possible solutions. We can write generic addColumnIfNotExists() method. Better place for this is PollyJDBC, but easier way is to write it in JaVers.
Second solution, would be quick hack, just a series of Ifs with proper DDLs,
like we did in line 57 in JaversSchemaManager
milanov
Jan 11, 2016
Author
Contributor
There will be a problem with the previously stored snapshots, because they would have to all be traversed and assigned versions in incrementing order..This might not be that easy.
There will be a problem with the previously stored snapshots, because they would have to all be traversed and assigned versions in incrementing order..This might not be that easy.
bartoszwalacik
Jan 12, 2016
Member
I think that one sub-select could be removed:
UPDATE jv_snapshot s SET s.version = (
SELECT COUNT(*) + 1 FROM jv_snapshot s2 WHERE s.global_id_fk = s2.global_id_fk and s2.snapshot_pk < s.snapshot_pk)
This query will be slow, but maybe good enough for migration script.
So PR is now ready to merge?
I think that one sub-select could be removed:
UPDATE jv_snapshot s SET s.version = (
SELECT COUNT(*) + 1 FROM jv_snapshot s2 WHERE s.global_id_fk = s2.global_id_fk and s2.snapshot_pk < s.snapshot_pk)
This query will be slow, but maybe good enough for migration script.
So PR is now ready to merge?
milanov
Jan 12, 2016
Author
Contributor
Now it should be, I separated the test according what we discussed. As a note to the query from your last comment - I also tried it at first, but it gives an error in MySQL ("You can't specify target table 's' for update in FROM clause", as well as in PostgreSQL("ERROR: column "s" of relation "employees" does not exist") and I haven't tested it on the others.
Now it should be, I separated the test according what we discussed. As a note to the query from your last comment - I also tried it at first, but it gives an error in MySQL ("You can't specify target table 's' for update in FROM clause", as well as in PostgreSQL("ERROR: column "s" of relation "employees" does not exist") and I haven't tested it on the others.
bartoszwalacik
Jan 13, 2016
Member
ok, give us a few more days for merge and release
ok, give us a few more days for merge and release
bartoszwalacik
Jan 16, 2016
Member
@milanov there are few bugs here:
- NPE is thrown when version field is missing in a snapshot stored in SQL or Mongo (created by javers <= 1.4.2)
migration script is optional, so JaVers cant crash when someone doesnt run it
(I will fix it)
- version is ALWAYS deserialized to 1
take a look at line 60 in CdoSnapshotTypeAdapter. When snapshots are deserialized from DB, previous field is null,
and builder overwrites number set in .withVersion(version)
- version field is never write to Sql database, you simply forgot to add it to
CdoSnapshotRepository.insertSnapshot()
@milanov there are few bugs here:
- NPE is thrown when version field is missing in a snapshot stored in SQL or Mongo (created by javers <= 1.4.2)
migration script is optional, so JaVers cant crash when someone doesnt run it
(I will fix it) - version is ALWAYS deserialized to 1
take a look at line 60 in CdoSnapshotTypeAdapter. When snapshots are deserialized from DB, previous field is null,
and builder overwrites number set in.withVersion(version)
- version field is never write to Sql database, you simply forgot to add it to
CdoSnapshotRepository.insertSnapshot()
…dy existing snapshots databases
I did fixes on my branch, and just merged all into master |
released in 1.4.4 |
I'm happy to accept any comments, suggestions, etc :)