New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

migrate mesh 0.22.0 to 0.27.2 #546

Closed
lilian-benoit opened this Issue Nov 13, 2018 · 12 comments

Comments

Projects
None yet
2 participants
@lilian-benoit

lilian-benoit commented Nov 13, 2018

Gentics Mesh Version, operating system, or hardware.

  • v0.22.0 (old version)
  • v0.27.2 (new version)
    I have two instances but i will migrate one first instance.

Operating System

  • Linux 64bit

JVM

  • Oracle JDK 1.8.0_60

Problem

orientdb haven't migrate after startup. It see releaseimpl in orientdb's studio. But normally Releases is renamed to Branches (v0.23.0)

Reproducer

I have backuped directory graphdb
I have started mesh with new war

2018-11-13 08:41:35:219 WARNI Requested command 'Committing the active transaction to create the new type 'BranchRootImpl' as subclass of 'V'. The transaction will be reopen right after that. To avoid this behavior create the classes outside the transaction' must be executed outside active transaction: the transaction will be committed and reopen right after it. To avoid this behavior execute it outside a transaction [OrientGraph]

To be sure, i stopped mesh
I have restored directory graphdb
I have started mesh with new war.

mesh's log => https://framabin.org/p/?9fc41ac37d0e4b39#+Q1+NqMQXtKBoq/oDNIdWJD0o1jZXyNquZI5A54zYiI=
orientdb's log => https://framabin.org/p/?5593e50b86b3f321#QNj+pOMGH1zPd9P+A2zeXhr0lbcE4ObBLJLYKJ0DBoI=

Expected behaviour and actual behaviour

orientdb should be migrate.
I should see branchimpl in orientdb's studio

I launched endpoint /api/v1/admin/consistency/check. Result is InternalServeError.

2018-11-13 22:16:08.662 [app1_cms0] ERROR [vert.x-eventloop-thread-2] - Error for request in path: /api/v1/admin/consistency/check
2018-11-13 22:16:08.663 [app1_cms0] ERROR [vert.x-eventloop-thread-2] - Error:
java.lang.IllegalArgumentException: Cannot find class 'BranchRootImpl' in database schema
        at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.getVerticesOfClass(OrientBaseGraph.java:729) ~[mesh-server-0.27.2.jar:na]
        at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.getVerticesOfClass(OrientBaseGraph.java:713) ~[mesh-server-0.27.2.jar:na]
        at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.getVertices(OrientBaseGraph.java:750) ~[mesh-server-0.27.2.jar:na]
        at com.gentics.mesh.graphdb.OrientDBDatabase.getVerticesForType(OrientDBDatabase.java:639) ~[mesh-server-0.27.2.jar:na]
        at com.gentics.mesh.core.endpoint.admin.consistency.check.BranchCheck.invoke(BranchCheck.java:30) ~[mesh-server-0.27.2.jar:na]
        at com.gentics.mesh.core.endpoint.admin.consistency.ConsistencyCheckHandler.lambda$invokeCheck$0(ConsistencyCheckHandler.java:76) ~[mesh-server-0.27.2.jar:na]
        at com.gentics.mesh.graphdb.spi.Database.lambda$null$6(Database.java:175) ~[mesh-server-0.27.2.jar:na]
        at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:273) ~[mesh-server-0.27.2.jar:na]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_60]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_60]
@Jotschi

This comment has been minimized.

Member

Jotschi commented Nov 15, 2018

As mentioned in the gitter channel. Please execute the following steps:

  • Stop the server
  • Copy the backup data of version 0.22.x into the data folder
  • Disable clustering in the config
  • Start Mesh 0.27.x
  • Send log to me

@Jotschi Jotschi self-assigned this Nov 15, 2018

@lilian-benoit

This comment has been minimized.

lilian-benoit commented Nov 15, 2018

Migrate tasks appears in a log file. But i have a error

2018-11-16 00:23:13.673 [app1_cms0] INFO [main] - Changelog completed.
2018-11-16 00:23:13.673 [app1_cms0] INFO [main] - Updating stored database revision and mesh version.
2018-11-16 00:23:13.701 [app1_cms0] INFO [main] - Creating database indices. This may take a few seconds...
2018-11-16 00:23:15.682 [app1_cms0] ERROR [main] - Error while starting mesh
com.orientechnologies.orient.core.storage.ORecordDuplicatedException: Cannot index record #62:72: found duplicated key 'OCompositeKey{keys=[8f1ac513a0af4f8e9ac513a0afdf8e29, D, ]}' in index 'e.has_field_container_webrootpathinfoindex' previously assigned to the record #62:1
DB name="storage"
at com.orientechnologies.orient.core.index.OIndexUnique$1.validate(OIndexUnique.java:47) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexUnique$1.validate(OIndexUnique.java:37) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1539) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.put(OLocalHashTable.java:1459) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.validatedPut(OLocalHashTable.java:412) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.engine.OHashTableIndexEngine.validatedPut(OHashTableIndexEngine.java:156) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doValidatedPutIndexValue(OAbstractPaginatedStorage.java:2519) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.validatedPutIndexValue(OAbstractPaginatedStorage.java:2498) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexUnique.put(OIndexUnique.java:82) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexUnique.put(OIndexUnique.java:35) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexAbstract.populateIndex(OIndexAbstract.java:940) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexAbstract.indexCluster(OIndexAbstract.java:1040) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexAbstract.fillIndex(OIndexAbstract.java:539) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexAbstract.create(OIndexAbstract.java:250) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexOneValue.create(OIndexOneValue.java:129) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexOneValue.create(OIndexOneValue.java:40) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexManagerShared.createIndex(OIndexManagerShared.java:171) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexManagerProxy.createIndex(OIndexManagerProxy.java:87) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.metadata.schema.OClassImpl.createIndex(OClassImpl.java:1900) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.metadata.schema.OClassImpl.createIndex(OClassImpl.java:1870) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.metadata.schema.OClassAbstractDelegate.createIndex(OClassAbstractDelegate.java:397) ~[mesh-server-0.27.2.jar:na]
at com.tinkerpop.blueprints.impls.orient.OrientElementType.access$701(OrientElementType.java:35) ~[mesh-server-0.27.2.jar:na]
at com.tinkerpop.blueprints.impls.orient.OrientElementType$8.call(OrientElementType.java:123) ~[mesh-server-0.27.2.jar:na]
at com.tinkerpop.blueprints.impls.orient.OrientElementType$8.call(OrientElementType.java:120) ~[mesh-server-0.27.2.jar:na]
at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.executeOutsideTx(OrientBaseGraph.java:1771) ~[mesh-server-0.27.2.jar:na]
at com.tinkerpop.blueprints.impls.orient.OrientElementType.createIndex(OrientElementType.java:120) ~[mesh-server-0.27.2.jar:na]
at com.gentics.mesh.graphdb.OrientDBDatabase.addCustomEdgeIndex(OrientDBDatabase.java:546) ~[mesh-server-0.27.2.jar:na]
at com.gentics.mesh.core.data.impl.GraphFieldContainerEdgeImpl.init(GraphFieldContainerEdgeImpl.java:52) ~[mesh-server-0.27.2.jar:na]
at com.gentics.mesh.core.data.impl.DatabaseHelper.init(DatabaseHelper.java:80) ~[mesh-server-0.27.2.jar:na]
at com.gentics.mesh.cli.BootstrapInitializerImpl.initLocalData(BootstrapInitializerImpl.java:223) ~[mesh-server-0.27.2.jar:na]
at com.gentics.mesh.cli.BootstrapInitializerImpl.init(BootstrapInitializerImpl.java:296) ~[mesh-server-0.27.2.jar:na]
at com.gentics.mesh.cli.MeshImpl.run(MeshImpl.java:134) [mesh-server-0.27.2.jar:na]
at com.gentics.mesh.cli.MeshImpl.run(MeshImpl.java:98) [mesh-server-0.27.2.jar:na]
at com.gentics.mesh.server.ServerRunner.main(ServerRunner.java:55) [mesh-server-0.27.2.jar:na]
2018-11-16 00:23:15.682 [app1_cms0] INFO [main] - Mesh shutting down...
2018-11-16 00:23:16.287 [app1_cms0] INFO [main] - Closing Elasticsearch REST client.
2018-11-16 00:23:16.288 [app1_cms0] INFO [main] - Stopping Elasticsearch server.
2018-11-16 00:23:17.288 [] INFO [main] - Shutdown completed...
2018-11-16 00:23:36.288 [] INFO [Thread-0] - Mesh shutting down...
2018-11-16 00:23:36.288 [] INFO [Thread-0] - Closing Elasticsearch REST client.
2018-11-16 00:23:36.288 [] INFO [Thread-0] - Stopping Elasticsearch server.
2018-11-16 00:23:36.288 [] INFO [Thread-0] - Shutdown completed...

Mesh stopped after migration successfully

@lilian-benoit

This comment has been minimized.

@lilian-benoit

This comment has been minimized.

lilian-benoit commented Nov 20, 2018

in gitter, it is three options

  • Restore data of 0.22.x and start the latest 0.22.x version and try to repair the inconsistency and re-run the upgrade to 0.24.x -
  • Import the dump into a dedicated OrientDB studio
  • Dump all the data of 0.22.x via an export script using the API and re-import it into 0.27.x
@lilian-benoit

This comment has been minimized.

lilian-benoit commented Nov 20, 2018

I try first option.

  • I restored data of 0.22.x but database is consistency (without repair)
  • I upgraded to 0.24.x. I have the same error
@Jotschi

This comment has been minimized.

Member

Jotschi commented Nov 21, 2018

@lilian-benoit I was able to reproduce the issue. The problem happens if you have at least two nodes in your system which have an empty segment field. This can happen if you:

  1. Create a schema with two field (name, name2) - (No segment field selected, name2 is the display field)
  2. Create two nodes of this schema in which only name2 field set (name field is left empty)
  3. Update the schema and set the segment field to "name". Nodes will be migrated but the field remains empty.

It may also be possible to produce this issue if the segment field is not set to required and just left empty.

If you upgrade this data state to 0.24.x you will see the error. It should however be possible to update the nodes (draft and published versions) and set the missing value in version 0.22.x - If you update the nodes the error should not happen when launching 0.24.x.

I however have not yet confirmed this in my setup. I'll also work on a fix to resolve this problem in 0.24.x.

@lilian-benoit

This comment has been minimized.

lilian-benoit commented Nov 21, 2018

if my understand is good, i must verify value of segment field of all my nodes

@Jotschi

This comment has been minimized.

Member

Jotschi commented Nov 21, 2018

@lilian-benoit Right - in both versions: draft and published

@Jotschi Jotschi added the f/webroot label Nov 23, 2018

@lilian-benoit

This comment has been minimized.

lilian-benoit commented Nov 24, 2018

@Jotschi Two document was on error with segment field null and all folder (same root folder). I updated all nodes but I had same error on migration

@Jotschi

This comment has been minimized.

Member

Jotschi commented Nov 25, 2018

@lilian-benoit In that case the error might still be happening due to older version which will also be migrated and still have the "wrong" data in them. I'll add a dedicated test for this case and see whether that reveals the issue.

You could try to move your nodes around and delete the affected nodes and see whether that resolves the issue.

@Jotschi

This comment has been minimized.

Member

Jotschi commented Nov 25, 2018

@philippguertler I think the error happens because I changed the webroot index mechanism from a simple single key of the NodeImpl to a composite index on HAS_FIELD_CONTAINER edge.

db.addCustomEdgeIndex(HAS_FIELD_CONTAINER, WEBROOT_INDEX_POSTFIX_NAME, fields, true);

Now there are there are 2 components of the index non-null and the last one is null. Before there was only a single key which was null and the index was setup to ignore null values. For composite indices this is no longer possible. This change could potentially also cause problems on new webroot indices.

@Jotschi

This comment has been minimized.

Member

Jotschi commented Nov 26, 2018

@lilian-benoit I fixed a bug in the changelog entry that was supposed to update the database. The fix will be included in the next release.

@Jotschi Jotschi closed this in 5dedb60 Nov 26, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment