Problem importing/uploading new models into Molgenis

[jjettenn]

When loading multiple models after each other you get an error.

Reproduce:

1. Load: BioSample-Model.xlsx
2. Load: Chado-Model.xlsx
3. Load: DAS-MODELv1.6.xlsx
4. Load: DICOM-MODEL.xlsx .... Then you get an error

Workaround: wait a minute or two and try again

Stack trace
463273 INFO [org.molgenis.data.mysql.MysqlRepository] Added 1 attributes entities.
463295 INFO [org.molgenis.data.mysql.MysqlRepository] Added 1 attributes entities.
463856 INFO [org.molgenis.data.importer.ImportJob] Import failed.
org.elasticsearch.common.util.concurrent.UncategorizedExecutionException: Failed execution
at org.elasticsearch.action.support.AdapterActionFuture.rethrowExecutionException(AdapterActionFuture.java:92)
at org.elasticsearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:50)
at org.molgenis.data.elasticsearch.ElasticSearchService.index(ElasticSearchService.java:717)
at org.molgenis.data.elasticsearch.ElasticSearchService.index(ElasticSearchService.java:708)
at org.molgenis.data.elasticsearch.AbstractElasticsearchRepository.update(AbstractElasticsearchRepository.java:186)
at org.molgenis.data.elasticsearch.ElasticsearchRepositoryDecorator.update(ElasticsearchRepositoryDecorator.java:95)
at org.molgenis.data.validation.RepositoryValidationDecorator.update(RepositoryValidationDecorator.java:38)
at org.molgenis.data.CrudRepositorySecurityDecorator.update(CrudRepositorySecurityDecorator.java:140)
at org.molgenis.data.support.DataServiceImpl.update(DataServiceImpl.java:177)
at org.molgenis.omx.MolgenisDbSettings.setProperty(MolgenisDbSettings.java:84)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.molgenis.security.runas.RunAsSystemProxy.invoke(RunAsSystemProxy.java:36)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy54.setProperty(Unknown Source)
at org.molgenis.ui.menumanager.MenuManagerServiceImpl.saveMenu(MenuManagerServiceImpl.java:72)
at org.molgenis.ui.menumanager.MenuManagerServiceImpl$$FastClassBySpringCGLIB$$b795770d.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:708)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.molgenis.security.runas.RunAsSystemProxy.invoke(RunAsSystemProxy.java:36)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:64)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:98)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:262)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:95)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:644)
at org.molgenis.ui.menumanager.MenuManagerServiceImpl$$EnhancerBySpringCGLIB$$f3449487.saveMenu()
at org.molgenis.data.importer.ImportPostProcessingService.addMenuItems(ImportPostProcessingService.java:36)
at org.molgenis.data.importer.ImportPostProcessingService$$FastClassBySpringCGLIB$$49c09e11.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:708)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.molgenis.security.runas.RunAsSystemProxy.invoke(RunAsSystemProxy.java:47)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:644)
at org.molgenis.data.importer.ImportPostProcessingService$$EnhancerBySpringCGLIB$$b3a088ff.addMenuItems()
at org.molgenis.data.importer.ImportJob.run(ImportJob.java:53)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Document contains at least one immense term in field="Value.raw" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[123, 34, 116, 121, 112, 101, 34, 58, 34, 109, 101, 110, 117, 34, 44, 34, 105, 100, 34, 58, 34, 109, 97, 105, 110, 34, 44, 34, 108, 97]...', original message: bytes can be at most 32766 in length; got 36790
at org.elasticsearch.common.util.concurrent.BaseFuture$Sync.getValue(BaseFuture.java:288)
at org.elasticsearch.common.util.concurrent.BaseFuture$Sync.get(BaseFuture.java:275)
at org.elasticsearch.common.util.concurrent.BaseFuture.get(BaseFuture.java:113)
at org.elasticsearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:45)
... 47 more
Caused by: java.lang.IllegalArgumentException: Document contains at least one immense term in field="Value.raw" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[123, 34, 116, 121, 112, 101, 34, 58, 34, 109, 101, 110, 117, 34, 44, 34, 105, 100, 34, 58, 34, 109, 97, 105, 110, 34, 44, 34, 108, 97]...', original message: bytes can be at most 32766 in length; got 36790
at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:671)
at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:342)
at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:301)
at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:241)
at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:451)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1539)
at org.elasticsearch.index.engine.internal.InternalEngine.innerIndex(InternalEngine.java:572)
at org.elasticsearch.index.engine.internal.InternalEngine.index(InternalEngine.java:492)
at org.elasticsearch.index.shard.service.InternalIndexShard.index(InternalIndexShard.java:409)
at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:195)
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:535)
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:434)
... 3 more
Caused by: org.apache.lucene.util.BytesRefHash$MaxBytesLengthExceededException: bytes can be at most 32766 in length; got 36790
at org.apache.lucene.util.BytesRefHash.add(BytesRefHash.java:284)
at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:151)
at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:645)
... 14 more

[dennishendriksen]

Same as #2087?

[erwinwinder]

Fixed by not indexing RuntimeProperty. ElasticSearch has a max of 32766, RuntimePorperty can exceed this