Skip to content

Commit

Permalink
ignore on media_black_point
Browse files Browse the repository at this point in the history
  • Loading branch information
@ArtificialOwl
ArtificialOwl committed Oct 27, 2016
1 parent 11c3965 commit c11986e
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions lib/Service/SolrService.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -265,6 +265,9 @@ public function extractDocument(&$document, &$error = '')
$query->addFieldMapping('ul', 'ignored_');
$query->addFieldMapping('li', 'ignored_');

$query->addFieldMapping('media_black_point', 'ignored_');
$query->addFieldMapping('media_white_point', 'ignored_');

$query->setFile($document->getAbsolutePath());
$query->setCommit(true);
$query->setOmitHeader(true);
Expand Down

7 comments on commit c11986e

@devz3r0
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm in my file the insert is on line 265 and 266 not from 268is that an issue?
Anyhow, I rerun it with the inserted lines.

@ArtificialOwl
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, don't worry about that small shift with the line number

@devz3r0
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it took longer, but died again.

nextant v0.6.0 (beta)

* Extracting files:
/dev-wa/files                  10/   10 [============================] 100%
+devzero/files              12412/12965 [==========================>-]  95%
    [extracting] Solr memory: 338.2 MB (%68.9)

  [Solarium\Exception\HttpException]
  Solr HTTP error: HTTP request failed, Recv failure: Connection reset by peer


nextant:index [--output [OUTPUT]] [-d|--debug] [-u|--unlock] [-f|--force] [-c|--background] [-b|--bookmarks] [-i|--files] [-x|--files_extract] [-r|--files_update]


62776 ERROR (qtp1654589030-17) [   x:nextant] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: ERROR: [doc=files_12259] Error adding field 'media_black_point'='(0.0, 0.0, 0.0)' msg=For input string: "(0.0"
        at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:177)
        at org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:82)
        at org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:277)
        at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:211)
        at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:166)
        at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:67)
        at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
        at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:939)
        at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1094)
        at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:720)
        at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
        at org.apache.solr.handler.extraction.ExtractingDocumentLoader.doAdd(ExtractingDocumentLoader.java:126)
        at org.apache.solr.handler.extraction.ExtractingDocumentLoader.addDoc(ExtractingDocumentLoader.java:131)
        at org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:237)
        at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)
        at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
116899 ERROR (qtp1654589030-1700) [   x:nextant] o.a.s.s.HttpSolrCall null:java.lang.RuntimeException: java.lang.NoClassDefFoundError: com/healthmarketscience/jackcess/CryptCodecProvider
        at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:605)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:474)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
        at org.eclipse.jetty.server.Server.handle(Server.java:518)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
        at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: com.uwyn.jhighlight.renderer.XhtmlRendererFactory
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:814)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 57 more
214045 INFO  (qtp1654589030-6611) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_51376&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rowsed_&fmap.h3=ignored_&resource.name=org.dmfs.jb.workaround.carddavsync-20160625-085310.tar.gz.d1467490953&literal.id=files_52354&fmap.h1=ignored_&commit=true&fmap.link=ignored_&fmap.ul=ignored_&uprefix=nextant_attr_&fmap.form=ignored_&fmap.i=ignored_&fmap.media_white_point=ignored_&fmap.img=ignored_&literal.nextant_path=/devzero/files_trashbin/files/org.dmfs.jb.workaround.carddavsync-20160625-085310.tar.gz.d1467490953&fmap.a=ignored_&fmap.b=ignored_&fmap.table=ignored_&fmap.content=text&fmap.script=ignored_&literal.nextant_owner=devzero&fmap.html=ignored_&fmap.div=ignored_&ignoreTikaException=true&wt=json&literal.nextant_mtime=1466845674&json.nl=flat&fmap.media_black_point=ignored_&extractOnly=false&literal.nextant_source=files&fmap.td=ignored_&fmap.tr=ignored_&fmap.li=ignored_&omitHeader=true&literal.nextant_deleted=true&captureAttr=true&fmap.input=ignored_&fmap.span=ignored_}{add=[files_52354 (1549358912672628736)],commit=} 0 825
233622 INFO  (qtp1654589030-4282) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system params={q=*:*&json.nl=flat&omitHeader=true&fl=*,score&start=0&rows=10&wt=json} status=0 QTime=66
233692 INFO  (qtp1654589030-6415) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_51223&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=60
238710 INFO  (qtp1654589030-6415) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_51710&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=0
239332 INFO  (qtp1654589030-6415) [   x:nextant] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
239332 INFO  (qtp1654589030-6415) [   x:nextant] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@97ca05d
239898 INFO  (qtp1654589030-6415) [   x:nextant] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
        commit{dir=NRTCachingDirectory(MMapDirectory@/home/martin/solr/solr-6.2.1/server/solr/nextant/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@34bf3557; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3wb,generation=5051}
        commit{dir=NRTCachingDirectory(MMapDirectory@/home/martin/solr/solr-6.2.1/server/solr/nextant/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@34bf3557; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3wc,generation=5052}
239898 INFO  (qtp1654589030-6415) [   x:nextant] o.a.s.c.SolrDeletionPolicy newest commit generation = 5052
239906 INFO  (qtp1654589030-6415) [   x:nextant] o.a.s.s.SolrIndexSearcher Opening [Searcher@2ee66949[nextant] main]
239908 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@2ee66949[nextant] main{Exitable):C1) Uninverting(_4ej(6.2.1):c109) Uninverting(_4ek(6.2.1):C1)))}
388183 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.QuerySenderListener QuerySenderListener done.
388183 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.SolrCore [nextant] Registered new searcher Searcher@4c76e79c[nextant] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2hs(6.2.1):C2908) Uninverting(_3rp(6.2.1):C1486) Uninverting(_470(6.2.1):C501) Uninverting(_4b7(6.2.1):C137) Uninverting(_4ap(6.2.1):C1) Uninverting(_4ej(6.2.1):c109) Uninverting(_4ek(6.2.1):C1)))}
388700 INFO  (qtp1654589030-4980) [   x:nextant] o.a.s.u.p.LogUpdateProcessorFactory [nextant]  webapp=/solr path=/update/extract params={fmap.p=ignored_&fmap.h2=ignored_&fmap.style=ignored_&fmap.h3=ignored_&resource.name=com.google.android.apps.docs.editors.slides-20151029-122535+(1).properties.d1472384733&literal.id=files_51541&fmap.h1=ignored_&commit=true&fmap.link=ignored_&fmap.ul=ignored_&uprefix=nextant_attr_&fmap.form=ignored_&fmap.i=ignored_&fmap.media_white_point=ignored_&fmap.img=ignored_&literal.nextant_path=/devzero/files_trashbin/files/com.google.android.apps.docs.editors.slides-20151029-122535+(1).properties.d1472384733&fmap.a=ignored_&fmap.b=ignored_&fmap.table=ignored_&fmap.content=text&fmap.script=ignored_&literal.nextant_owner=devzero&fmap.html=ignored_&fmap.div=ignored_&ignoreTikaException=true&wt=json&literal.nextant_mtime=1466589522&json.nl=flat&fmap.media_black_point=ignored_&extractOnly=false&literal.nextant_source=files&fmap.td=ignored_&fmap.tr=ignored_&fmap.li=ignored_&omitHeader=true&literal.nextant_deleted=true&captureAttr=true&fmap.input=ignored_&fmap.span=ignored_}{add=[files_51541 (1549359074693349376)],commit=} 0 1415
388712 INFO  (qtp1654589030-11077) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_51180&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rowatus=0 QTime=53
539312 INFO  (qtp1654589030-20420) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_51821&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=0
539362 INFO  (qtp1654589030-20498) [   x:nextant] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
539363 INFO  (qtp1654589030-20498) [   x:nextant] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@97ca05d
540071 INFO  (qtp1654589030-20498) [   x:nextant] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
1236481 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@322c5258[nextant] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2hs(6.2.1):C2908) Uninverting(_3rp(6.2.1):C1486) Uninverting(_470(6.2.1):C501) Uninverting(_4hd(6.2.1):C336) Uninverting(_4kz(6.2.1):c120) Uninverting(_4kp(6.2.1):C1) Uninverting(_4kv(6.2.1):C1) Uninverting(_4ky(6.2.1):C1) Uninverting(_4l0(6.2.1):C1) Uninverting(_4l1(6.2.1):C1) Uninverting(_4l2(6.2.1):C1)))}
1236481 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.QuerySenderListener QuerySenderListener done.
1236481 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.SolrCore [nextant] Registered new searcher Searcher@322c5258[nextant] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2hs(6.2.1):C2908) Uninverting(_3rp(6.2.1):C1486) Uninverting(_470(6.2.1):C501) Uninverting(_4hd(6.2.1):C336) Uninverting(_4kz(6.2.1):c120) Uninverting(_4kp(6.2.1):C1) Uninverting(_4kv(6.2.1):C1) Uninverting(_4ky(6.2.1):C1) Uninverting(_4l0(6.2.1):C1) Uninverting(_4l1(6.2.1):C1) Uninverting(_4l2(6.2.1):C1)))}
1237294 INFO  (qtp1654589030-53199) [   x:nextant] o.a.s.u.DirectUpdateHandler2 end_commit_flush
1237294 INFO  (qtp1654589030-53199) [   x:nextant] o.a.s.u.p.LogUpdateProcessorFactory [nextant]  webapp=/solr path=/update/extract params={fmap.p=ignored_&fmap.h2=ignored_&fmap.style=ignored_&fmap.h3=ignored_&resource.name=com.google.android.apps.docs.editors.sheets-20151029-121644.tar+(1).gz.d1472384713&literal.id=files_51568&fmap.h1=ignored_&commit=true&fmap.link=ignored_&fmap.ul=ignored_&uprefix=nextant_attr_&fmap.form=ignored_&fmap.i=ignored_&fmap.media_white_point=ignored_&fmap.img=ignored_&literal.nextant_path=/devzero/files_trashbin/files/com.google.android.apps.docs.editors.sheets-20151029-121644.tar+(1).gz.d1472384713&fmap.a=ignored_&fmap.b=ignored_&fmap.table=ignored_&fmap.content=text&fmap.script=ignored_&literal.nextant_owner=devzero&fmap.html=ignored_&fmap.div=ignored_&ignoreTikaException=true&wt=json&literal.nextant_mtime=1466589640&json.nl=flat&fmap.media_black_point=ignored_&extractOnly=false&literal.nextant_source=files&fmap.td=ignored_&fmap.tr=ignored_&fmap.li=ignored_&omitHeader=true&literal.nextant_deleted=true&captureAttr=true&fmap.input=ignored_&fmap.span=ignored_}{add=[files_51568 (1549359962761723904)],commit=} 0 4999
1238226 INFO  (qtp1654589030-55014) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system params={q=*:*&json.nl=flat&omitHeader=true&fl=*,score&start=0&rows=10&wt=json} status=0 QTime=48
1239354 INFO  (qtp1654589030-52190) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_25229&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=0
1239405 INFO  (qtp1654589030-54173) [   x:nextant] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
1239405 INFO  (qtp1654589030-54173) [   x:nextant] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@97ca05d
1241609 INFO  (qtp1654589030-54173) [   x:nextant] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
_&fmap.div=ignored_&ignoreTikaException=true&wt=json&literal.nextant_mtime=1466840674&json.nl=flat&fmap.media_black_point=ignored_&extractOnly=false&literal.nextant_source=files&fmap.td=ignored_&fmap.tr=ignored_&fmap.li=ignored_&omitHeader=true&literal.nextant_deleted=true&captureAttr=true&fmap.input=ignored_&fmap.span=ignored_}{add=[files_51766 (1549360461698301952)],commit=} 0 13389
1720469 INFO  (qtp1654589030-73616) [   x:nextant] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
1720470 INFO  (qtp1654589030-73616) [   x:nextant] o.a.s.c.SolrCore SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
1720470 INFO  (qtp1654589030-73616) [   x:nextant] o.a.s.u.DirectUpdateHandler2 end_commit_flush
1720470 INFO  (qtp1654589030-73616) [   x:nextant] o.a.s.u.p.LogUpdateProcessorFactory [nextant]  webapp=/solr path=/update/extract params={fmap.p=ignored_&fmap.h2=ignored_&fmap.style=ignored_&fmap.h3=ignored_&resource.name=Pass.kdbx.d1473489703&literal.id=files_59983&fmap.h1=ignored_&commit=true&fmap.link=ignored_&fmap.ul=ignored_&uprefix=nextant_attr_&fmap.form=ignored_&fmap.i=ignored_&fmap.media_white_point=ignored_&fmap.img=ignored_&literal.nextant_path=/devzero/files_trashbin/files/Pass.kdbx.d1473489703&fmap.a=ignored_&fmap.b=ignored_&fmap.table=ignored_&fmap.content=text&fmap.script=ignored_&literal.nextant_owner=devzero&fmap.html=ignored_&fmap.div=ignored_&ignoreTikaException=true&wt=json&literal.nextant_mtime=1473340054&json.nl=flat&fmap.media_black_point=ignored_&extractOnly=false&literal.nextant_source=files&fmap.td=ignored_&fmap.tr=ignored_&fmap.li=ignored_&omitHeader=true&literal.nextant_deleted=true&captureAttr=true&fmap.input=ignored_&fmap.span=ignored_}{add=[files_59983 (1549360466902384640)],commit=} 0 5495
1720705 INFO  (qtp1654589030-78248) [   x:nextant] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
1720705 INFO  (qtp1654589030-78248) [   x:nextant] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@97ca05d
1721276 INFO  (qtp1654589030-78248) [   x:nextant] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
        commit{dir=NRTCachingDirectory(MMapDirectory@/home/martin/solr/solr-6.2.1/server/solr/nextant/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@34bf3557; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_47g,generation=5452}
        commit{dir=NRTCachingDirectory(MMapDirectory@/home/martin/solr/solr-6.2.1/server/solr/nextant/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@34bf3557; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_47h,generation=5453}
1832364 INFO  (qtp1654589030-82911) [   x:nextant] o.a.s.c.SolrDeletionPolicy newest commit generation = 5453
1832368 INFO  (qtp1654589030-82911) [   x:nextant] o.a.s.s.SolrIndexSearcher Opening [Searcher@1c766311[nextant] main]
1832369 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@1c766311[nextant] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2hs(6.2.1):C2908) Uninverting(_3rp(6.2.1):C1486) Uninverting(_470(6.2.1):C501) Uninverting(_4hd(6.2.1):C336) Uninverting(_4nq(6.2.1):c195) Uninverting(_4ob(6.2.1):c37) Uninverting(_4nj(6.2.1):C1) Uninverting(_4nt(6.2.1):C1) Uninverting(_4oc(6.2.1):C1) Uninverting(_4od(6.2.1):C1) Uninverting(_4oe(6.2.1):C1) Uninverting(_4of(6.2.1):C1) Uninverting(_4og(6.2.1):C1) Uninverting(_4oh(6.2.1):C1)))}
1832369 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.QuerySenderListener QuerySenderListener done.
1832369 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.SolrCore [nextant] Registered new searcher Searcher@1c766311[nextant] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2hs(6.2.1):C2908) Uninverting(_3rp(6.2.1):C1486) Uninverting(_470(6.2.1):C501) Uninverting(_4hd(6.2.1):C336) Uninverting(_4nq(6.2.1):c195) Uninverting(_4ob(6.2.1):c37) Uninverting(_4nj(6.2.1):C1) Uninverting(_4nt(6.2.1):C1) Uninverting(_4oc(6.2.1):C1) Uninverting(_4od(6.2.1):C1) Uninverting(_4oe(6.2.1):C1) Uninverting(_4of(6.2.1):C1) Uninverting(_4og(6.2.1):C1) Uninverting(_4oh(6.2.1):C1)))}
1833465 INFO  (qtp1654589030-82911) [   x:nextant] o.a.s.u.DirectUpdateHandler2 end_commit_flush
1833465 INFO  (qtp1654589030-82911) [   x:nextant] o.a.s.u.p.LogUpdateProcessorFactory [nextant]  webapp=/solr path=/update/extract params={fmap.p=ignored_&fmap.h2=ignored_&fmap.style=ignored_&fmap.h3=ignored_&resource.name=com.ghisler.tcplugins.FTP-20160621-114126+(1).properties.d1472384567&literal.id=files_51601&fmap.h1=ignored_&commit=true&fmap.link=ignored_&fmap.ul=ignored_&uprefix=nextant_attr_&fmap.form=ignored_&fmap.i=ignored_&fmap.media_white_point=ignored_&fmap.img=ignored_&literal.nextant_path=/devzero/files_trashbin/files/com.ghisler.tcplugins.FTP-20160621-114126+(1).properties.d1472384567&fmap.a=ignored_&fmap.b=ignored_&fmap.table=ignored_&fmap.content=text&fmap.script=ignored_&literal.nextant_owner=devzero&fmap.html=ignored_&fmap.div=ignored_&ignoreTikaException=true&wt=json&literal.nextant_mtime=1466589761&json.nl=flat&fmap.media_black_point=ignored_&extractOnly=false&literal.nextant_source=files&fmap.td=ignored_&fmap.tr=ignored_&fmap.li=ignored_&omitHeader=true&literal.nextant_deleted=true&captureAttr=true&fmap.input=ignored_&fmap.span=ignored_}{add=[files_51601 (1549360587763351552)],commit=} 0 8397
1837884 INFO  (qtp1654589030-82911) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_52619&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&roMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4as,generation=5572}
        commit{dir=NRTCachingDirectory(MMapDirectory@/home/martin/solr/solr-6.2.1/server/solr/nextant/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@34bf3557; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4ca,generation=5626}
2916422 INFO  (qtp1654589030-134825) [   x:nextant] o.a.s.c.SolrDeletionPolicy newest commit generation = 5626
2916424 INFO  (qtp1654589030-134825) [   x:nextant] o.a.s.s.SolrIndexSearcher Opening [Searcher@4d4ad7ca[nextant] main]
2916425 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@4d4ad7ca[nextant] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2hs(6.2.1):C2908) Uninverting(_3rp(6.2.1):C1486) Uninverting(_470(6.2.1):C501) Uninverting(_4hd(6.2.1):C336) Uninverting(_4sr(6.2.1):c379) Uninverting(_4nt(6.2.1):C1) Uninverting(_4q7(6.2.1):C1) Uninverting(_4tw(6.2.1):c41) Uninverting(_4tv(6.2.1):C1) Uninverting(_4tx(6.2.1):C2) Uninverting(_4ty(6.2.1):C1) Uninverting(_4tz(6.2.1):C1) Uninverting(_4u0(6.2.1):C1)))}
2916425 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.QuerySenderListener QuerySenderListener done.
2916425 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.SolrCore [nextant] Registered new searcher Searcher@4d4ad7ca[nextant] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2hs(6.2.1):C2908) Uninverting(_3rp(6.2.1):C1486) Uninverting(_470(6.2.1):C501) Uninverting(_4hd(6.2.1):C336) Uninverting(_4sr(6.2.1):c379) Uninverting(_4nt(6.2.1):C1) Uninverting(_4q7(6.2.1):C1) Uninverting(_4tw(6.2.1):c41) Uninverting(_4tv(6.2.1):C1) Uninverting(_4tx(6.2.1):C2) Uninverting(_4ty(6.2.1):C1) Uninverting(_4tz(6.2.1):C1) Uninverting(_4u0(6.2.1):C1)))}
2916905 INFO  (qtp1654589030-134825) [   x:nextant] o.a.s.u.DirectUpdateHandler2 end_commit_flush
2916905 INFO  (qtp1654589030-134825) [   x:nextant] o.a.s.u.p.LogUpdateProcessorFactory [nextant]  webapp=/solr path=/update/extract params={fmap.p=ignored_&fmap.h2=ignored_&fmap.style=ignored_&fmap.h3=ignored_&resource.name=com.xtralogic.android.rdpclient-20160628-183806.properties.d1467487082&literal.id=files_52630&fmap.h1=ignored_&commit=true&fmap.link=ignored_&fmap.ul=ignored_&uprefix=nextant_attr_&fmap.form=ignored_&fmap.i=ignored_&fmap.media_white_point=ignored_&fmap.img=ignored_&literal.nextant_path=/devzero/files_trashbin/files/com.xtralogic.android.rdpclient-20160628-183806.properties.d1467487082&fmap.a=ignored_&fmap.b=ignored_&fmap.table=ignored_&fmap.content=text&fmap.script=ignored_&literal.nextant_owner=devzero&fmap.html=ignored_&fmap.div=ignored_&ignoreTikaException=true&wt=json&literal.nextant_mtime=1467140774&json.nl=flat&fmap.media_black_point=ignored_&extractOnly=false&literal.nextant_source=files&fmap.td=ignored_&fmap.tr=ignored_&fmap.li=ignored_&omitHeader=true&literal.nextant_deleted=true&captureAttr=true&fmap.input=ignored_&fmap.span=ignored_}{add=[files_52630 (1549361722865745920)],commit=} 0 4133
2916926 INFO  (qtp1654589030-131135) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_52379&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=0
2917006 INFO  (qtp1654589030-134391) [   x:nextant] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
2917006 INFO  (qtp1654589030-134391) [   x:nextant] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@97ca05d
2918605 INFO  (qtp1654589030-134391) [   x:nextant] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
        commit{dir=NRTCachingDirectory(MMapDirectory@/home/martin/solr/solr-6.2.1/server/solr/nextant/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@34bf3557; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4ca,generation=5626}
        commit{dir=NRTCachingDirectory(MMapDirectory@/home/martin/solr/solr-6.2.1/server/solr/nextant/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@34bf3557; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4cb,generation=5627}
2918605 INFO  (qtp1654589030-134391) [   x:nextant] o.a.s.c.SolrDeletionPolicy newest commit generation = 5627
2918608 INFO  (qtp1654589030-134391) [   x:nextant] o.a.s.s.SolrIndexSearcher Opening [Searcher@1949300b[nextant] main]
2918610 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@1949300b[nextant] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2hs(6.2.1):C2908) Uninverting(_3rp(6.2.1):C1486) Uninverting(_470(6.2.1):C501) Uninverting(_4hd(6.2.1):C336) Uninverting(_4sr(6.2.1):c379) Uninverting(_4nt(6.2.1):C1) Uninverting(_4q7(6.2.1):C1) Uninverting(_4tw(6.2.1):c41) Uninverting(_4tv(6.2.1):C1) Uninverting(_4tx(6.2.1):C2) Uninverting(_4ty(6.2.1):C1) Uninverting(_4tz(6.2.1):C1) Uninverting(_4u0(6.2.1):C1) Uninverting(_4u1(6.2.1):C1)))}
2918610 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.QuerySenderListener QuerySenderListener done.
2918610 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.SolrCore [nextant] Registered new searcher Searcher@1949300b[nextant] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2hs(6.2.1):C2908) Uninverting(_3rp(6.2.1):C1486) Uninverting(_470(6.2.1):C501) Uninverting(_4hd(6.2.1):C336) Uninverting(_4sr(6.2.1):c379) Uninverting(_4nt(6.2.1):C1) Uninverting(_4q7(6.2.1):C1) Uninverting(_4tw(6.2.1):c41) Uninverting(_4tv(6.2.1):C1) Uninverting(_4tx(6.2.1):C2) Uninverting(_4ty(6.2.1):C1) Uninverting(_4tz(6.2.1):C1) Uninverting(_4u0(6.2.1):C1) Uninverting(_4u1(6.2.1):C1)))}
2922691 INFO  (qtp1654589030-134391) [   x:nextant] o.a.s.u.DirectUpdateHandler2 end_commit_flush
2922692 INFO  (qtp1654589030-134391) [   x:nextant] o.a.s.u.p.LogUpdateProcessorFactory [nextant]  webapp=/solr path=/update/extract params={fmap.p=ignored_&fmap.h2=ignored_&fmap.style=ignored_&fmap.h3=ignored_&resource.name=com.google.android.talk-e18a347cfbaac004317970e3fe0b9338.apk.gz.d1467490783&literal.id=files_52379&fmap.h1=ignored_&commit=true&fmap.link=ignored_&fmap.ul=ignored_&uprefix=nextant_attr_&fmap.form=ignored_&fmap.i=ignored_&fmap.media_white_point=ignored_&fmap.img=ignored_&literal.nextant_path=/devzero/files_trashbin/files/com.google.android.talk-e18a347cfbaac004317970e3fe0b9338.apk.gz.d1467490783&fmap.a=ignored_&fmap.b=ignored_&fmap.table=ignored_&fmap.content=text&fmap.script=ignored_&literal.nextant_owner=devzero&fmap.html=ignored_&fmap.div=ignored_&ignoreTikaException=true&wt=json&literal.nextant_mtime=1466845742&json.nl=flat&fmap.media_black_point=ignored_&extractOnly=false&literal.nextant_source=files&fmap.td=ignored_&fmap.tr=ignored_&fmap.li=ignored_&omitHeader=true&literal.nextant_deleted=true&captureAttr=true&fmap.input=ignored_&fmap.span=ignored_}{add=[files_52379 (1549361727249842176)],commit=} 0 5758
#
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError="/home/martin/solr/solr-6.2.1/bin/oom_solr.sh 8983 /home/martin/solr/solr-6.2.1/server/logs"
#   Executing /bin/sh -c "/home/martin/solr/solr-6.2.1/bin/oom_solr.sh 8983 /home/martin/solr/solr-6.2.1/server/logs"...
Running OOM killer script for process 14156 for Solr on port 8983
Killed process 14156

@ArtificialOwl
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I just saw that you only allocate 512Mo of memory to the solr servlet
I would advice to try at least with 2Go

Thanks anyway for that report, this should slightly lower the needed memory and I will still investigate the issue with some Jpeg and this should fix

@devz3r0
Copy link

@devz3r0 devz3r0 commented on c11986e Oct 27, 2016 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i just came to the same idea:

~/solr/solr-6.2.1/bin$ ./solr start -m 2g

but still:


39942 INFO  (qtp1654589030-19) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system params={q=*:*&json.nl=flat&omitHeader=true&fl=*,score&start=0&rows=10&wt=json} statusdpclient-20151020-111641+(1).TiBkp.d1472386316&fmap.a=ignored_&fmap.b=ignored_&fmap.table=ignored_&fmap.content=text&fmap.script=ignored_&literal.nextant_owner=devzero&fmap.html=ignored_&fmap.div=ignored_&ignoreTikaException=true&wt=json&literal.nextant_mtime=1466589238&json.nl=flat&fmap.media_black_point=ignored_&extractOnly=false&literal.nextant_source=files&fmap.td=ignored_&fmap.tr=ignored_&fmap.li=ignored_&omitHeader=true&literal.nextant_deleted=true&captureAttr=true&fmap.input=ignored_&fmap.span=ignored_}{} 0 304
151202 ERROR (qtp1654589030-3924) [   x:nextant] o.a.s.s.HttpSolrCall null:java.lang.RuntimeException: java.lang.NoClassDefFoundError: com/uwyn/jhighlight/renderer/XhtmlRendererFactory
        at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:605)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:474)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
        at org.eclipse.jetty.server.Server.handle(Server.java:518)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
        at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: com/uwyn/jhighlight/renderer/XhtmlRendererFactory
        at org.apache.tika.parser.code.SourceCodeParser.getRenderer(SourceCodeParser.java:132)
        at org.apache.tika.parser.code.SourceCodeParser.parse(SourceCodeParser.java:111)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
        at org.eclipse.jetty.server.Server.handle(Server.java:518)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
        at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: com.uwyn.jhighlight.renderer.XhtmlRendererFactory
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:814)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 57 more
151784 INFO  (qtp1654589030-3924) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_60596&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=0
208882 INFO  (qtp1654589030-5609) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system params={q=*:*&json.nl=flat&omitHeader=true&fl=*,score&start=0&rows=10&wt=json} status=0 QTime=209
210233 INFO  (qtp1654589030-6962) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_51482&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=1
215275 INFO  (qtp1654589030-6967) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system params={q=*:*&json.nl=flat&omitHeader=true&fl=*,score&start=0&rows=10&wt=json} status=0 QTime=37
215764 INFO  (qtp1654589030-6967) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_60588&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=0
221884 INFO  (qtp1654589030-6962) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system params={q=*:*&json.nl=flat&omitHeader=true&fl=*,score&start=0&rows=10&wt=json} status=0 QTime=928
221894 INFO  (qtp1654589030-6668) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_51129&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=0
227242 INFO  (qtp1654589030-6962) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system params={q=*:*&json.nl=flat&omitHeader=true&fl=*,score&start=0&rows=10&wt=json} status=0 QTime=76
228007 INFO  (qtp1654589030-7656) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_60599&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=0
233502 INFO  (qtp1654589030-6962) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system params={q=*:*&json.nl=flat&omitHeader=true&fl=*,score&start=0&rows=10&wt=json} status=0 QTime=487
234239 INFO  (qtp1654589030-8388) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_51555&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=1
239291 INFO  (qtp1654589030-8014) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system params={q=*:*&json.nl=flat&omitHeader=true&fl=*,score&start=0&rows=10&wt=json} status=0 QTime=48
239387 INFO  (qtp1654589030-8351) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_50963&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=0
244683 INFO  (qtp1654589030-8351) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_51423&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=0
254377 INFO  (qtp1654589030-8806) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system params={q=*:*&json.nl=flat&omitHeader=true&fl=*,score&start=0&rows=10&wt=json} status=0 QTime=4650
258703 INFO  (qtp1654589030-8806) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_60585&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=0
265056 INFO  (qtp1654589030-8695) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system params={q=*:*&json.nl=flat&omitHeader=true&fl=*,score&start=0&rows=10&wt=json} status=0 QTime=901
267222 INFO  (qtp1654589030-8979) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_51329&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=0
276175 INFO  (qtp1654589030-9693) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system params={q=*:*&json.nl=flat&omitHeader=true&fl=*,score&start=0&rows=10&wt=json} status=0 QTime=2815
281027 INFO  (qtp1654589030-10456) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_51293&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=0
288032 INFO  (qtp1654589030-10518) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system params={q=*:*&json.nl=flat&omitHeader=true&fl=*,score&start=0&rows=10&wt=json} status=0 QTime=1408
288049 INFO  (qtp1654589030-10518) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_51522&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=0
293864 INFO  (qtp1654589030-10518) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_51611&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=0
300191 INFO  (qtp1654589030-11285) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system params={q=*:*&json.nl=flat&omitHeader=true&fl=*,score&start=0&rows=10&wt=json} status=0 QTime=68
301315 INFO  (qtp1654589030-11462) [   x:nextant] o.a.s.c.S.Request [nextant]  webapp=/solr path=/select params={q=id:files_50974&json.nl=flat&omitHeader=true&fl=nextant_mtime&start=0&rows=10&wt=json} hits=0 status=0 QTime=0
311603 INFO  (qtp1654589030-11747) [   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system params={q=*:*&json.nl=flat&omitHeader=true&fl=*,score&start=0&rows=10&wt=json} status=0 QTime=4700
2035127 INFO  (qtp1654589030-8014) [   x:nextant] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
2035127 INFO  (qtp1654589030-8014) [   x:nextant] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@bfee7cb
2037916 INFO  (qtp1654589030-8014) [   x:nextant] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
        commit{dir=NRTCachingDirectory(MMapDirectory@/home/martin/solr/solr-6.2.1/server/solr/nextant/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3e377ce2; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4cb,generation=5627}
        commit{dir=NRTCachingDirectory(MMapDirectory@/home/martin/solr/solr-6.2.1/server/solr/nextant/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3e377ce2; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4cc,generation=5628}
2037916 INFO  (qtp1654589030-8014) [   x:nextant] o.a.s.c.SolrDeletionPolicy newest commit generation = 5628
2037939 INFO  (qtp1654589030-8014) [   x:nextant] o.a.s.s.SolrIndexSearcher Opening [Searcher@6b4c941c[nextant] main]
2037980 INFO  (qtp1654589030-8014) [   x:nextant] o.a.s.u.DirectUpdateHandler2 end_commit_flush
2039780 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@6b4c941c[nextant] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2hs(6.2.1):C2908) Uninverting(_3rp(6.2.1):C1486) Uninverting(_470(6.2.1):C501) Uninverting(_4hd(6.2.1):C336) Uninverting(_4sr(6.2.1):c379) Uninverting(_4nt(6.2.1):C1) Uninverting(_4q7(6.2.1):C1) Uninverting(_4tw(6.2.1):c41) Uninverting(_4tv(6.2.1):C1) Uninverting(_4tx(6.2.1):C2) Uninverting(_4ty(6.2.1):C1) Uninverting(_4tz(6.2.1):C1) Uninverting(_4u0(6.2.1):C1) Uninverting(_4u1(6.2.1):C1) Uninverting(_4u2(6.2.1):C1)))}
2039780 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.QuerySenderListener QuerySenderListener done.
2039781 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.SolrCore [nextant] Registered new searcher Searcher@6b4c941c[nextant] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2hs(6.2.1):C2908) Uninverting(_3rp(6.2.1):C1486) Uninverting(_470(6.2.1):C501) Uninverting(_4hd(6.2.1):C336) Uninverting(_4sr(6.2.1):c379) Uninverting(_4nt(6.2.1):C1) Uninverting(_4q7(6.2.1):C1) Uninverting(_4tw(6.2.1):c41) Uninverting(_4tv(6.2.1):C1) Uninverting(_4tx(6.2.1):C2) Uninverting(_4ty(6.2.1):C1) Uninverting(_4tz(6.2.1):C1) Uninverting(_4u0(6.2.1):C1) Uninverting(_4u1(6.2.1):C1) Uninverting(_4u2(6.2.1):C1)))}
2039786 INFO  (qtp1654589030-8014) [   x:nextant] o.a.s.u.p.LogUpdateProcessorFactory [nextant]  webapp=/solr path=/update/extract params={fmap.p=ignored_&fmap.h2=ignored_&fmap.style=ignored_&fmap.h3=ignored_&resource.name=com.google.android.apps.inputmethod.hindi-e3619f527611a236c5abe7af354c4573.apk.gz.d1466694607&literal.id=files_50963&fmap.h1=ignored_&commit=true&fmap.link=ignored_&fmap.ul=ignored_&uprefix=nextant_attr_&fmap.form=ignored_&fmap.i=ignored_&fmap.media_white_point=ignored_&fmap.img=ignored_&literal.nextant_path=/devzero/files_trashbin/files/com.google.android.apps.inputmethod.hindi-e3619f527611a236c5abe7af354c4573.apk.gz.d1466694607&fmap.a=ignored_&fmap.b=ignored_&fmap.table=ignored_&fmap.content=text&fmap.script=ignored_&literal.nextant_owner=devzero&fmap.html=ignored_&fmap.div=ignored_&ignoreTikaException=true&wt=json&literal.nextant_mtime=1466587191&json.nl=flat&fmap.media_black_point=ignored_&extractOnly=false&literal.nextant_source=files&fmap.td=ignored_&fmap.tr=ignored_&fmap.li=ignored_&omitHeader=true&literal.nextant_deleted=true&captureAttr=true&fmap.input=ignored_&fmap.span=ignored_}{add=[files_50963 (1549366837555232768)],commit=} 0 1800381
2193748 INFO  (qtp1654589030-10456) [   x:nextant] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
2193748 INFO  (qtp1654589030-10456) [   x:nextant] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@bfee7cb
2195604 INFO  (qtp1654589030-10456) [   x:nextant] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
        commit{dir=NRTCachingDirectory(MMapDirectory@/home/martin/solr/solr-6.2.1/server/solr/nextant/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3e377ce2; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4cc,generation=5628}
        commit{dir=NRTCachingDirectory(MMapDirectory@/home/martin/solr/solr-6.2.1/server/solr/nextant/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3e377ce2; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4cd,generation=5629}
2195604 INFO  (qtp1654589030-10456) [   x:nextant] o.a.s.c.SolrDeletionPolicy newest commit generation = 5629
2195715 INFO  (qtp1654589030-10456) [   x:nextant] o.a.s.s.SolrIndexSearcher Opening [Searcher@4b845258[nextant] main]
2195738 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@4b845258[nextant] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2hs(6.2.1):C2908) Uninverting(_3rp(6.2.1):C1486) Uninverting(_470(6.2.1):C501) Uninverting(_4hd(6.2.1):C336) Uninverting(_4sr(6.2.1):c379) Uninverting(_4nt(6.2.1):C1) Uninverting(_4q7(6.2.1):C1) Uninverting(_4tw(6.2.1):c41) Uninverting(_4tv(6.2.1):C1) Uninverting(_4tx(6.2.1):C2) Uninverting(_4ty(6.2.1):C1) Uninverting(_4tz(6.2.1):C1) Uninverting(_4u0(6.2.1):C1) Uninverting(_4u1(6.2.1):C1) Uninverting(_4u2(6.2.1):C1) Uninverting(_4u3(6.2.1):C1)))}
2195738 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.QuerySenderListener QuerySenderListener done.
2195738 INFO  (searcherExecutor-7-thread-1-processing-x:nextant) [   x:nextant] o.a.s.c.SolrCore [nextant] Registered new searcher Searcher@4b845258[nextant] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2hs(6.2.1):C2908) Uninverting(_3rp(6.2.1):C1486) Uninverting(_470(6.2.1):C501) Uninverting(_4hd(6.2.1):C336) Uninverting(_4sr(6.2.1):c379) Uninverting(_4nt(6.2.1):C1) Uninverting(_4q7(6.2.1):C1) Uninverting(_4tw(6.2.1):c41) Uninverting(_4tv(6.2.1):C1) Uninverting(_4tx(6.2.1):C2) Uninverting(_4ty(6.2.1):C1) Uninverting(_4tz(6.2.1):C1) Uninverting(_4u0(6.2.1):C1) Uninverting(_4u1(6.2.1):C1) Uninverting(_4u2(6.2.1):C1) Uninverting(_4u3(6.2.1):C1)))}
2195738 INFO  (qtp1654589030-10456) [   x:nextant] o.a.s.u.DirectUpdateHandler2 end_commit_flush
2195746 INFO  (qtp1654589030-10456) [   x:nextant] o.a.s.u.p.LogUpdateProcessorFactory [nextant]  webapp=/solr path=/update/extract params={fmap.p=ignored_&fmap.h2=ignored_&fmap.style=ignored_&fmap.h3=ignored_&resource.name=com.noinnion.android.greader.readerpro-20151020-111205+(1).TiBkp.d1472385757&literal.id=files_51293&fmap.h1=ignored_&commit=true&fmap.link=ignored_&fmap.ul=ignored_&uprefix=nextant_attr_&fmap.form=ignored_&fmap.i=ignored_&fmap.media_white_point=ignored_&fmap.img=ignored_&literal.nextant_path=/devzero/files_trashbin/files/com.noinnion.android.greader.readerpro-20151020-111205+(1).TiBkp.d1472385757&fmap.a=ignored_&fmap.b=ignored_&fmap.table=ignored_&fmap.content=text&fmap.script=ignored_&literal.nextant_owner=devzero&fmap.html=ignored_&fmap.div=ignored_&ignoreTikaException=true&wt=json&literal.nextant_mtime=1466588367&json.nl=flat&fmap.media_black_point=ignored_&extractOnly=false&literal.nextant_source=files&fmap.td=ignored_&fmap.tr=ignored_&fmap.li=ignored_&omitHeader=true&literal.nextant_deleted=true&captureAttr=true&fmap.input=ignored_&fmap.span=ignored_}{add=[files_51293 (1549367005100900352)],commit=} 0 1914716
#
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError="/home/martin/solr/solr-6.2.1/bin/oom_solr.sh 8983 /home/martin/solr/solr-6.2.1/server/logs"
#   Executing /bin/sh -c "/home/martin/solr/solr-6.2.1/bin/oom_solr.sh 8983 /home/martin/solr/solr-6.2.1/server/logs"...
Running OOM killer script for process 9492 for Solr on port 8983
Killed process 9492

i check your next thing

With the other patch it worked.
The question is because it already went that far as it were, or not:

  90 file(s) processed ; 4425 extracted documents ; 0 orphan(s) removed
  4 file(s) were not processed (failure)

@ArtificialOwl
Copy link
Member Author

@ArtificialOwl ArtificialOwl commented on c11986e Oct 27, 2016 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it seems that you have some apk (android) file in your trashbin.

When you're deleting a file in NC, you're loosing the mimetype (from the database, fileinfo will return application/octet-stream).
So, nextant will index your file from the trashbin whatever its content. And when Tika (Solr) encounter some Java (or Android, in our case), it will try to read it using an external source highlighter (which is not installed on your box):
http://freecode.com/projects/jhighlight

The other patch will check differently the mimetype if it is a deleted file. Meaning that your apk will not be indexed.

@devz3r0
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes i am backuping my android devices via titanium backup to my nc.
Thanks for your quick fixes :)

Please sign in to comment.