Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New PrimarySearch CASMultiplier causing ConcurrentModificationException #23

Closed
zednis opened this issue Dec 8, 2015 · 2 comments
Closed

New PrimarySearch CASMultiplier causing ConcurrentModificationException #23

zednis opened this issue Dec 8, 2015 · 2 comments

Comments

@zednis
Copy link

zednis commented Dec 8, 2015

I am attempting to add a new data source which references a local ElasticSearch service.

I have created a EsDocPrimarySearch CAS multiplies and a EsDocAnswerProducer based on the current Solr classes and I have added EsDocPrimarySearch AE to the pipeline in YodaQA.java.

You can see the updates in my repo's pull request - #22

edit - link to pull request correctly opened in my fork - zednis#1

When I run the application I get a ConcurrentModificationException after the EsDocPrimarySearch:process method has executed.

Do you have any idea why this might be happening?

Thanks,
Stephan

INFO SolrDocPrimarySearch -  QUERY: +("wrote")^1.0 +("Ender's Game")^1.11  ("wrote Ender's Game "~4)^2.1100000000000003 ("wrote Ender's Game "~12)^1.0550000000000002 ("wrote Ender's Game "~36)^0.5275000000000001
Dec 07, 2015 9:04:38 PM cz.brmlab.yodaqa.pipeline.esdoc.EsDocPrimarySearch process(106)
INFO: querying elasticsearch for: [wrote, Ender's Game]
Dec 07, 2015 9:04:38 PM cz.brmlab.yodaqa.pipeline.esdoc.EsDocPrimarySearch process(117)
INFO: elasticsearch response status: OK
Dec 07, 2015 9:04:38 PM org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl processAndOutputNewCASes(274)
SEVERE: Exception occurred
org.apache.uima.analysis_engine.AnalysisEngineProcessException
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.processUntilNextOutputCas(MultiThreadASB.java:1074)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.<init>(MultiThreadASB.java:496)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB.process(MultiThreadASB.java:416)
    at org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes(AggregateAnalysisEngine_impl.java:266)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator$1.call(MultiThreadASB.java:772)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator$1.call(MultiThreadASB.java:754)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.ConcurrentModificationException
    at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429)
    at java.util.HashMap$KeyIterator.next(HashMap.java:1453)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.nextCasToProcess(MultiThreadASB.java:711)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.processUntilNextOutputCas(MultiThreadASB.java:1039)
    ... 9 more

Dec 07, 2015 9:04:38 PM org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl processAndOutputNewCASes(274)
SEVERE: Exception occurred
org.apache.uima.analysis_engine.AnalysisEngineProcessException
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.processUntilNextOutputCas(MultiThreadASB.java:1074)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.<init>(MultiThreadASB.java:496)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB.process(MultiThreadASB.java:416)
    at org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes(AggregateAnalysisEngine_impl.java:266)
    at cz.brmlab.yodaqa.flow.MultiCASPipeline.runAnalysisEngines(MultiCASPipeline.java:135)
    at cz.brmlab.yodaqa.flow.MultiCASPipeline.runPipeline(MultiCASPipeline.java:95)
    at cz.brmlab.yodaqa.YodaQA_Web.main(YodaQA_Web.java:45)
Caused by: java.util.concurrent.ExecutionException: org.apache.uima.analysis_engine.AnalysisEngineProcessException
    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.util.concurrent.FutureTask.get(FutureTask.java:192)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.collectCasInFlow(MultiThreadASB.java:814)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.casInFlowFromFuture(MultiThreadASB.java:603)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.nextCasToProcess(MultiThreadASB.java:635)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.processUntilNextOutputCas(MultiThreadASB.java:1039)
    ... 6 more
Caused by: org.apache.uima.analysis_engine.AnalysisEngineProcessException
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.processUntilNextOutputCas(MultiThreadASB.java:1074)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.<init>(MultiThreadASB.java:496)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB.process(MultiThreadASB.java:416)
    at org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes(AggregateAnalysisEngine_impl.java:266)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator$1.call(MultiThreadASB.java:772)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator$1.call(MultiThreadASB.java:754)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.ConcurrentModificationException
    at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429)
    at java.util.HashMap$KeyIterator.next(HashMap.java:1453)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.nextCasToProcess(MultiThreadASB.java:711)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.processUntilNextOutputCas(MultiThreadASB.java:1039)
    ... 9 more

Exception in thread "main" org.apache.uima.analysis_engine.AnalysisEngineProcessException
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.processUntilNextOutputCas(MultiThreadASB.java:1074)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.<init>(MultiThreadASB.java:496)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB.process(MultiThreadASB.java:416)
    at org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes(AggregateAnalysisEngine_impl.java:266)
    at cz.brmlab.yodaqa.flow.MultiCASPipeline.runAnalysisEngines(MultiCASPipeline.java:135)
    at cz.brmlab.yodaqa.flow.MultiCASPipeline.runPipeline(MultiCASPipeline.java:95)
    at cz.brmlab.yodaqa.YodaQA_Web.main(YodaQA_Web.java:45)
Caused by: java.util.concurrent.ExecutionException: org.apache.uima.analysis_engine.AnalysisEngineProcessException
    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.util.concurrent.FutureTask.get(FutureTask.java:192)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.collectCasInFlow(MultiThreadASB.java:814)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.casInFlowFromFuture(MultiThreadASB.java:603)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.nextCasToProcess(MultiThreadASB.java:635)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.processUntilNextOutputCas(MultiThreadASB.java:1039)
    ... 6 more
Caused by: org.apache.uima.analysis_engine.AnalysisEngineProcessException
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.processUntilNextOutputCas(MultiThreadASB.java:1074)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.<init>(MultiThreadASB.java:496)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB.process(MultiThreadASB.java:416)
    at org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes(AggregateAnalysisEngine_impl.java:266)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator$1.call(MultiThreadASB.java:772)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator$1.call(MultiThreadASB.java:754)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.ConcurrentModificationException
    at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429)
    at java.util.HashMap$KeyIterator.next(HashMap.java:1453)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.nextCasToProcess(MultiThreadASB.java:711)
    at cz.brmlab.yodaqa.flow.asb.MultiThreadASB$AggregateCasIterator.processUntilNextOutputCas(MultiThreadASB.java:1039)
    ... 9 more
@zednis
Copy link
Author

zednis commented Dec 9, 2015

I found several small issues with by branch and after fixing them I am no longer getting exceptions and the application appears to be working normally.

I am not 100% sure which of the bugs in my code were causing this issue, but I did resolve the following problems

  1. CAS Multiplier next( ) now returning empty JCas instead of null when there are no hits from ES
  2. "empty answer" CAS now correctly has AnswerInfo and ResultInfo being added to indexes
  3. Now using static index during next( ) processing like in SolrDocPrimarySearch

@zednis zednis closed this as completed Dec 9, 2015
@pasky
Copy link
Member

pasky commented Dec 18, 2015

Hi! Sorry I couldn't have been of more help! :-) Hope things work well now. If you are sometime in the mood for talking about what you are working on, I'd love to hear that! (By email, IRC, or here.) Or brainstorm how to improve accuracy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@pasky @zednis