Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Getting exception during process of Word Count example #319

Closed
mukul4u2005 opened this issue Feb 15, 2017 · 3 comments
Closed

Getting exception during process of Word Count example #319

mukul4u2005 opened this issue Feb 15, 2017 · 3 comments
Labels
defect
Milestone
0.3.1

Comments

@mukul4u2005
Copy link

Hi

Just trying with some example on my local and I found that there is a issue when we try to configure the Management center with Hazelcast Config and Jet Config as:

Config hzCfg = new Config();
	hzCfg.setManagementCenterConfig(new ManagementCenterConfig("http://localhost:8082/mancenter", 3).setEnabled(true));
    JetConfig cfg = new JetConfig();
    cfg.setInstanceConfig(new InstanceConfig().setCooperativeThreadCount(
            Math.max(1, getRuntime().availableProcessors() / 2)));
    cfg.setHazelcastConfig(hzCfg);

I tried with WordCount example available in Sample repository of Hazelcast Jet.

Creating Jet instance 1
Feb 15, 2017 1:06:28 PM com.hazelcast.instance.DefaultAddressPicker
INFO: [LOCAL] [dev] [0.3] [3.8-RC1] Prefer IPv4 stack is true.
Feb 15, 2017 1:06:29 PM com.hazelcast.instance.DefaultAddressPicker
INFO: [LOCAL] [dev] [0.3] [3.8-RC1] Picked [10.162.55.6]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
Feb 15, 2017 1:06:29 PM com.hazelcast.system
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Hazelcast 3.8-RC1 (20170201 - 609ebbc) starting at [10.162.55.6]:5701
Feb 15, 2017 1:06:29 PM com.hazelcast.system
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.
Feb 15, 2017 1:06:29 PM com.hazelcast.system
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Configured Hazelcast Serialization version : 1
Feb 15, 2017 1:06:29 PM com.hazelcast.spi.impl.operationservice.impl.BackpressureRegulator
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Backpressure is disabled
Feb 15, 2017 1:06:31 PM com.hazelcast.instance.Node
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Creating MulticastJoiner
Feb 15, 2017 1:06:31 PM com.hazelcast.jet.impl.JetService
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Starting Jet 0.3 (20170206 - 5c8738d)
Feb 15, 2017 1:06:31 PM com.hazelcast.jet.impl.JetService
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Setting number of cooperative threads and default parallelism to 2
Feb 15, 2017 1:06:31 PM com.hazelcast.jet.impl.JetService
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1]
o o o o---o o---o o o---o o o---o o-o-o o o---o o-o-o
| | / \ / | | | / \ | | | | |
o---o o---o o o-o | o o---o o---o | | o-o |
| | | | / | | | | | | | \ | | |
o o o o o---o o---o o---o o---o o o o---o o o--o o---o o
Feb 15, 2017 1:06:31 PM com.hazelcast.jet.impl.JetService
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Copyright (c) 2008-2017, Hazelcast, Inc. All Rights Reserved.
Feb 15, 2017 1:06:31 PM com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Starting 4 partition threads
Feb 15, 2017 1:06:31 PM com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Starting 3 generic threads (1 dedicated for priority tasks)
Feb 15, 2017 1:06:31 PM com.hazelcast.core.LifecycleService
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] [10.162.55.6]:5701 is STARTING
Feb 15, 2017 1:06:34 PM com.hazelcast.system
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Cluster version set to 3.8
Feb 15, 2017 1:06:34 PM com.hazelcast.internal.cluster.impl.MulticastJoiner
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1]

Members [1] {
Member [10.162.55.6]:5701 - d8f937ea-a992-458b-b172-f280f62f77a7 this
}

Feb 15, 2017 1:06:35 PM com.hazelcast.internal.management.ManagementCenterService
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Hazelcast will connect to Hazelcast Management Center on address:
http://localhost:8082/mancenter
Creating Jet instance 2
Feb 15, 2017 1:06:35 PM com.hazelcast.core.LifecycleService
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] [10.162.55.6]:5701 is STARTED
Feb 15, 2017 1:06:35 PM com.hazelcast.instance.DefaultAddressPicker
INFO: [LOCAL] [dev] [0.3] [3.8-RC1] Prefer IPv4 stack is true.
Feb 15, 2017 1:06:36 PM com.hazelcast.instance.DefaultAddressPicker
INFO: [LOCAL] [dev] [0.3] [3.8-RC1] Picked [10.162.55.6]:5703, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5703], bind any local is true
Feb 15, 2017 1:06:36 PM com.hazelcast.system
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Hazelcast 3.8-RC1 (20170201 - 609ebbc) starting at [10.162.55.6]:5703
Feb 15, 2017 1:06:36 PM com.hazelcast.system
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.
Feb 15, 2017 1:06:36 PM com.hazelcast.system
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Configured Hazelcast Serialization version : 1
Feb 15, 2017 1:06:36 PM com.hazelcast.spi.impl.operationservice.impl.BackpressureRegulator
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Backpressure is disabled
Feb 15, 2017 1:06:36 PM com.hazelcast.internal.partition.impl.PartitionStateManager
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Initializing cluster partition table arrangement...
Feb 15, 2017 1:06:38 PM com.hazelcast.instance.Node
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Creating MulticastJoiner
Feb 15, 2017 1:06:38 PM com.hazelcast.jet.impl.JetService
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Starting Jet 0.3 (20170206 - 5c8738d)
Feb 15, 2017 1:06:38 PM com.hazelcast.jet.impl.JetService
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Setting number of cooperative threads and default parallelism to 2
Feb 15, 2017 1:06:38 PM com.hazelcast.jet.impl.JetService
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1]
o o o o---o o---o o o---o o o---o o-o-o o o---o o-o-o
| | / \ / | | | / \ | | | | |
o---o o---o o o-o | o o---o o---o | | o-o |
| | | | / | | | | | | | \ | | |
o o o o o---o o---o o---o o---o o o o---o o o--o o---o o
Feb 15, 2017 1:06:38 PM com.hazelcast.jet.impl.JetService
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Copyright (c) 2008-2017, Hazelcast, Inc. All Rights Reserved.
Feb 15, 2017 1:06:38 PM com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Starting 4 partition threads
Feb 15, 2017 1:06:38 PM com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Starting 3 generic threads (1 dedicated for priority tasks)
Feb 15, 2017 1:06:38 PM com.hazelcast.core.LifecycleService
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] [10.162.55.6]:5703 is STARTING
Feb 15, 2017 1:06:38 PM com.hazelcast.internal.cluster.impl.MulticastJoiner
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Trying to join to discovered node: [10.162.55.6]:5701
Feb 15, 2017 1:06:38 PM com.hazelcast.nio.tcp.InitConnectionTask
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Connecting to /10.162.55.6:5701, timeout: 0, bind-any: true
Feb 15, 2017 1:06:38 PM com.hazelcast.nio.tcp.SocketAcceptorThread
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Accepting socket connection from /10.162.55.6:61739
Feb 15, 2017 1:06:38 PM com.hazelcast.nio.tcp.TcpIpConnectionManager
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Established socket connection between /10.162.55.6:61739 and /10.162.55.6:5701
Feb 15, 2017 1:06:38 PM com.hazelcast.nio.tcp.TcpIpConnectionManager
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Established socket connection between /10.162.55.6:5701 and /10.162.55.6:61739
Feb 15, 2017 1:06:44 PM com.hazelcast.internal.cluster.ClusterService
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1]

Members [2] {
Member [10.162.55.6]:5701 - d8f937ea-a992-458b-b172-f280f62f77a7 this
Member [10.162.55.6]:5703 - 4c910316-302b-4a1d-89ca-40ac9e065468
}

Feb 15, 2017 1:06:44 PM com.hazelcast.system
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Cluster version set to 3.8
Feb 15, 2017 1:06:44 PM com.hazelcast.internal.cluster.ClusterService
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1]

Members [2] {
Member [10.162.55.6]:5701 - d8f937ea-a992-458b-b172-f280f62f77a7
Member [10.162.55.6]:5703 - 4c910316-302b-4a1d-89ca-40ac9e065468 this
}

Feb 15, 2017 1:06:44 PM com.hazelcast.internal.partition.impl.MigrationManager
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Re-partitioning cluster data... Migration queue size: 271
Feb 15, 2017 1:06:46 PM com.hazelcast.instance.Node
WARNING: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Config seed port is 5701 and cluster size is 2. Some of the ports seem occupied!
Feb 15, 2017 1:06:46 PM com.hazelcast.internal.management.ManagementCenterService
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Hazelcast will connect to Hazelcast Management Center on address:
http://localhost:8082/mancenter
Feb 15, 2017 1:06:46 PM com.hazelcast.core.LifecycleService
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] [10.162.55.6]:5703 is STARTED
These books will be analyzed:
adventures-of-sherlock-holmes.txt
anderson-fairy-tales.txt
anna-karenina.txt
around-the-world-in-eighty-days.txt
a-tale-of-two-cities.txt
awakening.txt
benjamin-franklin-autobiography.txt
beyond-good-and-evil.txt
brothers-karamazov.txt
canterbury-tales.txt
clarissa-harlowe.txt
confessions-of-st-augustine.txt
connecticut-yankee.txt
crime-punishment.txt
critique-of-pure-reason.txt
david-copperfield.txt
decameron.txt
divine-comedy.txt
don-juan.txt
don-quixote.txt
dorian-gray.txt
dracula.txt
edgar-allan-poe-works-1.txt
edgar-allan-poe-works-2.txt
edgar-allan-poe-works-3.txt
emma.txt
faust.txt
frankenstein.txt
gilded-age.txt
great-expectations.txt
grimm-brothers.txt
gullivers-travels.txt
guy-de-maupassant-shorts.txt
hard-times.txt
heart-of-darkness.txt
history-of-peloponnesian-war.txt
hound-of-the-baskervilles.txt
huckleberry-finn.txt
iliad.txt
ivanhoe.txt
king-james-bible.txt
land-of-oz.txt
leaves-of-grass.txt
les-miserables.txt
leviathan.txt
little-women.txt
madame-bovary.txt
man-who-was-thursday.txt
mark-twain-mysterious-strangers.txt
meditations.txt
michel-montaigne-essays.txt
middlemarch.txt
moby-dick.txt
monte-cristo.txt
my-secret-life.txt
north-and-south.txt
notebooks-of-leonardo-da-vinci.txt
odyssey.txt
olaudah-equiano.txt
oliver-twist.txt
origin-of-species.txt
paradise-lost.txt
peter-pan.txt
portrait-of-artist-as-a-young-man.txt
practice-and-science-of-drawing.txt
pride-and-prejudice.txt
problems-of-philosophy.txt
relativity-special-and-general-theory.txt
return-of-sherlock-holmes.txt
room-with-a-view.txt
satyricon.txt
scarlet-letter.txt
sense-sensibility.txt
shakespeare-complete-works.txt
short-history-of-world.txt
siddharta.txt
sorrows-of-young-werther.txt
souls-of-black-folk.txt
swanns-way.txt
the-30000-dollar-request.txt
the-jungle.txt
the-republic.txt
the-trial.txt
three-musketeers.txt
through-the-looking-glass.txt
thus-spake-zarathustra.txt
time-machine.txt
tom-sawyer.txt
ulysses.txt
uncle-toms-cabin.txt
utopia.txt
walden-and-duty-of-civil-disobedience.txt
war-and-peace.txt
war-of-the-worlds.txt
wealth-of-nations.txt
what-is-man.txt
wuthering-heights.txt

Counting words... Feb 15, 2017 1:06:46 PM com.hazelcast.jet.impl.operation.ExecuteJobOperation
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Start executing job 0.
Feb 15, 2017 1:06:46 PM com.hazelcast.jet.impl.operation.InitOperation
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Initializing execution plan for job 0 from [10.162.55.6]:5701.
Feb 15, 2017 1:06:46 PM com.hazelcast.internal.partition.InternalPartitionService
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Remaining migration tasks in queue => 67
Feb 15, 2017 1:06:46 PM com.hazelcast.jet.impl.operation.InitOperation
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Initializing execution plan for job 0 from [10.162.55.6]:5701.
Feb 15, 2017 1:06:46 PM com.hazelcast.jet.impl.operation.ExecuteJobOperation
SEVERE: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] null
java.lang.NullPointerException
at com.hazelcast.jet.impl.operation.ExecuteJobOperation.doRun(ExecuteJobOperation.java:101)
at com.hazelcast.jet.impl.operation.AsyncExecutionOperation.run(AsyncExecutionOperation.java:56)
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:186)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.run(OperationExecutorImpl.java:373)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.runOrExecute(OperationExecutorImpl.java:400)
at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvokeLocal(Invocation.java:534)
at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvoke(Invocation.java:519)
at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke0(Invocation.java:490)
at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke(Invocation.java:200)
at com.hazelcast.spi.impl.operationservice.impl.InvocationBuilderImpl.invoke(InvocationBuilderImpl.java:59)
at com.hazelcast.jet.impl.JetInstanceImpl$JobImpl.execute(JetInstanceImpl.java:95)
at WordCount.go(WordCount.java:153)
at WordCount.main(WordCount.java:145)

Exception in thread "main" java.util.concurrent.ExecutionException: java.lang.NullPointerException
at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolveAndThrowIfException(InvocationFuture.java:88)
at com.hazelcast.spi.impl.AbstractInvocationFuture.get(AbstractInvocationFuture.java:147)
at WordCount.go(WordCount.java:153)
at WordCount.main(WordCount.java:145)
Caused by: java.lang.NullPointerException
at com.hazelcast.jet.impl.operation.ExecuteJobOperation.doRun(ExecuteJobOperation.java:101)
at com.hazelcast.jet.impl.operation.AsyncExecutionOperation.run(AsyncExecutionOperation.java:56)
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:186)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.run(OperationExecutorImpl.java:373)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.runOrExecute(OperationExecutorImpl.java:400)
at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvokeLocal(Invocation.java:534)
at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvoke(Invocation.java:519)
at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke0(Invocation.java:490)
at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke(Invocation.java:200)
at com.hazelcast.spi.impl.operationservice.impl.InvocationBuilderImpl.invoke(InvocationBuilderImpl.java:59)
at com.hazelcast.jet.impl.JetInstanceImpl$JobImpl.execute(JetInstanceImpl.java:95)
at WordCount.go(WordCount.java:153)
at WordCount.main(WordCount.java:145)
at ------ submitted from ------.(Unknown Source)
at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolve(InvocationFuture.java:114)
at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolveAndThrowIfException(InvocationFuture.java:75)
... 3 more
Feb 15, 2017 1:06:46 PM com.hazelcast.jet.impl.operation.ExecuteOperation
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Start execution of plan for job 0 from caller [10.162.55.6]:5701.
Feb 15, 2017 1:06:46 PM com.hazelcast.jet.impl.operation.ExecuteOperation
INFO: [10.162.55.6]:5703 [dev] [0.3] [3.8-RC1] Start execution of plan for job 0 from caller [10.162.55.6]:5701.
Feb 15, 2017 1:06:46 PM com.hazelcast.jet.impl.operation.ExecuteJobOperation
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] Execution of job 0 completed in 300ms.
Feb 15, 2017 1:06:48 PM com.hazelcast.internal.partition.impl.MigrationThread
INFO: [10.162.55.6]:5701 [dev] [0.3] [3.8-RC1] All migration tasks have been completed, queues are empty.
@cangencer
Copy link
Contributor

It seems to me like there is a migration happening at the same time you're starting to run the job. This issue should have been resolved in 0.3.1, could you try with the version 0.3.1-SNAPSHOT? Alternatively as a temp workaround, you can wait a few seconds after both nodes are started before starting the job execution.

@cangencer cangencer added this to the 0.3.1 milestone Feb 16, 2017
@mukul4u2005
Copy link
Author

mukul4u2005 commented Feb 16, 2017 via email

@cangencer
Copy link
Contributor

This issue should now be fixed for 0.3.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
defect
Projects
None yet
Milestone
0.3.1
Development

No branches or pull requests
2 participants
@cangencer @mukul4u2005