Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] [rest-api]/hazelcast/rest/maps/submit-job java.lang.NullPointerException: null #6530

Closed
3 tasks done
sandyustc opened this issue Mar 18, 2024 · 4 comments
Closed
3 tasks done
Labels

Comments

@sandyustc
Copy link

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

/hazelcast/rest/maps/submit-job , It‘s error when It is running on Postman .

An error occurred while handling request HttpCommand [HTTP_POST]{uri='/hazelcast/rest/maps/submit-job?jobId=2&jobName=job006&isStartWithSavePoint=false'}AbstractTextCommand[HTTP_POST]{requestId=12}
java.lang.NullPointerException: null

SeaTunnel Version

2.3.3

SeaTunnel Config

{
	"env": {
		"parallelism": 4,
		"job.mode": "batch",
		"checkpoint.interval": 10000
	},
	"source": [{
		"jdbc": {
			"url": "jdbc:oracle:thin:@*.*.*.*:1521:orcl",
			"driver": "oracle.jdbc.OracleDriver",
			"connection_check_timeout_sec": 100,
			"user": "xtgg",
			"password": "xtgg",
			"query": "select * from  user",
			"partition_column": "id",
			"split.size": 10000
		}
	}],
	"sink": [{
		"jdbc": {
			"url": "jdbc:postgresql://*.*.*.*:5432/testDb",
			"driver": "org.postgresql.Driver",
			"user": "test",
			"password": "test",
			"generate_sink_sql": true,
			"database": "testDb",
			"table": "public.user",
			"schema_save_mode": "CREATE_SCHEMA_WHEN_NOT_EXIST",
			"data_save_mode": "APPEND_DATA",
			"primary_keys": "id"

		}
	}]
}

Running Command

/hazelcast/rest/maps/submit-job

Error Exception

2024-03-18 18:11:43,946 INFO  org.apache.seatunnel.core.starter.utils.ConfigBuilder - Loading config file from objectMap
2024-03-18 18:11:43,947 INFO  org.apache.seatunnel.engine.server.job.JobImmutableInformationEnv - add common jar in plugins :[file:/opt/seatunnel-2.3.3/backend/apache-seatunnel-2.3.3/plugins/postgresql-42.7.2.jar, file:/opt/seatunnel-2.3.3/backend/apache-seatunnel-2.3.3/plugins/ojdbc8-23.3.0.23.09.jar, file:/opt/seatunnel-2.3.3/backend/apache-seatunnel-2.3.3/plugins/jdbc/lib/postgresql-42.7.2.jar]
2024-03-18 18:11:43,948 INFO  org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load SeaTunnelSink Plugin from /opt/seatunnel-2.3.3/backend/apache-seatunnel-2.3.3/connectors/seatunnel
2024-03-18 18:11:43,948 WARN  org.apache.seatunnel.engine.server.log.Log4j2HttpPostCommandProcessor - [localhost]:5801 [seatunnel] [5.1] An error occurred while handling request HttpCommand [HTTP_POST]{uri='/hazelcast/rest/maps/submit-job?jobId=2&jobName=job006&isStartWithSavePoint=false'}AbstractTextCommand[HTTP_POST]{requestId=12}
java.lang.NullPointerException: null
	at org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery.findPluginJarPath(AbstractPluginDiscovery.java:341) ~[seatunnel-starter.jar:2.3.3]
	at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) ~[?:1.8.0_401]
	at org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery.getPluginJarPath(AbstractPluginDiscovery.java:319) ~[seatunnel-starter.jar:2.3.3]
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_401]
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) ~[?:1.8.0_401]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_401]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_401]
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_401]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_401]
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_401]
	at org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery.getPluginJarPaths(AbstractPluginDiscovery.java:131) ~[seatunnel-starter.jar:2.3.3]
	at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.getConnectorJarList(MultipleTableJobConfigParser.java:227) ~[seatunnel-starter.jar:2.3.3]
	at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:160) ~[seatunnel-starter.jar:2.3.3]
	at org.apache.seatunnel.engine.core.job.AbstractJobEnvironment.getLogicalDag(AbstractJobEnvironment.java:109) ~[seatunnel-starter.jar:2.3.3]
	at org.apache.seatunnel.engine.server.job.JobImmutableInformationEnv.build(JobImmutableInformationEnv.java:76) ~[seatunnel-starter.jar:2.3.3]
	at org.apache.seatunnel.engine.server.rest.RestHttpPostCommandProcessor.handleSubmitJob(RestHttpPostCommandProcessor.java:112) ~[seatunnel-starter.jar:2.3.3]
	at org.apache.seatunnel.engine.server.rest.RestHttpPostCommandProcessor.handle(RestHttpPostCommandProcessor.java:68) ~[seatunnel-starter.jar:2.3.3]
	at org.apache.seatunnel.engine.server.rest.RestHttpPostCommandProcessor.handle(RestHttpPostCommandProcessor.java:47) ~[seatunnel-starter.jar:2.3.3]
	at com.hazelcast.internal.ascii.TextCommandServiceImpl$CommandExecutor.run(TextCommandServiceImpl.java:402) ~[seatunnel-starter.jar:2.3.3]
	at com.hazelcast.internal.util.executor.CachedExecutorServiceDelegate$Worker.run(CachedExecutorServiceDelegate.java:217) ~[seatunnel-starter.jar:2.3.3]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_401]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_401]
	at java.lang.Thread.run(Thread.java:750) [?:1.8.0_401]
	at com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76) ~[seatunnel-starter.jar:2.3.3]
	at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) ~[seatunnel-starter.jar:2.3.3]

Zeta or Flink or Spark Version

No response

Java or Scala Version

java 1.8

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@sandyustc sandyustc added the bug label Mar 18, 2024
@liunaijie
Copy link
Contributor

hi, when you use api to submit job, the config shoule be json format. can you provide the json config you posted, thx.

@sandyustc
Copy link
Author

sandyustc commented Mar 19, 2024

hi, when you use api to submit job, the config shoule be json format. can you provide the json config you posted, thx.

@ liunaijie thanks.

I've submitted it as json.

{
"env": {
"parallelism": 4,
"job.mode": "batch",
"checkpoint.interval": 10000
},
"source": [{
"jdbc": {
"url": "jdbc:oracle:thin:@...:1521:orcl",
"driver": "oracle.jdbc.OracleDriver",
"connection_check_timeout_sec": 100,
"user": "xtgg",
"password": "xtgg",
"query": "select * from user",
"partition_column": "id",
"split.size": 10000
}
}],
"sink": [{
"jdbc": {
"url": "jdbc:postgresql://...:5432/testDb",
"driver": "org.postgresql.Driver",
"user": "test",
"password": "test",
"generate_sink_sql": true,
"database": "testDb",
"table": "public.user",
"schema_save_mode": "CREATE_SCHEMA_WHEN_NOT_EXIST",
"data_save_mode": "APPEND_DATA",
"primary_keys": "id"

	}
}]

}

@liunaijie
Copy link
Contributor

@sandyustc the config format is not correct, please refer https://seatunnel.apache.org/docs/2.3.4/seatunnel-engine/rest-api#submit-job.
if you use api, the config is not like

source {
  jdbc
}

it should be

source [{
  "plugin_name" : "jdbc"
}]

You can update the config and retry.

@sandyustc
Copy link
Author

sandyustc commented Mar 19, 2024

@sandyustc the config format is not correct, please refer https://seatunnel.apache.org/docs/2.3.4/seatunnel-engine/rest-api#submit-job. if you use api, the config is not like

source {
  jdbc
}

it should be

source [{
  "plugin_name" : "jdbc"
}]

You can update the config and retry.
@liunaijie
I'll try it , It is running Ok ,thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants