Spark Job Server serving old jars

[chitenderkumar]

Used Spark version: v1.6.2

Used Spark Job Server version: v0.6.2

Deployed mode: Yarn

we are facing an issue where spark job server start serving the old jars for job execution. for example i have below jars at my filedao directory for Test
test-1.jar
test-2.jar
test-3.jar

and test-3.jar is the latest one, which should be picked by spark job-server for job execution. but sometimes it start picking up the test-2.jar or test-1.jar for job execution.

to fix this we manually remove the older jars from filedao path and restart the spark job server.

looking forward for suggestions/comments to deep dive into the issue and have the permanent fix.

[bsikander]

You are using a very old version of jobserver and fileDao is deprecated

[chitenderkumar]

@bsikander thanks for your response.
at this point of time we can not upgrade the spark version. is there any solution apart from upgrade.

[bsikander]

Sure, can you provide some script or testcase so that I can reproduce this problem?

[Nibooor]

Closing because of inactivity