Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[spark-agent]0.4.0 HttpLineageDispatcher.ensureProducerReady is missing the Success case #515

Closed
calvertj opened this issue Dec 7, 2019 · 0 comments
Closed

[spark-agent]0.4.0 HttpLineageDispatcher.ensureProducerReady is missing the Success case #515

calvertj opened this issue Dec 7, 2019 · 0 comments
Assignees
@calvertj
Labels
bug
Milestone
0.4.1

Comments

@calvertj
Copy link

calvertj commented Dec 7, 2019 •
edited
Loading

While trying to initialize the listener:

import za.co.absa.spline.harvester.SparkLineageInitializer._ spark.enableLineageTracking()

an scala.MatchError Exception is thrown
scala.MatchError: Success(true) (of class scala.util.Success)

at za.co.absa.spline.harvester.dispatcher.HttpLineageDispatcher.ensureProducerReady(HttpLineageDispatcher.scala:69) at za.co.absa.spline.harvester.SparkLineageInitializer$SparkSessionWrapper.enableLineageTracking(SparkLineageInitializer.scala:80)

This due to the result of:

val tryStatusOk = Try(Http(statusUrl) .method("HEAD") .asString .isSuccess)

and the match clause which does not have any case for a success.
@calvertj calvertj changed the title [spark-agent] HttpLineageDispatcher.ensureProducerReady is matching on a Try the value is a Boolean [spark-agent]0.4.0 HttpLineageDispatcher.ensureProducerReady is matching on a Try the value is a Boolean Dec 7, 2019
@calvertj calvertj changed the title [spark-agent]0.4.0 HttpLineageDispatcher.ensureProducerReady is matching on a Try the value is a Boolean [spark-agent]0.4.0 HttpLineageDispatcher.ensureProducerReady is missing the Success case Dec 7, 2019
@wajda wajda added this to the 0.4.1 milestone Dec 7, 2019
@wajda wajda mentioned this issue Dec 9, 2019
Closed
cerveada added a commit to calvertj/spline that referenced this issue Dec 10, 2019
@cerveada
spline AbsaOSS#515 add regression test for HttpLineageDispatcher
39ca50c
cerveada added a commit to calvertj/spline that referenced this issue Dec 10, 2019
@cerveada
spline AbsaOSS#515 add regression test for HttpLineageDispatcher
1348b40
cerveada added a commit to calvertj/spline that referenced this issue Dec 11, 2019
@cerveada
spline AbsaOSS#515 improve regression test for HttpLineageDispatcher
bc458c0
cerveada pushed a commit that referenced this issue Dec 11, 2019
@cerveada
spline #515 add missing default in match in HttpLineageDispatcher
7a86db7
cerveada added a commit that referenced this issue Dec 11, 2019
@cerveada
spline #515 add regression test for HttpLineageDispatcher
fc44f79
cerveada pushed a commit that referenced this issue Dec 11, 2019
@cerveada
spline #515 add missing default in match in HttpLineageDispatcher
0110b31
cerveada added a commit that referenced this issue Dec 11, 2019
@cerveada
spline #515 add regression test for HttpLineageDispatcher
498b3b3
@wajda wajda closed this as completed Dec 16, 2019
cerveada added a commit that referenced this issue Dec 19, 2019
@cerveada
spline #515 temp commit fro backup
54c9c4e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@calvertj calvertj

Labels
bug
Projects
Spline
Status: Done
Milestone
0.4.1
Development

No branches or pull requests
2 participants
@wajda @calvertj