Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ClassCastException using AnalyzeSpark #7680

Closed
dennislloydjr opened this issue May 6, 2019 · 2 comments

Comments

@dennislloydjr
Copy link

commented May 6, 2019

Issue Description

I'm executing the following method:
val dataAnalysis = AnalyzeSpark.analyze(mySchema, myRdd, maxHistogramBuckets)
where the first column is a String data type.

When this executes, I receive:

java.lang.ClassCastException: org.datavec.spark.transform.analysis.string.StringAnalysisCounter cannot be cast to org.datavec.api.transform.analysis.counter.StringAnalysisCounter
	at org.datavec.api.transform.analysis.DataVecAnalysisUtils.convertCounters(DataVecAnalysisUtils.java:73) ~[datavec-api-1.0.0-beta3.jar:?]
	at org.datavec.spark.transform.AnalyzeSpark.analyze(AnalyzeSpark.java:145) ~[datavec-spark_2.11-1.0.0-beta3_spark_2.jar:?]

It appears there was an effort to refactor out some of the counter conversion code to make is usable by both spark and local executions. This looks like the place where the trouble is at:

https://github.com/deeplearning4j/deeplearning4j/blob/master/datavec/datavec-api/src/main/java/org/datavec/api/transform/analysis/DataVecAnalysisUtils.java#L73

Version Information

I'm using version org.datavec:datavec-spark_2.11:1.0.0-beta3_spark_2

@AlexDBlack AlexDBlack self-assigned this May 13, 2019

AlexDBlack added a commit that referenced this issue May 13, 2019
@AlexDBlack

This comment has been minimized.

Copy link
Contributor

commented May 13, 2019

Thanks for reporting this - fixed here: #7724

AlexDBlack added a commit that referenced this issue May 14, 2019
AlexDBlack added a commit that referenced this issue May 15, 2019
Various fixes (#7724)
* #7715 Fix MergeVertex for CNN3D activations

* #7680 - AnalyzeSpark fix

* First round of uint TF import fixes

* More dtype fixes

* BFLOAT16 import

* another bunch of small fixes

* one more nano fix

* int16 fix

* int8/uint8 fix
@AlexDBlack

This comment has been minimized.

Copy link
Contributor

commented Jun 3, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.