Skip to content
This repository has been archived by the owner on Dec 12, 2022. It is now read-only.

find_model_collaborative.py job failed #7

Closed
joshuarcher opened this issue Apr 5, 2016 · 0 comments
Closed

find_model_collaborative.py job failed #7

joshuarcher opened this issue Apr 5, 2016 · 0 comments

Comments

@joshuarcher
Copy link

Initially the job ran on dataproc failed on line 60 of find_model_collaborative.py because the 'load' call was deprecated. Instead I changed that line to:
dfRates = sqlContext.read.load(source='jdbc', driver=jdbcDriver, url=jdbcUrl, dbtable='Rating')

ran the job again, came back with another error "java.util.NoSuchElementException: key not found: path" in callstack:
`16/04/05 19:30:01 INFO akka.event.slf4j.Slf4jLogger: Slf4jLogger started
16/04/05 19:30:01 INFO Remoting: Starting remoting
16/04/05 19:30:01 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriverActorSystem@xx.xxx.x.x:33193]
16/04/05 19:30:01 INFO org.spark-project.jetty.server.Server: jetty-8.y.z-SNAPSHOT
16/04/05 19:30:01 INFO org.spark-project.jetty.server.AbstractConnector: Started SelectChannelConnector@0.0.0.0:4040
16/04/05 19:30:02 INFO org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at producthunter-cluster-1-m/xx.xxx.x.x:8032
16/04/05 19:30:05 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl: Submitted application application_1459820580219_0002
Traceback (most recent call last):
File "/tmp/0df71cfb-bf81-4fff-9838-2d5c8b87db47/find_model_collaborative.py", line 60, in
dfRates = sqlContext.read.load(source='jdbc', driver=jdbcDriver, url=jdbcUrl, dbtable='Rating')
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 139, in load
File "/usr/lib/spark/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py", line 813, in call
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 45, in deco
File "/usr/lib/spark/python/lib/py4j-0.9-src.zip/py4j/protocol.py", line 308, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o47.load.
: java.util.NoSuchElementException: key not found: path
at scala.collection.MapLike$class.default(MapLike.scala:228)
at org.apache.spark.sql.execution.datasources.CaseInsensitiveMap.default(ddl.scala:150)
at scala.collection.MapLike$class.apply(MapLike.scala:141)
at org.apache.spark.sql.execution.datasources.CaseInsensitiveMap.apply(ddl.scala:150)
at org.apache.spark.sql.execution.datasources.ResolvedDataSource$$anonfun$10.apply(ResolvedDataSource.scala:168)
at org.apache.spark.sql.execution.datasources.ResolvedDataSource$$anonfun$10.apply(ResolvedDataSource.scala:168)
at scala.Option.getOrElse(Option.scala:120)
at org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:168)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:119)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
at py4j.Gateway.invoke(Gateway.java:259)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:209)
at java.lang.Thread.run(Thread.java:745)

16/04/05 19:30:10 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/05 19:30:10 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.`

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant