You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Dec 12, 2022. It is now read-only.
Initially the job ran on dataproc failed on line 60 of find_model_collaborative.py because the 'load' call was deprecated. Instead I changed that line to: dfRates = sqlContext.read.load(source='jdbc', driver=jdbcDriver, url=jdbcUrl, dbtable='Rating')
ran the job again, came back with another error "java.util.NoSuchElementException: key not found: path" in callstack:
`16/04/05 19:30:01 INFO akka.event.slf4j.Slf4jLogger: Slf4jLogger started
16/04/05 19:30:01 INFO Remoting: Starting remoting
16/04/05 19:30:01 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriverActorSystem@xx.xxx.x.x:33193]
16/04/05 19:30:01 INFO org.spark-project.jetty.server.Server: jetty-8.y.z-SNAPSHOT
16/04/05 19:30:01 INFO org.spark-project.jetty.server.AbstractConnector: Started SelectChannelConnector@0.0.0.0:4040
16/04/05 19:30:02 INFO org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at producthunter-cluster-1-m/xx.xxx.x.x:8032
16/04/05 19:30:05 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl: Submitted application application_1459820580219_0002
Traceback (most recent call last):
File "/tmp/0df71cfb-bf81-4fff-9838-2d5c8b87db47/find_model_collaborative.py", line 60, in
dfRates = sqlContext.read.load(source='jdbc', driver=jdbcDriver, url=jdbcUrl, dbtable='Rating')
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 139, in load
File "/usr/lib/spark/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py", line 813, in call
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 45, in deco
File "/usr/lib/spark/python/lib/py4j-0.9-src.zip/py4j/protocol.py", line 308, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o47.load.
: java.util.NoSuchElementException: key not found: path
at scala.collection.MapLike$class.default(MapLike.scala:228)
at org.apache.spark.sql.execution.datasources.CaseInsensitiveMap.default(ddl.scala:150)
at scala.collection.MapLike$class.apply(MapLike.scala:141)
at org.apache.spark.sql.execution.datasources.CaseInsensitiveMap.apply(ddl.scala:150)
at org.apache.spark.sql.execution.datasources.ResolvedDataSource$$anonfun$10.apply(ResolvedDataSource.scala:168)
at org.apache.spark.sql.execution.datasources.ResolvedDataSource$$anonfun$10.apply(ResolvedDataSource.scala:168)
at scala.Option.getOrElse(Option.scala:120)
at org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:168)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:119)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
at py4j.Gateway.invoke(Gateway.java:259)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:209)
at java.lang.Thread.run(Thread.java:745)
16/04/05 19:30:10 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/05 19:30:10 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.`
The text was updated successfully, but these errors were encountered:
Initially the job ran on dataproc failed on line 60 of find_model_collaborative.py because the 'load' call was deprecated. Instead I changed that line to:
dfRates = sqlContext.read.load(source='jdbc', driver=jdbcDriver, url=jdbcUrl, dbtable='Rating')
ran the job again, came back with another error "java.util.NoSuchElementException: key not found: path" in callstack:
`16/04/05 19:30:01 INFO akka.event.slf4j.Slf4jLogger: Slf4jLogger started
16/04/05 19:30:01 INFO Remoting: Starting remoting
16/04/05 19:30:01 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriverActorSystem@xx.xxx.x.x:33193]
16/04/05 19:30:01 INFO org.spark-project.jetty.server.Server: jetty-8.y.z-SNAPSHOT
16/04/05 19:30:01 INFO org.spark-project.jetty.server.AbstractConnector: Started SelectChannelConnector@0.0.0.0:4040
16/04/05 19:30:02 INFO org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at producthunter-cluster-1-m/xx.xxx.x.x:8032
16/04/05 19:30:05 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl: Submitted application application_1459820580219_0002
Traceback (most recent call last):
File "/tmp/0df71cfb-bf81-4fff-9838-2d5c8b87db47/find_model_collaborative.py", line 60, in
dfRates = sqlContext.read.load(source='jdbc', driver=jdbcDriver, url=jdbcUrl, dbtable='Rating')
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 139, in load
File "/usr/lib/spark/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py", line 813, in call
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 45, in deco
File "/usr/lib/spark/python/lib/py4j-0.9-src.zip/py4j/protocol.py", line 308, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o47.load.
: java.util.NoSuchElementException: key not found: path
at scala.collection.MapLike$class.default(MapLike.scala:228)
at org.apache.spark.sql.execution.datasources.CaseInsensitiveMap.default(ddl.scala:150)
at scala.collection.MapLike$class.apply(MapLike.scala:141)
at org.apache.spark.sql.execution.datasources.CaseInsensitiveMap.apply(ddl.scala:150)
at org.apache.spark.sql.execution.datasources.ResolvedDataSource$$anonfun$10.apply(ResolvedDataSource.scala:168)
at org.apache.spark.sql.execution.datasources.ResolvedDataSource$$anonfun$10.apply(ResolvedDataSource.scala:168)
at scala.Option.getOrElse(Option.scala:120)
at org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:168)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:119)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
at py4j.Gateway.invoke(Gateway.java:259)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:209)
at java.lang.Thread.run(Thread.java:745)
16/04/05 19:30:10 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/05 19:30:10 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.`
The text was updated successfully, but these errors were encountered: