cannot connect to sql server #820
Description
I have enable tcp port in sql server manager and set up user for admin in sql server management
I'm not sure that I install jdbc driver correct or set Environment correct
And not sure that the jdbc format correct
SparkSession spark = SparkSession
.Builder()
.AppName("Connect to SQL Server")
.GetOrCreate();
//string connection_url1 = @"jdbc:sqlserver://192.168.100.246:1433;instanceName=WIN-TLDRK7EE98E\SQLEXPRESS;databaseName=Outsider;";
//string connection_url2 = @"Server=WIN-TLDRK7EE98E\SQLEXPRESS;Database= Timeattend;Trusted_Connection=True";
//string connection_url3 = "jdbc:sqlserver://192.168.100.246:1433;instanceName=SQLEXPRESS;databaseName=Outsider;";
string connection_url4 = "jdbc:sqlserver://127.0.0.1:1433;instanceName=.\\MyInstance;databaseName=Outsider;"; //
string dbtable = "IDCard;";
string user = "user;";
string password = "brownien_2020;";
DataFrame jdbcDF = spark.Read()
.Format("jdbc:")
.Option("driver", "com.microsoft.sqlserver.jdbc.SQLServerDriver")
.Option("url", connection_url4)
.Option("dbtable", dbtable)
.Option("user", user)
.Option("password", password).Load();
jdbcDF.Show(); // Displays the content of the SQL table as a DataFrame
21/02/05 17:10:43 ERROR DotnetBackendHandler: Failed to execute 'load' on 'org.apache.spark.sql.DataFrameReader' with args=()
[2021-02-05T10:10:43.7620348Z] [WIN-TLDRK7EE98E] [Error] [JvmBridge] JVM method execution failed: Nonstatic method 'load' failed for class '7' when called with no arguments
[2021-02-05T10:10:43.7622993Z] [WIN-TLDRK7EE98E] [Error] [JvmBridge] java.lang.ClassNotFoundException: Failed to find data source: jdbc:. Please find packages at http://spark.apache.org/third-party-projects.html
at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:679)
at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSourceV2(DataSource.scala:733)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:248)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:221)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.spark.api.dotnet.DotnetBackendHandler.handleMethodCall(DotnetBackendHandler.scala:159)
at org.apache.spark.api.dotnet.DotnetBackendHandler.$anonfun$handleBackendRequest$2(DotnetBackendHandler.scala:99)
at org.apache.spark.api.dotnet.ThreadPool$$anon$1.run(ThreadPool.scala:34)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ClassNotFoundException: jdbc:.DefaultSource
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$lookupDataSource$5(DataSource.scala:653)
at scala.util.Try$.apply(Try.scala:213)
at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$lookupDataSource$4(DataSource.scala:653)
at scala.util.Failure.orElse(Try.scala:224)
at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:653)
... 15 more
[2021-02-05T10:10:43.8346002Z] [WIN-TLDRK7EE98E] [Exception] [JvmBridge] JVM method execution failed: Nonstatic method 'load' failed for class '7' when called with no arguments
at Microsoft.Spark.Interop.Ipc.JvmBridge.CallJavaMethod(Boolean isStatic, Object classNameOrJvmObjectReference, String methodName, Object[] args)
Unhandled exception. System.Exception: JVM method execution failed: Nonstatic method 'load' failed for class '7' when called with no arguments
---> Microsoft.Spark.JvmException: java.lang.ClassNotFoundException: Failed to find data source: jdbc:. Please find packages at http://spark.apache.org/third-party-projects.html
at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:679)
at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSourceV2(DataSource.scala:733)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:248)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:221)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.spark.api.dotnet.DotnetBackendHandler.handleMethodCall(DotnetBackendHandler.scala:159)
at org.apache.spark.api.dotnet.DotnetBackendHandler.$anonfun$handleBackendRequest$2(DotnetBackendHandler.scala:99)
at org.apache.spark.api.dotnet.ThreadPool$$anon$1.run(ThreadPool.scala:34)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ClassNotFoundException: jdbc:.DefaultSource
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$lookupDataSource$5(DataSource.scala:653)
at scala.util.Try$.apply(Try.scala:213)
at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$lookupDataSource$4(DataSource.scala:653)
at scala.util.Failure.orElse(Try.scala:224)
at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:653)
... 15 more
--- End of inner exception stack trace ---
at Microsoft.Spark.Interop.Ipc.JvmBridge.CallJavaMethod(Boolean isStatic, Object classNameOrJvmObjectReference, String methodName, Object[] args)
at Microsoft.Spark.Interop.Ipc.JvmBridge.CallNonStaticJavaMethod(JvmObjectReference objectId, String methodName, Object[] args)
at Microsoft.Spark.Interop.Ipc.JvmObjectReference.Invoke(String methodName, Object[] args)
at Microsoft.Spark.Sql.DataFrameReader.Load()
at SparkSQL.Program.Main(String[] args) in C:\james\SparkSQL\SparkSQL\Program.cs:line 22
21/02/05 17:10:47 INFO DotnetRunner: Closing DotnetBackend
21/02/05 17:10:47 INFO SparkContext: Invoking stop() from shutdown hook
21/02/05 17:10:47 INFO SparkUI: Stopped Spark web UI at http://host.docker.internal:4040
21/02/05 17:10:47 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
21/02/05 17:10:47 INFO MemoryStore: MemoryStore cleared
21/02/05 17:10:47 INFO BlockManager: BlockManager stopped
21/02/05 17:10:47 INFO BlockManagerMaster: BlockManagerMaster stopped
21/02/05 17:10:47 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
21/02/05 17:10:47 INFO SparkContext: Successfully stopped SparkContext
21/02/05 17:10:47 INFO ShutdownHookManager: Shutdown hook called
21/02/05 17:10:47 INFO ShutdownHookManager: Deleting directory C:\Users\Administrator\AppData\Local\Temp\spark-8c821ad8-322e-40f5-9754-0aaa4fa972f5
21/02/05 17:10:47 INFO ShutdownHookManager: Deleting directory C:\Users\Administrator\AppData\Local\Temp\spark-227447db-8f76-42fa-b7ec-c494d7aaadb9