You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I experienced this issue testing aws-data-wrangler in the "Spark to Redshift" use case.
I created a Redshift connection with the method Redshift.generate_connection and when I tried to load the dataframe to Redshift with the method session.spark.to_redshift the executions never finished so I had to manually cancel the execution. The temporary files on S3 were created, but nothing on Redshift. Later I realize that I was passing the port as string to the connection but I think that the method expects an int argument, when I changed this the load worked just fine!
My guess is that with a wrong connection setup wrangler can't reach Redshift but there is no timeout to stop the execution, so the code gets stuck.
The text was updated successfully, but these errors were encountered:
I experienced this issue testing aws-data-wrangler in the "Spark to Redshift" use case.
I created a Redshift connection with the method
Redshift.generate_connection
and when I tried to load the dataframe to Redshift with the methodsession.spark.to_redshift
the executions never finished so I had to manually cancel the execution. The temporary files on S3 were created, but nothing on Redshift. Later I realize that I was passing the port as string to the connection but I think that the method expects an int argument, when I changed this the load worked just fine!My guess is that with a wrong connection setup wrangler can't reach Redshift but there is no timeout to stop the execution, so the code gets stuck.
The text was updated successfully, but these errors were encountered: