Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Execution never ends with a bad Redshift connection #22

Closed
rafaelleinio opened this issue Sep 16, 2019 · 1 comment
Closed

Execution never ends with a bad Redshift connection #22

rafaelleinio opened this issue Sep 16, 2019 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@rafaelleinio
Copy link

I experienced this issue testing aws-data-wrangler in the "Spark to Redshift" use case.

I created a Redshift connection with the method Redshift.generate_connection and when I tried to load the dataframe to Redshift with the method session.spark.to_redshift the executions never finished so I had to manually cancel the execution. The temporary files on S3 were created, but nothing on Redshift. Later I realize that I was passing the port as string to the connection but I think that the method expects an int argument, when I changed this the load worked just fine!

My guess is that with a wrong connection setup wrangler can't reach Redshift but there is no timeout to stop the execution, so the code gets stuck.

@igorborgest
Copy link
Contributor

Thanks @rafaelleinio!

I solved the port typing issue, and also created a more generic connection validation mechanism.

Make sense?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants