-
Notifications
You must be signed in to change notification settings - Fork 530
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SSH Hook Problem - Failed to create remote temp file #38
Comments
Hi, Did you test a ssh connection inside container (outside airflow dag) ? |
Hi @puckel , |
This error occurred for me when the permissions for the private key file were not sufficiently strict. It was solved by running a command like For reference, the working configuration has an SSHHook created with
where the connection
|
(And it required creating a new docker container based on |
@expz thank you for your answer. But there is something that still unclear. do you generate the id_rsa key inside the container (worker?scheduler?webserver? or all?) or you generate it on the host machine and you define a volume in the docker-compose-CeleryExecutor file for each service to the id_rsa key that is found on the host machine ? |
Generate id_rsa on the host and mount the .ssh directory as a volume inside the container. This will guarantee persistence of the key. Otherwise the key will disappear if you destroy and remake the container. |
@expz docker is on a host with ip @ 1.2.3.4 and the host that i want my docker conatiner to coneect to is my hadoop cluster with ip @ 1.2.3.5.. when i use you suggested ip @ 172.17.0.1 it will connect to the docker0 but the docker it self is not connected somehow to the 2sn host. Did i miss something? how to make docker conect to the seconf host without install it there? |
@bdrjonny - did you resolve this? I am seeing the same error. I am trying to use SSHExecuteOperator https://github.com/OpenGov/airflow/blob/master/airflow/contrib/operators/ssh_execute_operator.py#L74 and I create a task like : when I pass bash_command = 'echo $hostname' then it gives me an error Failed to create remote temp file |
may not be directly related, just wasted five minutes on this myself, so will restate for others. make sure not to miss the extra json that @expz listed for disabling known host check. without that ( or a valid entry in if checking the remote key is required, you will need to request/accept the key as part of provisioning or persist the this can sneak by manual testing, as you might test the connection, accept the prompt "confirming" that it works, retry in airflow again confirming that it works, and then lose the known hosts file on next spin-up. this is not specific to |
Hi Puckel,
i´ve got a Problem with Airflow SSH Hook. When i start my dag, i got an error, that Airflow "Failed to create remote temp file". I want to open a ssh-connection via Airflow. I set all settings in Airflow Connections and my DAG. Maybe you can help me ?!
When i run this DAG (sshjonas.py) outside of a docker-container on a local Airflow-Installation then everythink works...
BR,
Jonas
dag.txt
The text was updated successfully, but these errors were encountered: