Connection closed error

[groovybayo]

Hi Oliver – Nice work on the script. I am experiencing the same issues with lost connection when copying over the install.sh.

copying install.sh to 1 server(s)…lost connection
.grep: /Users/workspace/testing-ws/jmeter-ec2/sample-project/1364519238*scpinstall.out: No such file or directory

I have verified my security group is configured correctly and can ssh into the box by doing a telnet ping on 22

telnet ec2-23-23-69-1.compute-1.amazonaws.com 22 ⏎
Trying 23.23.69.1…
 Connected to ec2-23-23-69-1.compute-1.amazonaws.com.
 Escape character is ‘^]’.
 SSH-2.0-OpenSSH_5.9p1 Debian-5ubuntu1
  ^C

However, the scp of install.sh fails and I can’t get past that point. I have even put a sleep 60s as suggested by Zack on (http://www.http503.com/2012/run-jmeter-on-amazon-ec2-cloud/), to no avail.

Any ideas why this might be happening? Is there a work around?

Thanks in advance.

[klubi]

which version of ec2-api-tools you're using?
I used 1.6.7.1 and it looks like it returns description of instances in a bit different format than script expects it.
substitute line 165 with below and it should work (it did for me)

hosts=(ec2-describe-instances –region $REGION ${attempted_instanceids[@]} | awk ‘/PRIVATEIPADDRESS/ {print $2}’)

[oliverlloyd]

The change above will work where you are using a VPC but will not work if you are simply using normal, public, EC2. I suspect the change you mention in the return format for the ec2-describe-instances command is related to if it returns data of Private Instances or not?

[klubi]

I think you are right.
I have all instances in vpc so it worked for me (previous version was returning "running" as a host in my case).

[groovybayo]

Hi Oliver - do you have any suggestions or workaround to the issue I raised above? What klubi was facing is not my issue. I am using vanilla ec2 instances and the script fails on copying install step (as described above) even though I can ssh into the box just fine

[jtg-github]

I ran into the same issue intermittently. I had to keep on re-trying the script until it worked.

[oliverlloyd]

Closed as connection issue.

[groovybayo]

Way to go oliverlloyd. You just close the issue without addressing the issue after months of opening it? I was actually in the process of writing a book then and wanted to recommend this as a easy way to do distributed testing. I never got it to work. I strongly doubt it is a connection issue as I tried over the course of many days at different times to no avail. At the same times, I was able to connect to ec2 just fine using other tools including the CLI client.

[oliverlloyd]

Sigh. The problem here is that the script works for me and many others, but not for you. So you see, it is therefore logical for me to assume that you have some form of connectivity issue that is causing your client to lose the connection to the server. Technically, it is not the script that fails here, it is your ssh connection that drops out.

It is possible that this is not the case, perhaps there is a scenario that the script is not managing well - it is true others have reported the same thing - but I don't have enough information to work out what is going on and I have never been able to replicate the problem, so what more can I do?

[nik0kin]

I am having the same problem too, I am using an AWS instance to run this script so I don't think its a connection problem

[oliverlloyd]

Do you have any more specific information regarding the issue that could help me replicate it?

Are you able to telnet to the server in question on port 22?

Are you using any kind of VPN?

Also, if you see the problem when running on an AWS instance then what happens if you run locally?

[nik0kin]

I spawned an ubuntu 64bit instance (micro on us-east-1d), installed the tools using this guide: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/SettingUp_CommandLine.html

Setup a security group with just port 22 open to anyone. Then used 'git clone' on your project, configured jmeter-ec2.properties.

I am able to telnet to it on 22

I am not on a VPN

If I run it locally I get the same problem, "copying install.sh to 1 server(s)…lost connection"

[oliverlloyd]

Could you do this: put the text 'set -x' on the second line of the jmeter-ec2.sh file, save it, run again and then paste here the output you get from that.

Thanks

[nik0kin]

STDERR(set -x): http://pastebin.com/---------
STDOUT: http://pastebin.com/-------

[oliverlloyd]

OK, so reading through the stdout you've given, I can see that the error occurs when you run this scp command:

scp -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i /home/ubuntu/nikskey.pem -P 22 /home/ubuntu/jmeter-ec2_old/install.sh /home/ubuntu/jmeter-ec2_old/jmeter-ec2.properties ubuntu@ec2-54-221-75-34.compute-1.amazonaws.com:/tmp

What happens if you simply try running this command by itself (you'll obviously need to spin up another instance using your ami to test this). You could perhaps simplify it to:

scp -vvv -i /home/ubuntu/nikskey.pem /home/ubuntu/jmeter-ec2_old/install.sh ubuntu@[YOURSERVER]:/tmp

I added the -vvv option to print debug info. Use this option and hopefully the output from this command will explain the issue.

[nik0kin]

I'm so sorry, but it was all due to not chmod-ing my .pem file with 600, i had it readable by group aswell. Maybe the script should check for that.

But thank you so much for all your help 👍

[oliverlloyd]

An old issue but I wanted to list my findings on this. So, I have, in my travels, seen this lost connection issue several times now. In each and every case I was able to get around it by turning on my vpn. This tends to suggest that the problem is indeed that the connection is failing due to some configuration or restriction on the local network. I always see it in cafes, esp. where the wifi is the type that pops up a dialogue.

[jandom]

Hi, i'm using the connection from a university network. Not a cafe, etc, but could be dodgy in some way. I've checked ssh, i can connect (haven't checked telnet), set *.pem permissions to be 600. No luck, still getting lost connection. Then, tried departmental VPN, still same problem. How do i best debug what's going on?

[oliverlloyd]

The first thing you can do is to try and telnet to the server on port 22:

$ telnet someserver.com 22

If you see Trying 123.456.789.012... then you do not have connectivity and need to work around this somehow (use a different network is the easiest).

You could also try putting

set -xv

at the top of the file jmeter-ec2.sh. This will run it in debug mode.

[jandom]

Did the telnet trick, I could connect to host:

Trying xxx...
Connected to xxxx.
Escape character is '^]'.
SSH-2.0-OpenSSH_6.6p1 Ubuntu-2ubuntu1
^C
Connection closed by foreign host.

So that's good. With the debug mode I get

+ scp -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i $HOME/.ec2/xxx.pem -P 22 xxx/jmeter-ec2/install.sh xxxx/jmeter-ec2/jmeter-ec2.properties ubuntu@xxxx.compute.amazonaws.com:/tmp
lost connection

So it sees that there is a problem with the scp command. It turns out that I was using a pem file that wasn't matching the AMAZON_KEYPAIR_NAME. As soon as I fixed that, everything worked.

[jandom]

Obviously this was an idiotic mistake on my part. Hopefully it will suggest to others what may be broken in their case. Is there any room for validating the input (check if keypair and pem file are matching)?

[oliverlloyd]

So there's scope for validating local vars and file permissions - that's worth doing.

As an aside, I gave 10 mins thought to validating the connection before trying to run the instances. Result: Pretty much impossible. You have to first create the instance and only then can I check if I am able to connect to it (by trying to connect to it!). Even if there was a public server that was accessible of port 22 to the world and even if they didn't mind my script randomly establishing a connection, the vagaries of ssh access and firewalls in general are so vast that even if I could connect to this test box it's no guarantee the connection to EC2 would work.

[snikic]

I ran into the same error message as in the first post. In my case, the problem was that i didn´t update the username in the jmeter.properties. It is set to ubuntu in jmeter.properties, but the EC2 instance expected the user "ec2-user".

Hope this saves someone some time.

[DmitriyMazur]

I fight with the same issue. Solution in my case was to change security group rules. I edit service parameters in inbound rules section. Added myIP (or customIP or anywhere) to include my public IP address.

[tegryan]

For anyone having troubles with the infamous "copying install.sh to 1 server(s)…lost connection" issue and are using EC2 try changing your security group to be any IP on port 22, not just the host you are trying to access from.

I can't explain it, but from one EC2 host to the Jmeter host I could SSH, but not SCP. I tried all suggestions in thread and others and nothing worked until I did that. I suspect you couldn't reproduce it Oliver because you were not going from EC2 host to EC2 host (just a guess).

I also tried disabling MOTD, changing to the ec2-user (I need to use ubuntu), creating a .hushlogin file and so on. If you recreate the SCP command, add -vvv and get rid of -q while the jmeter node is up you might get some additional info. Certainly if the permissions on the .pem file are wrong that will at least tell you.

[manusovich]

I also meet this issue and my fault was because i changed default user to ec2-user (instead of ubuntu) in properties and forgot to change AMI_ID. So, my instances didn't have such user (ec2-user). My bad. As soon as i change user back to ubuntu, everything starts working well.