SoftLayer servers fail TLS 1.1 negotiation #17
Comments
|
EDIT: I'm a derp, just read which package this is for. It's for object storage, not the API bindings. Will respond in a second. Yipes! Ok cool, thanks for the heads up. Not denying that it exist, just trying to reproduce it so we know when it's fixed. I have a 12.04 server and using |
|
Hrm, wasn't able to reproduce it using that command either. I'll have to test with the bindings a tad later. I'd like to nail down a reproducible test so we can verify changing the SSL settings have indeed fixed the solution. |
|
This should work for reproducing: |
|
Thanks. Will investigate on our side of things. |
|
Were you able to reproduce? |
|
I deployed a new 12.04 CCI, replicated the above commands and I was not able to reproduce. Does the error happen instantly or eventually? |
|
Ah, it's eventual. Checking some more |
|
I was able to eventually reproduce the issue using the scripts you provided. However looking at the script, it was exhibiting some really bad behavior anyway, so I patched that up and was not able to reproduce the issue even with 300 threads and 2000 uploads. Please review my changes and tell my if I am off the path? diff --git a/runtest.py b/runtest.py
index 80f8e8c..9db0c37 100755
--- a/runtest.py
+++ b/runtest.py
@@ -6,6 +6,7 @@ import random
import string
import time
import traceback
+from itertools import repeat
from multiprocessing import Pool
parser = argparse.ArgumentParser(description='Load test object storage')
@@ -13,6 +14,11 @@ parser.add_argument('--username', type=str, dest='username', required=True,
help='object storage username')
parser.add_argument('--password', type=str, dest='password', required=True,
help='object storage password')
+
+parser.add_argument('--threads', '-t', type=int, dest='threads', required=False,
+ help='threads to spawn', default=100)
+parser.add_argument('--count', '-c', type=int, dest='limit', required=False,
+ help='total objects to upload', default=2000)
args = parser.parse_args()
def upload(data):
@@ -26,13 +32,18 @@ def upload(data):
print 'Received ' + e.status + ': ' + e.reason
except:
print traceback.format_exc()
- time.sleep(1)
- upload(data)
+ else:
+ time.sleep(1)
with open ('test.html', 'r') as myfile:
data = myfile.read()
- pool = Pool(processes=10)
- for i in range(20):
- pool.apply_async(upload, args=(data,))
- pool.close()
- pool.join()
+ pool = Pool(processes=args.threads)
+ async_res = pool.map_async(upload, repeat(data, args.limit))
+ while not async_res.wait(5):
+ if async_res.ready():
+ print("results are ready")
+ break
+ else:
+ print("Waiting for results")
+
+ pool.terminate() |
|
Hmm, that's strange. I thought it was happening with a single request previously, but I can't reproduce that now. Thanks for looking at my load test code. The real problem we're having is from our servers accessing object storage in Java, which I think are better behaved, but still overwhelm object store. Object store cannot take enough load from us. We aren't actually being blocked as a DOS attack are we? What's the limit to the number of requests per send that we can make? I'm surprised that one laptop could overwhelm the object storage no matter how fast it's making requests. |
|
Just for future reference, this problem is caused by having client that uses openssl that supports TLSv1.1 or 1.2 trying to connect to a server that only supports TLSv1. CrackerJackMax could not reproduce because his client supported TLSv1 too so the handshake went through |
|
As of October, we currently support TLSv1.2, v1.1, and v1.0, so excessive TLS renegotiation shouldn't be occurring anymore. If so, please open a new issue if this still occurs. Thanks! |
Using this client library results in the following error on Ubuntu 12.04:
SSLError: [Errno 8] _ssl.c:504: EOF occurred in violation of protocol
The bug needs to be fixed on SoftLayer's servers. See https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/965371
The text was updated successfully, but these errors were encountered: