Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add protocol encryption #119

Merged
77 commits merged into from Nov 25, 2014
Merged
Changes from all commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
291dfd5
add travis-ci file
abma Nov 20, 2014
3f61b37
travis: create protcol
abma Nov 20, 2014
fced438
travis: install depends
abma Nov 20, 2014
a47eb54
fix some python3 errors
abma Nov 20, 2014
b86164a
Merge branch 'master' into crypto
abma Nov 21, 2014
5a761ae
Merge branch 'crypto' of github.com:spring/uberserver into crypto
abma Nov 21, 2014
e48f846
try to fix travis
abma Nov 21, 2014
040ea9f
try to fix travis, no. two
abma Nov 21, 2014
24d4bf7
update testclient
Nov 21, 2014
763bc2b
-
Nov 21, 2014
ec93c67
remove ACKSHAREDKEY from ALLOWED_OPEN_COMMANDS
Nov 21, 2014
05b4db3
fix + improve test
abma Nov 21, 2014
da171d9
travis: disable test with python 3.*
abma Nov 21, 2014
0c187ee
testclient: skip registration since accounts already exist
Nov 21, 2014
ca48f54
fix ./Protocol.py
abma Nov 21, 2014
69ab4c9
testclient: handle batch-encrypted server commands
Nov 21, 2014
bee5e1f
tweak for trailing NLs
Nov 21, 2014
ce7570d
fix
Nov 21, 2014
280bfc7
-
Nov 21, 2014
7010ea1
testclient: some fixes / more verbose error msg
abma Nov 21, 2014
bdf2995
travis: add delays for start/stop of server
abma Nov 21, 2014
07a88d7
testclient: fix pub-key import
Nov 21, 2014
c37c92f
add extra comment
Nov 21, 2014
982ead6
fix testclient
abma Nov 21, 2014
16155cd
travis: increase time to wait for server start
abma Nov 21, 2014
8ea3760
continue instead of return
Nov 21, 2014
389305d
travis: further increase time oO
abma Nov 21, 2014
79697ac
catch exception in DECODE_FUNC, too
abma Nov 21, 2014
b7c0774
increase default user data-limits for testing
Nov 21, 2014
7f1c889
Protocol: fix typo and disallow spaces in naked passwords
Nov 21, 2014
7d3e63b
Protocol: extend validNewPasswordSyntax
Nov 21, 2014
726e3bc
testclient: do not drop ACKSHAREDKEY
Nov 21, 2014
d0cec0b
Protocol: fix typo
Nov 21, 2014
2cbf69e
Protocol: ducktyping *sucks*
Nov 21, 2014
87dcdef
Protocol: more typos
Nov 21, 2014
024bb5b
SQLUsers: typo
Nov 21, 2014
8992302
SQLUsers: handle unicode passwords
Nov 21, 2014
e9238fc
fix a few more encoding-related issues
Nov 22, 2014
76a50f5
last one
Nov 22, 2014
93303fe
TestLobbyClient: set want_secure_session = True again
Nov 22, 2014
45d062f
misc minor refactoring
Nov 22, 2014
65afc60
protocol: add marker bytes for encrypted blobs
Nov 22, 2014
3e9f059
TestLobbyClient: increase startup speed
Nov 22, 2014
dca8876
add RSA_NULL_KEY_OBJ
Nov 22, 2014
ce4660d
s/INVALID/REJECTED; s/CLEARED/DISABLED
Nov 22, 2014
b4f6441
fix typo (encode whole string and not only first byte)
abma Nov 22, 2014
78a3b63
fix b4f64413
Nov 22, 2014
fadd206
make key files readable only by user
abma Nov 22, 2014
6bc8f5c
-
abma Nov 22, 2014
405aaca
handle session-key renegotiation
Nov 22, 2014
0a0f62c
finalize (protocol is now feature-complete)
Nov 22, 2014
6dc5c3d
testclient: try to login when registration fails
abma Nov 23, 2014
efe6bb6
testclient: run single-threaded if possible
abma Nov 23, 2014
75a5f77
Protocol: echo back ENFORCED if client tries to disable server-enforc…
Nov 23, 2014
d2130ae
do not let session_key_id equal DATA_PARTIT_BYTE
Nov 23, 2014
4374b2f
TestClient: fix LOGINs being sent unencrypted
Nov 23, 2014
91c0ee4
TestClient: fix copypasta errors in out_LOGIN
Nov 23, 2014
1e27be3
TestClient: log DENIED responses
Nov 23, 2014
6c57bca
SQLUsers: fix User.password instantiation (pull value from DB)
Nov 23, 2014
c251614
minor odds&ends
Nov 24, 2014
d46131b
Protocol: switch to a-posteriori key exchange (simpler)
Nov 24, 2014
ea1af25
-
Nov 24, 2014
c054f9b
--
Nov 24, 2014
8561dac
Protocol: remove the HASH command (too insane to support)
Nov 24, 2014
cc2e2de
Protocol/SQLUsers: expect new-style passwords b64-encoded
Nov 25, 2014
e643489
minor cleanup
Nov 25, 2014
5f406c8
safety (continue instead of assert)
Nov 25, 2014
cd22091
fix
Nov 25, 2014
1b05109
TestClient: add signature verification
Nov 25, 2014
352c2e0
-
Nov 25, 2014
1e774b4
TestClient: busy-loop on socket creation
Nov 25, 2014
eaee58b
Protocol: encode signatures
Nov 25, 2014
76f0e9d
-
Nov 25, 2014
5be2ab2
--
Nov 25, 2014
c591cea
fix typo
Nov 25, 2014
4f843b9
sign using SHA256 (not SHA160)
Nov 25, 2014
b7958d8
TestClient: fix maximum ping-time values (minor)
Nov 25, 2014
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.

Always

Just for now

@@ -0,0 +1,17 @@
language: python
python:
# - "2.6"
- "2.7"
# - "3.2"
# - "3.3"
# - "3.4"
install:
- pip install SQLAlchemy pycrypto

script:
- protocol/Protocol.py
- ./server.py &
- sleep 30 # wait for server to start up
- ./TestLobbyClient.py
- sleep 10 # give server some time to write log, etc
- killall python
@@ -9,12 +9,12 @@ def has_insecure_password(self):
return (len(self.randsalt) == 0)


def set_user_pwrd_salt(self, user_name = "", user_pass_salt = ("", "")):
assert(type(user_pass_salt) == type(()))
def set_user_pwrd_salt(self, user_name = "", pwrd_hash_salt = ("", "")):
assert(type(pwrd_hash_salt) == type(()))

self.username = user_name
self.password = user_pass_salt[0]
self.randsalt = user_pass_salt[1]
self.password = pwrd_hash_salt[0]
self.randsalt = pwrd_hash_salt[1]

def set_pwrd_salt(self, pwrd_hash_salt):
assert(type(pwrd_hash_salt) == type(()))
@@ -1,5 +1,6 @@
import time, traceback
from Client import Client
from CryptoHandler import UNICODE_ENCODING

class ChanServ:
def __init__(self, client, root):
@@ -29,7 +30,7 @@ def Handle(self, msg):
def HandleMessage(self, chan, user, msg):
if len(msg) <= 0:
return
if msg[0] != "!":
if msg[0] != "!":
return
msg = msg.lstrip('!')
args = None
@@ -198,9 +199,9 @@ def HandleCommand(self, chan, user, cmd, args=None):
def Send(self, msg):
if type(msg) == list or type(msg) == tuple:
for s in msg:
self.client.HandleProtocolCommand(s.encode("utf-8"))
self.client.HandleProtocolCommand(s.encode(UNICODE_ENCODING))
else:
self.client.HandleProtocolCommand(msg.encode("utf-8"))
self.client.HandleProtocolCommand(msg.encode(UNICODE_ENCODING))

class ChanServClient(Client):
'this object is chanserv implemented through the standard client interface'
112 Client.py
@@ -3,9 +3,13 @@

from BaseClient import BaseClient
from CryptoHandler import aes_cipher
from CryptoHandler import safe_base64_decode as SAFE_DECODE_FUNC
from CryptoHandler import DATA_MARKER_BYTE
from CryptoHandler import DATA_PARTIT_BYTE
from CryptoHandler import UNICODE_ENCODING

class Client(BaseClient):
'this object represents one connected client'
'this object represents one server-side connected client'

def __init__(self, root, connection, address, session_id):
'initial setup for the connected client'
@@ -64,18 +68,17 @@ def __init__(self, root, connection, address, session_id):
self.hostport = None
self.udpport = 0
self.bot = 0
self.floodlimit = {'fresh':{'msglength':1024, 'bytespersecond':1024, 'seconds':2},
'user':{'msglength':1024, 'bytespersecond':1024, 'seconds':10},
'bot':{'msglength':1024, 'bytespersecond':10000, 'seconds':5},
'mod':{'msglength':10000, 'bytespersecond':10000, 'seconds':10},
'admin':{'msglength':10000, 'bytespersecond':100000, 'seconds':10},}
self.floodlimit = {
'fresh':{'msglength':1024*32, 'bytespersecond':1024*32, 'seconds':2},
'user':{'msglength':1024*32, 'bytespersecond':1024*32, 'seconds':10},
'bot':{'msglength':1024, 'bytespersecond':10000, 'seconds':5},
'mod':{'msglength':10000, 'bytespersecond':10000, 'seconds':10},
'admin':{'msglength':10000, 'bytespersecond':100000, 'seconds':10},
}
self.msglengthhistory = {}
self.lastsaid = {}
self.nl = '\n'
self.current_channel = ''

self.tokenized = False
self.hashpw = False
self.debug = False
self.data = ''

@@ -97,43 +100,25 @@ def __init__(self, root, connection, address, session_id):

self._root.console_write('Client connected from %s:%s, session ID %s.' % (self.ip_address, self.port, session_id))

self.set_aes_cipher_obj(None)
self.set_session_key_acknowledged(False)
self.set_num_pushed_session_key_acks(0)
## AES cipher used for encrypted protocol communication
## with this client; starts with a NULL session-key and
## becomes active when client sends SETSHAREDKEY
self.set_aes_cipher_obj(aes_cipher(""))
self.set_session_key("")

self.set_session_key_received_ack(False)


## AES cipher used for encrypted protocol communication
## (obviously with a different instance and key for each
## connected client)
def set_aes_cipher_obj(self, obj): self.aes_cipher_obj = obj
def get_aes_cipher_obj(self): return self.aes_cipher_obj

## NOTE:
## only for in-memory clients, not DB User instances
## when true, aes_cipher_obj always contains a valid
## key
def use_secure_session(self):
return (self.aes_cipher_obj != None)


def set_num_pushed_session_key_acks(self, n): self.num_pushed_session_key_acks = n
def get_num_pushed_session_key_acks(self): return self.num_pushed_session_key_acks
def push_session_key_acknowledged(self): self.set_num_pushed_session_key_acks(self.num_pushed_session_key_acks + 1)
def pop_session_key_acknowledged(self): self.set_num_pushed_session_key_acks(max(self.num_pushed_session_key_acks - 1, 0))

def set_session_key_acknowledged(self, b): self.session_key_acknowledged = b
def get_session_key_acknowledged(self): return self.session_key_acknowledged

def set_session_key(self, key):
if (self.get_aes_cipher_obj() == None):
self.set_aes_cipher_obj(aes_cipher(""))
def set_session_key_received_ack(self, b): self.session_key_received_ack = b
def get_session_key_received_ack(self): return self.session_key_received_ack

self.aes_cipher_obj.set_key(key)
def set_session_key(self, key): self.aes_cipher_obj.set_key(key)
def get_session_key(self): return (self.aes_cipher_obj.get_key())

def get_session_key(self):
if (self.aes_cipher_obj == None):
return ""
return (self.aes_cipher_obj.get_key())
def use_secure_session(self): return (len(self.get_session_key()) != 0)


def set_msg_id(self, msg):
@@ -164,6 +149,9 @@ def Bind(self, handler=None, protocol=None):
self._protocol = protocol


##
## handle data from client
##
def Handle(self, data):
if (self.access in self.floodlimit):
msg_limits = self.floodlimit[self.access]
@@ -192,7 +180,7 @@ def Handle(self, data):
if total > (bytespersecond * seconds):
if not self.access in ('admin', 'mod'):
if (self.bot != 1):
# FIXME: no flood limit for these atm, need to rewrite flood limit to server-side shaping/bandwith limiting
# FIXME: no flood limit for these atm, need to do server-side shaping/bandwith limiting
self.Send('SERVERMSG No flooding (over %s per second for %s seconds)' % (bytespersecond, seconds))
self.Remove('Kicked for flooding (%s)' % (self.access))
return
@@ -203,7 +191,7 @@ def Handle(self, data):
if (self.data.count('\n') == 0):
return

self.HandleProtocolCommands(self.data.split('\n'), msg_limits)
self.HandleProtocolCommands(self.data.split(DATA_PARTIT_BYTE), msg_limits)


def HandleProtocolCommands(self, split_data, msg_limits):
@@ -229,6 +217,9 @@ def HandleProtocolCommands(self, split_data, msg_limits):

for raw_data_blob in raw_data_blobs:
if (self.use_secure_session()):
if (raw_data_blob[0] != DATA_MARKER_BYTE):
continue

## handle an encrypted client command, using the AES session key
## previously exchanged between client and server by SETSHAREDKEY
## (this includes LOGIN and REGISTER, key can be set before login)
@@ -237,9 +228,9 @@ def HandleProtocolCommands(self, split_data, msg_limits):
## ENCODE(ENCRYPT_AES("CMD ARG1 ARG2 ...", AES_KEY))
## where ENCODE is the standard base64 encoding scheme
##
## if this is not the case (e.g. if a command was sent unencrypted
## if this is not the case (e.g. if a command was sent UNENCRYPTED
## by client after session-key exchange) the decryption will yield
## garbage and command will be rejected (or maybe crash the server)
## garbage and command will be rejected
##
## NOTE:
## blocks of encrypted data are always base64-encoded and will be
@@ -254,11 +245,15 @@ def HandleProtocolCommands(self, split_data, msg_limits):
## (there should not be any encoding on top of base64
## blobs in the first place since the b64 alphabet is
## ASCII)
cmd_data_blob = self.aes_cipher_obj.decode_decrypt_bytes_utf8(raw_data_blob)
enc_data_blob = raw_data_blob[1: ]
dec_data_blob = self.aes_cipher_obj.decode_decrypt_bytes_utf8(enc_data_blob, SAFE_DECODE_FUNC)

split_commands = cmd_data_blob.split('\n')
split_commands = dec_data_blob.split(DATA_PARTIT_BYTE)
strip_commands = [(cmd.rstrip('\r')).lstrip(' ') for cmd in split_commands]
else:
if (raw_data_blob[0] == DATA_MARKER_BYTE):
continue

## strips leading spaces and trailing carriage returns
strip_commands = [(raw_data_blob.rstrip('\r')).lstrip(' ')]

@@ -271,6 +266,7 @@ def HandleProtocolCommands(self, split_data, msg_limits):
self.HandleProtocolCommand(command)

def HandleProtocolCommand(self, cmd):
## probably caused by trailing newline ("abc\n".split("\n") == ["abc", ""])
if (len(cmd) <= 1):
return

@@ -282,8 +278,11 @@ def Remove(self, reason='Quit'):
self.FlushBuffer()
self.handler.finishRemove(self, reason)

##
## send data to client
##
def Send(self, msg, binary = False):
# don't append new data to send buffer when client gets removed
## don't append new data to buffer when client gets removed
if ((not msg) or self.removing):
return

@@ -293,9 +292,16 @@ def Send(self, msg, binary = False):

if (self.use_secure_session()):
## apply server-to-client encryption
msg = self.aes_cipher_obj.encrypt_encode_bytes_utf8(msg)

if ((not self.get_session_key_acknowledged()) and (self.get_num_pushed_session_key_acks() == 0)):
##
## add marker byte so client does not have to guess
## if data is of the form ENCODE(ENCRYPTED(...)) or
## in plaintext
##
hdr = DATA_MARKER_BYTE
pay = self.aes_cipher_obj.encrypt_encode_bytes_utf8(msg + DATA_PARTIT_BYTE)
msg = hdr + pay

if (not self.get_session_key_received_ack()):
## buffer encrypted data until we get client ACK
## (the most recent message will be at the back)
##
@@ -310,19 +316,17 @@ def Send(self, msg, binary = False):
## pop from back so client receives in the proper
## order, with <msg> itself after the queued data
while (len(self.enc_sendbuffer) > 0):
self.msg_sendbuffer.append(self.enc_sendbuffer.pop() + self.nl)
self.msg_sendbuffer.append(self.enc_sendbuffer.pop())

## send the output as-is (base64 is all ASCII)
binary = True


if (binary):
self.msg_sendbuffer.append(msg + self.nl)
self.msg_sendbuffer.append(msg + DATA_PARTIT_BYTE)
else:
self.msg_sendbuffer.append(msg.encode("utf-8") + self.nl)
self.msg_sendbuffer.append(msg.encode(UNICODE_ENCODING) + DATA_PARTIT_BYTE)

self.handler.poller.setoutput(self.conn, True)
self.pop_session_key_acknowledged()


def FlushBuffer(self):
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.