Python Howto

vixentael edited this page Feb 5, 2018 · 29 revisions
Clone this wiki locally

Using Themis in Python

Introduction

The pythemis extension provides access to the features and functions of the Themis cryptographic library in Python (2.7+ and 3):

  • Key generation: the creation of public/private key pairs, used in Secure Message and Secure Session.
  • Secure Message: the secure exchange of messages between two parties. RSA + PSS + PKCS#7 or ECC + ECDSA (based on key choice), AES GCM container.
  • Secure Storage (aka Secure Cell): provides secure storage of record based data through symmetric encryption and data authentication. AES GCM / AES CTR containers.
  • Secure Session: the establishment of a session between two peers, within which the data can be securely exchanged with higher security guarantees. EC + ECDH, AES container.
  • Secure Comparator: compare the secret between two parties without leaking anything related to the secret: Zero-Knowledge Proof-based authentication system. Hardened Socialist Millionaire Protocol + ed25519.

You can learn more about the Themis library in our general documentation.

There are also example console utils available for the Python wrapper for Themis (as well as for some other wrappers — see the full list here). They help understand the specific mechanics of encryption/decryption processes of this specific wrapper. You can find the example console utils for the Python wrapper here.

Quickstart

Requirements

In addition to the common set of build tools, Themis currently requires either the OpenSSL or LibreSSL package with the developer version of the package (as it provides header files). In either case, we strongly recommend you using the most recent version of these packages.

Installing Themis

Get Themis source code from GitHub:

git clone https://github.com/cossacklabs/themis.git

Note that the default installation process assumes the use of the standard LibreSSL/OpenSSL library and will install Themis to the standard /usr/lib and /usr/include locations.

In a typical case, all you need to do is (depending on your rights, sudo might be necessary):

make install

If your path is different, please see Building and installing for more details on how to change it (but basically you'll just need to remember into which folder you directed Themis and navigate there).

To build and run tests, you can use:

make test

All the tests need to be passed with no errors.

Building Themis for Python

To install Themis for Python, type:

pip install pythemis

To build and run exactly pythemis test suit (after it's installation), you can use:

make prepare_tests_all test_python

Using Themis in your Python project

Insert:

import pythemis

into your code.

To import a certain module (skeygen for example):

from pythemis import skeygen

Examples

Using Themis

Keypair generation

Themis supports both Elliptic Curve and RSA algorithms for asymmetric cryptography. Algorithm type is chosen according to the generated key type. Asymmetric keys are needed for Secure Message and Secure Session objects.

WARNING: When you distribute private keys to your users, make sure the keys are sufficiently protected. You can find the guidelines here.

NOTE: When using public keys of other peers, make sure they come from trusted sources.

Keypair generation interface
class KEY_PAIR_TYPE(object):
    EC = 'EC'
    RSA = 'RSA'
    CHOICES = (EC, RSA)
    
    
class GenerateKeyPair(object):
    def __init__(self, alg)
    def export_private_key(self) -> bytes
    def export_public_key(self) -> bytes

Description:

  • __init__(self, alg) - Generates key pair. Parameter alg sets the algorithm to be used: KEY_PAIR_TYPE.EC - for Elliptic Curve or KEY_PAIR_TYPE.RSA for RSA. Throws ThemisError on failure.
  • export_private_key(self) - Return bytes with generated private key.
  • export_public_key(self) - Return bytes with generated public key.
Example
from pythemis.skeygen import GenerateKeyPair, KEY_PAIR_TYPE

# or KEY_PAIR_TYPE.RSA for RSA
obj = GenerateKeyPair(KEY_PAIR_TYPE.EC)
private_key = obj.export_private_key()
public_key = obj.export_public_key()

Secure Message

The Secure Message functions provide a sequence-independent, stateless, contextless messaging system. This may be preferred in cases that don't require frequent sequential message exchange and/or in low-bandwidth contexts. This is secure enough to exchange messages from time to time, but if you'd like to have Perfect Forward Secrecy and higher security guarantees, please consider using Secure Session instead.

The Secure Message functions offer two modes of operation:

In Sign/Verify mode, the message is signed using the sender's private key and is verified by the receiver using the sender's public key. The message is packed in a suitable container and ECDSA is used by default to sign the message (when RSA key is used, RSA+PSS+PKCS#7 digital signature is used).

In Encrypt/Decrypt mode, the message will be encrypted with a randomly generated key (in RSA) or a key derived by ECDH (in ECDSA), via symmetric algorithm with Secure Cell in seal mode (keys are 256 bits long).

The mode is selected by the sender supplying a valid public key of the receiver (encrypt/decrypt) or setting this parameter to NULL, or an empty string to use sign/verify.

Read more about Secure Message's cryptographic internals here.

Secure Message interface
class SMessage(object):
   def __init__(self, private_key: bytes, peer_public_key: bytes)
   def wrap(self, message) -> bytes
   def unwrap(self, message) -> bytes

def ssign(private_key: bytes, message: bytes) -> bytes
def sverify(public_key: bytes, message: bytes) -> bytes

Description:

  • class SMessage - encrypted secure message
    • __init__(self, private_key: bytes, peer_public_key: bytes) - initialise Secure Message object with private_key and peer_public_key. Throws ThemisError on failure.
    • wrap(self, message: bytes) -> bytes - encrypt message. Return encrypted Secure Message container as binary string. Throws ThemisError on failure. Asymmetric keys are needed for Secure Message and Secure Session.
    • unwrap(self, message: bytes) -> bytes - decrypt encrypted Secure Message container passed as binary string (message). Return decrypted string. Throws ThemisError on failure.
  • signed Secure Message
    • ssign(private_key: bytes, message: bytes) -> bytes - Sign message with private_key. Return signed Secure Message container as binary string. Throws ThemisError on failure.
    • sverify(public_key: bytes, message: bytes) -> bytes - Verify binary vector contained signed secure message container with public_key. Return binary string. Throws ThemisError on failure.
Example

Initialise encrypter:

from pythemis.skeygen import KEY_PAIR_TYPE, GenerateKeyPair
from pythemis.smessage import SMessage, ssign, sverify
from pythemis.exception import ThemisError

keypair1 = GenerateKeyPair(KEY_PAIR_TYPE.EC)
keypair2 = GenerateKeyPair(KEY_PAIR_TYPE.EC)

smessage = SMessage(keypair1.export_private_key(), keypair2.export_public_key())

Encrypt message:

try:
    encrypted_message = smessage.wrap(b'some message')
except ThemisError as e:
    print(e)

Decrypt message:

try:
    message = smessage.unwrap(encrypted_message)
except ThemisError as e:
    print(e)

Sign message:

try:
    signed_message = ssign(keypair1.export_private_key(), b'some message')
except ThemisError as e:
    print(e)

Verify message:

try:
    message = sverify(keypair1.export_public_key(), signed_message)
except ThemisError as e:
    print(e)

Secure Cell

The Secure Сell functions provide the means of protection for arbitrary data contained in stores, such as database records or filesystem files. These functions provide both strong symmetric encryption and data authentication mechanisms.

The general approach is that given:

  • input: some source data to protect
  • key: a password
  • context: plus an optional "context information"

Secure Cell functions will produce:

  • cell: the encrypted data
  • authentication tag: some authentication data

The purpose of the optional "context information" (i.e. a database row number or file name) is to establish a secure association between this context and the protected data. In short, even when the password is known, if the context is incorrect, then decryption will fail.

The purpose of the authentication data is to validate that given a correct password (and context), the decrypted data is indeed the same as the original source data.

The authentication data must be stored somewhere. The most convenient way is to simply be appended it to the encrypted data, but this is not always possible due to the storage architecture of your application. The Secure Cell functions offer variants that address this issue in different ways.

The encryption algorithm used by Secure Cell (by default) is AES-256. The generated authentication data is 16 bytes long.

Secure Cell is available in 3 modes:

  • Seal mode: the mode that is the most secure and easy to use. Your best choice most of the time.
  • Token protect mode: the mode that is the most secure and easy to use. Your best choice most of the time.
  • Context imprint mode: length-preserving version of Secure Cell with no additional data stored. Should be used carefully.

You can learn more about the underlying considerations, limitations, and features here.

Secure Cell Seal mode interface
class SCellSeal(object):
   def __init__(self, key: bytes)
   def encrypt(self, message: bytes, context=None: bytes) -> bytes
   def decrypt(self, message: bytes, context=None: bytes) -> bytes

Description:

  • __init__(self, key: bytes) - initialise scell seal mode object with key.
  • encrypt(self, message: bytes, context=None: bytes) -> bytes - encrypt message with an optional context. Return encrypted message. Throws ThemisError on failure.
  • def decrypt(self, message: bytes, context=None: bytes) -> bytes - decrypt message with an optional context. Return plain message. Throws ThemisError on failure.
Example

Initialise encrypter/decrypter:

from pythemis.scell import SCellSeal
scell = SCellSeal(b'password')

Encrypt:

encrypted_message = scell.encrypt(b'message', b'context') 

Decrypt:

message = scell.decrypt(encrypted_message, b'context')  
Secure Cell Token-protect Mode
Token-protect mode interface
class SCellTokenProtect(object)
   def __init__(self, key: bytes)
   def encrypt(self, message: bytes, context=None: bytes) -> (bytes, bytes)
   def decrypt(self, message: bytes, additional_auth_data: bytes, context=None: bytes) -> bytes

Description:

  • __init__(self, key) - initialise scell token-protect mode object with key.
  • encrypt(self, message: bytes, context=None: bytes) -> (bytes, bytes) - encrypt message with an optional context. Return two binary strings containing encrypted message and token. Throws ThemisError on failure.
  • decrypt(self, message: bytes, additional_auth_data: bytes, context=None: bytes) -> bytes - decrypt message with additional_auth_data (aka token) and an optional context. Return plain message. Throws ThemisError on failure.
Example

Initialise encrypter/decrypter:

from pythemis.scell import SCellTokenProtect
scell = SCellTokenProtect(b'password')

Encrypt:

encrypted_message, additional_auth_data = scell.encrypt(b'message', b'some context') 

Decrypt:

message = scell.decrypt(encrypted_message, additional_auth_data, b'some context') 
Secure Cell Context-Imprint Mode
Context-imprint mode interface
class SCellContextImprint(object):
   def __init__(self, key: bytes)
   def encrypt(self, message: bytes, context: bytes) -> bytes
   def decrypt(self, message: bytes, context: bytes) -> bytes

Description:

  • __init__(self, key: bytes) - initialise scell context imprint mode object with key.
  • encrypt(self, message: bytes, context: bytes) -> bytes - encrypt message with context. Return encrypted message. Throws ThemisError on failure.
  • decrypt(self, message: bytes, context: bytes) -> bytes - decrypt message with context. Return plain message. Throws ThemisError on failure.
Example

Initialise encrypter/decrypter

from pythemis.scell import SCellContextImprint
scell = SCellContextImprint(b'some password')

Encrypt

encrypted_message = scell.encrypt(b'test message', b'test context') 

Decrypt

message = scell.decrypt(encrypted_message, b'test context') 

Secure Session

Secure Session is a sequence- and session- dependent, stateful messaging system. It is suitable for protecting long-lived peer-to-peer message exchanges where the secure data exchange is bound to a specific session context.

Secure Session operates in two stages: session negotiation where the keys are established and cryptographic material is exchanged to generate ephemeral keys and data exchange the where exchanging of messages can be carried out between peers.

You can read a more detailed description of the process here.

Put simply, Secure Session takes the following form:

  • Both clients and server construct a Secure Session object, providing
    • an arbitrary identifier,
    • a private key, and
    • a callback function that enables it to acquire the public key of the peers with which they may establish communication.
  • A client will generate a "connect request" and by whatever means it will dispatch that to the server.
  • A server will enter a negotiation phase in response to a client's "connect request"
  • Clients and servers will exchange messages until a "connection" is established.
  • Once a connection is established, clients and servers may exchange secure messages according to whatever application level protocol was chosen.
Secure Session interface:
class SSession(object):
   def __init__(self, id: bytes, private_key: bytes, transport: TransportStruct)
   def __del__(self)
   def connect(self)
   def send(self, message: bytes) -> int
   def receive(self) -> bytes
   def connect_request(self) -> bytes
   def wrap(self, message: bytes) -> bytes
   def unwrap(self, message: bytes) -> bytes
   def is_established(self) -> bool

Description:

  • __init__(self, id: bytes, private_key: bytes, transport: TransportStruct) - initialise Secure Session object with id, private_key and transport. Throws ThemisError on failure.
  • __del__(self) - delete Secure Session object correctly.
  • connect_request(self) -> bytes - return a Secure Session initialisation message. Throws ThemisError on failure.
  • connect(self) - create and send to peer a Secure Session initialisation message. Return nothing. Throws ThemisError on failure.
  • wrap(self, message: bytes) -> bytes - return wrapped message. Throws ThemisError on failure.
  • send(self, message: bytes) -> int - create wrapped message and send it to peer. Return internal code status. Throws ThemisError on failure.
  • unwrap(self, message: bytes) -> bytes - return pair: unwrapped message status and unwrapped message. If unwrapped message status equals to THEMIS_CODE.SEND_AS_IS, the unwrapped message must be sent to peer without any corrections. Throws ThemisError on failure.
  • receive(self) -> bytes - return message received from peer (as plain decrypted message). Throws ThemisError on failure.
Secure Session Workflow

Secure Session can be used in two ways:

  • send/receive - when communication flow is fully controlled by the Secure Session object.
  • wrap/unwrap - when communication is controlled by the user.

Secure Session has two parties that are called client and server for the sake of simplicity, but they could be more precisely called initiator and acceptor - the only difference between them is in who starts the communication.

Secure Session relies on the user's passing a number of callback functions to send/receive messages - and the keys are retrieved from local storage (see more in Secure Session cryptosystem description).

Communication flow is fully controlled by the Secure Session object

Send/Receive

Initialise callbacks class

class CustomTransport(object):
    def __init__(self, *args, **kwargs)
        #init communication channel with peer

    def send(self, message):
        # send message to peer

    def receive(self, buffer_length):
        # wait and receive buffer_length bytes from peer 
        return accepted message

    def get_pub_key_by_id(self, user_id):
        # retrieve public key for peer user_id from trusted storage (file, db etc.)    
        return public_key
Secure Session client

First, initialise session:

from pythemis.ssession import SSession
session = SSession(b'some client id', client_private_key, CustomTransport())
session.connect()
while not session.is_established():
    session.receive()

After the loop finishes, Secure Session is established and is ready to be used.

To send a message over the established session, use:

session.send(message)

To receive the message from session:

message = ssession.receive()
Secure Session server

First, initialise session:

session = SSession(b'some server id', server_private_key, CustomTransport())
# there is no need to call connect() method on the server side
while not session.is_established():
    session.receive()

Sending/receiving message works in a manner similar to the client side.

To encrypt an outgoing message, use:

encrypted_message = session.wrap(message)
# send encrypted_message to peer using any method you prefer

To decrypt the received message, use:

# receive encrypted_message from peer 
message = session.unwrap(encrypted_message)

Communication controlled by user

Secure Session wrap/unwrap

Initialise callback class:

from pythemis.ssession import MemoryTransport


class CustomSimpleTransport(MemoryTransport):
   def __init__(self, *args, **kwargs)
       # initialise trusted public keys storage
       super(CustomSimpleTransport, self).__init__()
       
   def get_pub_key_by_id(self, user_id):
       # retreive public key for peer user_id from trusted storage (file, db etc.)    
       return public_key
Secure Session client

First, the initialisation:

session = SSession(b'user_id2', client_private_key, CustomSimpleTransport())
# this call define client part
encrypted_message = session.connect_request()

# send connect request
response_bytes = user_communication_send_method(encrypted_message)    
message = session.unwrap(response_bytes)

# establish session
while not session.is_established():
    response_bytes = user_communication_send_method(message)    
    message = session.unwrap(response_bytes)

After the loop finishes, Secure Session is established and is ready to be used.

To encrypt the outgoing message, use:

encrypted_message = session.wrap(message)
# send encrypted_message to peer by any prefered method

To decrypt the received message, use:

# receive encrypted_message from peer 
message = session.unwrap(encrypted_message)
Secure Session server

First, initialise everything:

session = SSession(b'server_id_1', server_private_key, CustomSimpleTransport())
# there is no need to call connect() method on the server side
encrypted_message = user_communication_recieve_method() 
message = session.unwrap(encrypted_message)
# NOTE: The condition is different for the server because we need to send the last piece of data to 
# the client after establishing the session.
while message.is_control:
    # just return the unwrapped message to the user 
    user_communication_send_method(message)

Secure Session is ready.

Send/receive works in the same way as the client's example above.

Secure Comparator

Secure Comparator is an interactive protocol for two parties that compares whether they share the same secret or not. It is built around a Zero Knowledge Proof-based protocol (Socialist Millionaire's Protocol), with a number of security enhancements.

Secure Comparator is transport-agnostic and only requires the user(s) to pass messages in a certain sequence. The protocol itself is ingrained into the functions and requires minimal integration efforts from the developer.

Secure Comparator interface
class SComparator:
   def __init__(self, shared_secret: bytes)
   def __del__(self)
   def begin_compare(self) -> bytes
   def proceed_compare(self, message: bytes) -> bytes
   def is_compared(self) -> bool
   def is_equal(self) -> bool
   def result(self)

Description:

  • __init__(self, shared_secret: bytes) - initialise secure comparator object with a shared_secret. Throws ThemisError on failure.
  • __del__(self) - delete Secure Comparator object correctly.
  • begin_compare(self) - return a Secure Comparator initialisation message. Throws ThemisError on failure.
  • proceed_compare(self, message) - proceed with current and create the next comparation messages (when necessary). Throws ThemisError on failure.
  • is_compared(self) -> bool - return True if comparison is finished. Throws ThemisError on failure.
  • is_equal(self) -> bool - return True if secrets are equal, otherwise return False. Throws ThemisError on failure.
Secure Comparator workflow

Secure Comparator has two parties — called client and server — the only difference between them is in who starts the comparison.

Secure Comparator client
from pythemis.scomparator import SComparator
comparator = SComparator(b'shared_secret')
# this call defines the client part
comparation_message = comparator.begin_compare()
while not comparator.is_compared():
    user_send_function(comparation_message)
    response = user_recieve_function()
    comparation_message = comparator.proceed_compare(response)

After the loop finishes, the comparison is over and its result can be checked by calling: comparator.is_equal()

Secure Comparator server
from pythemis.scomparator import SComparator

comparator = SComparator(b'shared_secret')
while comparator.is_compared():
     comparation_message = user_receive_function()
     comparation_message = comparator.proceed_compare(comparation_message)
     user_send_function(comparation_message)

After the loop finishes, the comparison is over and its result can be checked by calling: comparator.is_equal().