Data Handlers

Introduction

The network library defines support for serialisation and descriptions of native Python types. This can be extended for custom types using the handler API.

Supported Types

• Integer (int)
• Floating Point (float)
• String (str)
• Boolean (bool)
• Bytes (bytes)
• Sets (set)
• Lists (list)
• Replicable reference (Replicable)
• Replicable class reference (type(Replicable))

Sequences

Sets and lists are variable sized and iterable constructs which can be replicated at full length across the network. Currently the elements of each object must be of the same (specified type). These are dangerous to use because of their bandwidth footprint with large structures.

Handler API

Requesting Handlers

To access a handler for a data type, first we must import some machinery

from network.handler_interfaces import get_handler
from network.descriptors import TypeFlag

The TypeFlag object represents a value which has certain attributes, hence should not be shared between handler requests, save for identical data variables (such as two 8 bit Integers)

eight_bit_int_flag = TypeFlag(int)
thirty_two_bit_int_flag = TypeFlag(int, max_bits=32)
# Or using another argument integers leverage
thirty_two_bit_int_flag = TypeFlag(int, max_value=(2**32-1))

Now we ask for a handler for this type

eight_bit_int_flag = TypeFlag(int)

int_handler = get_handler(eight_bit_int_flag)

Handlers have a number of methods for interfacing with Bytes, such as pack, unpack_from, unpack_merge and size. Mutable types may have defined unpack_merge in their handler, whilst this is not useful for immutable values.

We can write an integer to bytes using our handler

int_bytes = int_handler.pack(135)
# Or read bytes and return an integer
int_value = int_handler.unpack_from(int_bytes)

unpack_from will read only the necessary bytes for the handler type, so it can be used to read a length prefix which is followed by an encoded string. This is how the String handler is implemented.

Registering Handlers

To register a handler, we will need a provided function

from network.handler_interfaces import register_handler

Let's look at the Bytes handler. It is defined in the serialiser module, and uses composition to pack individual values. Handlers may be specific to individual requests, which we leverage in this case to determine the maximum length of the byte string

class BytesHandler:

    def __init__(self, type_flag):
        header_max_value = type_flag.data.get("max_length", 255)
        self.packer = handler_from_int(header_max_value)

    def pack(self, bytes_string):
        return self.packer.pack(len(bytes_string)) + bytes_string

    def size(self, bytes_string):
        length = self.packer.unpack_from(bytes_string)
        return length + self.packer.size()

    def unpack(self, bytes_string):
        return bytes_string[self.packer.size():]

    def unpack_from(self, bytes_string):
        length = self.size(bytes_string)
        return self.unpack(bytes_string[:length])

register_handler(bytes, BytesHandler, is_condition=True)

For mutable values, unpack_merge is defined as

def unpack_merge(cls, value, bytes_string):
    # Do something with value
    return None

We register the handler using register_handler, which accepts the value type it is requested for, the Handler class that is used for that value type, and an optional is_condition argument which indicates that the handler argument is callable and accepts the TypeFlag (or subclass, e.g Attribute) instance used to request it.

For integers we use a selector, rather than a class, for our condition, and return an individual handler. Calling a class object returns an instance of the handler class, and calling the below selector will return an individual handler instance.

def int_selector(type_flag):
    if "max_value" in type_flag.data:
        return handler_from_int(type_flag.data["max_value"])

    return handler_from_bit_length(type_flag.data.get('max_bits', 8))

Network Description

Typically, we use hash() for descriptions of objects, but this is not always possible. In order to solve this, there are two different ways of implementing a description method

Firstly, we can give the values to be handled a __description__" method. This is an example description defined for the network.enums.Roles` Enumeration

def __description__(self):
    return hash((self.context, self.local, self.remote))