Serialization Article

articles/service-fabric/service-fabric-develop-your-service-index.md

@@ -73,6 +73,7 @@ Reliable Services gives you a simple, powerful, top-level programming model to h
 - [Architecture](service-fabric-reliable-services-platform-architecture.md)
 - [Reliable Collections](service-fabric-reliable-services-reliable-collections.md)
 - [Configuring Stateful Reliable Services](../Service-Fabric/service-fabric-reliable-services-configuration.md)
+- [Serialization](../Service-Fabric/service-fabric-reliable-services-serialization.md)
 - [Reliable Services Programming Model Advanced Usage](../Service-Fabric/service-fabric-reliable-services-advanced-usage.md)
 
 Communicating with Reliable Services and the abstractions which clients can use to discover and communicate with the service endpoints are described in the following:

articles/service-fabric/service-fabric-reliable-services-serialization.md

@@ -0,0 +1,147 @@
+<properties
+   pageTitle="Reliable Service Serialization | Microsoft Azure"
+   description="Conceptual Documentation for Service Fabric Reliable Service Serialization"
+   services="service-fabric"
+   documentationCenter=".net"
+   authors="mcoskun"
+   manager="timlt"
+   editor="subramar,jessebenson,tyadam"/>
+
+<tags
+   ms.service="service-fabric"
+   ms.devlang="dotnet"
+   ms.topic="article"
+   ms.tgt_pltfrm="na"
+   ms.workload="na"
+   ms.date="11/18/2015"
+   ms.author="mcoskun"/>
+
+# Serialization in Reliable Services
+Reliable State Manager can hold multiple Reliable Objects.
+Some of these Reliable Objects can be generic data structures like the out of the box Reliable Dictionary and Reliable Queue.
+Since these are reliable generic data structures, the generic objects they hold must be serialized so that they can be replicated and persisted to disk.
+
+Reliable State Manager supports three kinds of serializers
+* Custom serializers
+* Built-in serializers for limited number of types
+* DataContractSerilizer
+
+When a Reliable Object needs to serialize an object, it queries the Reliable State Manager for a serializer for the given type.
+Reliable State Manager will first check if there is a custom serializer registered for the input type.
+If not, it will check if one of the built-in serializers can serialize the it.
+Reliable State Manager has built-in serializers for the following types: guid, bool, byte, sbyte, char, decimal, double, float, int, uint, long, ulong, short, ushort and string.
+If not, it will return the DataContractSerializer.
+
+This article focuses on when and how to use custom serialization.
+
+## When to use custom serialization
+There are two common reasons to use custom serialization
+* Performance
+* Encryption
+
+For types that are not covered by the built-in types, considerable performance improvements can be achieved by using custom built serializers instead of DataContractSerializer.
+One reason is that custom serializers do not need to serialize type information.
+Service Fabric guarantees that a state serializer for a given type will only be given that type to serialize and deserialize.
+
+The serialized data generated by the serializers are replicated and persisted on disk as is.
+For data that is confidential, it might be desirable to make sure the bits on the wire an disk are encrypted.
+
+## How to use custom serialization
+To use a custom serializer for a given type, we need to
+* Build a custom state serializer
+* Register the custom state serializer in the Reliable Service
+
+### How to implement a custom serializer
+Custom serializers need to implement the IStateSerializer<T> interface.
+The two core methods in this interface are
+- T Read(BinaryReader binaryReader);
+- void Write(T value, BinaryWriter binaryWriter);
+
+First is used by the ReliableObject to read the serialized object from a stream using a BinaryReader.
+Second is used for the reverse operation: to write the object into a stream using a Binary Writer.
+
+Below is an example custom class and a serializer for it.
+
+```C#
+public class OrderKey : IComparable<OrderKey>, IEquatable<OrderKey>
+{
+    public byte Warehouse { get; set; }
+    public short District { get; set; }
+    public int Customer { get; set; }
+    public long Order { get; set; }
+```
+
+```C#
+public class OrderKeySerializer : IStateSerializer<OrderKey>
+{
+    void IStateSerializer<OrderKey>.Write(OrderKey value, BinaryWriter writer)
+    {
+        writer.Write(value.Warehouse);
+        writer.Write(value.District);
+        writer.Write(value.Customer);
+        writer.Write(value.Order);
+    }
+
+    OrderKey IStateSerializer<OrderKey>.Read(BinaryReader reader)
+    {
+        OrderKey value = new OrderKey();
+        value.Warehouse = reader.ReadByte();
+        value.District = reader.ReadInt16();
+        value.Customer = reader.ReadInt32();
+        value.Order = reader.ReadInt64();
+
+        return value;
+    }
+
+    void IStateSerializer<OrderKey>.Write(OrderKey currentValue, OrderKey newValue, BinaryWriter writer)
+    {
+        ((IStateSerializer<OrderKey>)this).Write(newValue, writer);
+    }
+
+    OrderKey IStateSerializer<OrderKey>.Read(OrderKey baseValue, BinaryReader reader)
+    {
+        return ((IStateSerializer<OrderKey>)this).Read(reader);
+    }
+}
+```
+>[AZURE.NOTE] In the above example, implementation of the Read and Write overloads simply call their counterpart overloads.
+This is because the following two methods are going to be used for a feature that is not yet available.
+
+### How to register a customer serializer
+To register a custom serializer, we first need a method that can register all the custom serializers.
+This method needs to take no arguments and be able to return a Task.
+In this method, IReliableStateManager.TryAddStateSerializer<T> must be used to register all custom serializers for the Reliable Service.
+
+```C#
+protected Task InitializeStateSerializers()
+{
+    this.StateManager.TryAddStateSerializer(new OrderKeySerializer());
+    return Task.FromResult(false);
+}
+```
+
+Next step is to register the above method as a delegate to be called by the Reliable State Manager at the time when all custom state serializers should be registered.
+This method is only called at the start of the Reliable Service replica before the local recovery starts, since serializers might be required to read the serialized data from the disk.
+Once the serializer is registered, the relevant type from all Reliable Objects will use this serializer to serialize their objects.
+
+```C#
+protected override IReliableStateManager CreateReliableStateManager()
+{
+    return new ReliableStateManager(
+        new ReliableStateManagerConfiguration(
+            onInitializeStateSerializersEvent : this.InitializeStateSerializers));
+}
+```
+### Versioning
+Service Fabric expects the serializers to be infinitely forwards and backwards compatible.
+For the types that are using built-in serializers, Service Fabric ensures forwards and backwards compatibility.
+For the types that using DataContractSerializer or custom serializer, user is required to never do a breaking change.
+If a breaking change is required, state needs to be moved from a service instance with the old data version to a service with the new data version at application level.
+
+### When is a Serializer used
+ 1. Write operations on Reliable Objects will cause it to be serialized, replicated, and stored in a Log.
+ 2. After enough operations are logged, the latest data from memory is serialized and checkpointed to disk.
+ 3. For re-building a replica, the checkpointed files on disk and recent data from the Log are directly sent byte-for-byte (i.e. the data is not re-serialized) from a primary replica. These bytes are deserialized into C# objects on the secondary replica.
+ 4. During recovery, the checkpoint files and recent data Log are deserialized.
+ 5. During backup, the checkpoint files and recent log data are copied byte-for-byte.
+ 6. During restore, the previously backed up checkpoint files and Log data are copied back into place and will be deserialized.