Permalink
Find file
Fetching contributors…
Cannot retrieve contributors at this time
266 lines (258 sloc) 13 KB
<html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"><title>The C Code Generator</title><meta name="generator" content="DocBook XSL Stylesheets V1.71.0"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="article" lang="en"><div class="titlepage"><div><div><h2 class="title"><a name="id2425612"></a>The C Code Generator</h2></div></div><hr></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="#id2478318">Design</a></span></dt><dt><span class="section"><a href="#id2517105">The Generated Code</a></span></dt><dt><span class="section"><a href="#id2478728">The protobuf-c Library</a></span></dt><dt><span class="section"><a href="#id2478765">protobuf-c: the Descriptor structures</a></span></dt><dt><span class="section"><a href="#id2478802">protobuf-c: helper structures and typedefs</a></span></dt><dd><dl><dt><span class="section"><a href="#buffers">Buffers</a></span></dt></dl></dd><dt><span class="section"><a href="#id2479168">protobuf-c: packing and unpacking messages</a></span></dt><dt><span class="section"><a href="#id2479561">Author</a></span></dt></dl></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="id2478318"></a>Design</h2></div></div></div><p>The overall goal is to keep the code-generator as simple
as possible. Hopefully performance isn't sacrificed to that end!</p><p>Anyways, we generate very little code: we mostly generate
structure definitions (for example enums and structures
for messages) and some metadata which is basically
reflection-type data.</p><p>The serializing and deserializing is implemented in a library,
called libprotobuf-c rather than generated code.</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="id2517105"></a>The Generated Code</h2></div></div></div><p>
For each enum, we generate a C enum.
For each message, we generate a C structure
which can be cast to a <span class="type">ProtobufCMessage</span>.
</p><p>
For each enum and message, we generate a descriptor
object that allows us to implement a kind of reflection
on the structures.
</p><p>First, some naming conventions:
</p><div class="itemizedlist"><ul type="disc"><li><p>
The name of the type for enums and messages and services
is camel case (meaning WordsAreCrammedTogether)
except that double-underscores are used to delimit
scopes. For example:
</p><pre class="programlisting">
package foo.bar;
message BazBah {
int32 val;
}
</pre><p>
would generate a C type <span class="type">Foo__Bar__BazBah</span>.</p></li><li><p>Functions and globals are all lowercase, with camel-case
words separated by single underscores.
For example:
</p><pre class="programlisting">
Foo__Bar__BazBah *foo__bar__baz_bah__unpack
(ProtobufCAllocator *allocator,
size_t length,
const unsigned char *data);
</pre><p>
</p></li><li><p>Enums values are all uppercase.</p></li><li><p>
Stuff we dd to your symbol names will also be
separated by a double-underscore. For example,
the unpack method above.</p></li></ul></div><p>
</p><p>
We also generate descriptor objects for messages
and enums. These are declared in the .h files:
</p><pre class="programlisting">
extern const ProtobufCMessageDescriptor
foo__bar__baz_bah__descriptor;
</pre><p>
</p><p>
The message structures all begin with <span class="type">ProtobufCMessageDescriptor*</span>
which is sufficient to allow them to be cast to <span class="type">ProtobufCMessage</span>.
</p><p>
We generate some functions for each message:
</p><div class="itemizedlist"><ul type="disc"><li><p><code class="function">unpack()</code>. Unpack data for a particular
message-format:
</p><pre class="programlisting">
Foo__Bar__BazBah *
foo__bar__baz_bah__unpack (ProtobufCAllocator *allocator,
size_t length,
const unsigned char *data);
</pre><p>
Note that <em class="parameter"><code>allocator</code></em> may be NULL.
</p></li><li><p><code class="function">free_unpacked()</code>. Free a message
that you obtained with the unpack method:
</p><pre class="programlisting">
void
foo__bar__baz_bah__free_unpacked (Foo__Bar__BazBah *baz_bah,
ProtobufCAllocator *allocator);
</pre><p>
</p></li><li><p><code class="function">get_packed_size()</code>. Find how long
the serialized representation of the data will be:
message-format:
</p><pre class="programlisting">
size_t
foo__bar__baz_bah__get_packed_size
(const Foo__Bar__BazBah *message);
</pre><p>
</p></li><li><p><code class="function">pack()</code>. Pack message
into buffer; assumes that buffer is long enough (use get_packed_size first!).
</p><pre class="programlisting">
size_t
foo__bar__baz_bah__pack
(const Foo__Bar__BazBah *message,
unsigned char *packed_data_out);
</pre><p>
</p></li><li><p><code class="function">pack_to_buffer()</code>. Pack message
into virtualize buffer.
</p><pre class="programlisting">
size_t
foo__bar__baz_bah__pack_to_buffer
(const Foo__Bar__BazBah *message,
ProtobufCBuffer *buffer);
</pre><p>
</p></li></ul></div><p>
</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="id2478728"></a>The protobuf-c Library</h2></div></div></div><p>This library is used by the generated code;
it includes common structures and enums,
as well as functions that most users of the generated code
will want.</p><p>
There are three main components:
</p><div class="orderedlist"><ol type="1"><li><p>the Descriptor structures</p></li><li><p>helper structures and objects</p></li><li><p>packing and unpacking code</p></li></ol></div><p>
</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="id2478765"></a>protobuf-c: the Descriptor structures</h2></div></div></div><p>For example, enums are described in terms of structures:
</p><pre class="programlisting">
struct _ProtobufCEnumValue
{
const char *name;
const char *c_name;
int value;
};
struct _ProtobufCEnumDescriptor
{
const char *name;
const char *short_name;
const char *package_name;
/* sorted by value */
unsigned n_values;
const ProtobufCEnumValue *values;
/* sorted by name */
unsigned n_value_names;
const ProtobufCEnumValue *values_by_name;
};
</pre><p>Likewise, messages are described by:
</p><pre class="programlisting">
struct _ProtobufCFieldDescriptor
{
const char *name;
int id;
ProtobufCFieldLabel label;
ProtobufCFieldType type;
unsigned quantifier_offset;
unsigned offset;
void *descriptor; /* for MESSAGE and ENUM types */
};
struct _ProtobufCMessageDescriptor
{
const char *name;
const char *short_name;
const char *package_name;
/* sorted by field-id */
unsigned n_fields;
const ProtobufCFieldDescriptor *fields;
};
</pre><p>
And finally services are described by:
</p><pre class="programlisting">
struct _ProtobufCMethodDescriptor
{
const char *name;
const ProtobufCMessageDescriptor *input;
const ProtobufCMessageDescriptor *output;
};
struct _ProtobufCServiceDescriptor
{
const char *name;
unsigned n_methods;
ProtobufCMethodDescriptor *methods; // sorted by name
};
</pre></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="id2478802"></a>protobuf-c: helper structures and typedefs</h2></div></div></div><p>We defined typedefs for a few types
which are used in .proto files but do not
have obvious standard C equivalents:
</p><div class="itemizedlist"><ul type="disc"><li><p>a boolean type (<span class="type">protobuf_c_boolean</span>)</p></li><li><p>a binary-data (bytes) type (<span class="type">ProtobufCBinaryData</span>)</p></li><li><p>the various int types (<span class="type">int32_t</span>, <span class="type">uint32_t</span>, <span class="type">int64_t</span>, <span class="type">uint64_t</span>)
are obtained by including <code class="filename">inttypes.h</code></p></li></ul></div><p>
</p><p>We also define a simple allocator object, ProtobufCAllocator
that let's you control how allocations are done.
This is predominately used for parsing.</p><p>There is a virtual buffer facility that
only has to implement a method to append binary-data
to the buffer. This can be used to serialize messages
to different targets (instead of a flat slab of data).</p><p>We define a base-type for all messages,
for code that handles messages generically.
All it has is the descriptor object.</p><div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="buffers"></a>Buffers</h3></div></div></div><p>One important helper type is the <span class="type">ProtobufCBuffer</span>
which allows you to abstract the target of serialization. The only
thing that a buffer has is an <code class="function">append</code> method:
</p><pre class="programlisting">
struct _ProtobufCBuffer
{
void (*append)(ProtobufCBuffer *buffer,
size_t len,
const unsigned char *data);
}
</pre><p>
ProtobufCBuffer subclasses are often defined on the stack.
</p><p>
For example, to write to a <span class="type">FILE</span> you could make:
</p><pre class="programlisting">
typedef struct
{
ProtobufCBuffer base;
FILE *fp;
} BufferAppendToFile
static void my_buffer_file_append (ProtobufCBuffer *buffer,
unsigned len,
const unsigned char *data)
{
BufferAppendToFile *file_buf = (BufferAppendToFile *) buffer;
fwrite (data, len, 1, file_buf-&gt;fp); // XXX: no error handling!
}
</pre><p>
</p><p>
To use this new type of Buffer, you would do something like:
</p><pre class="programlisting">
...
BufferAppendToFile tmp;
tmp.base.append = my_buffer_file_append;
tmp.fp = fp;
protobuf_c_message_pack_to_buffer (&amp;message, &amp;tmp);
...
</pre><p>
</p><p>
A commonly builtin subtype is the BufferSimple
which is declared on the stack and uses a scratch buffer provided by the user
for its initial allocation. It does exponential resizing.
To create a BufferSimple, use code like:
</p><pre class="programlisting">
unsigned char pad[128];
ProtobufCBufferSimple buf = PROTOBUF_C_BUFFER_SIMPLE_INIT (pad);
ProtobufCBuffer *buffer = (ProtobufCBuffer *) &amp;simple;
protobuf_c_buffer_append (buffer, 6, (unsigned char *) "hi mom");
</pre><p>
You can access the data as buf.len and buf.data. For example,
</p><pre class="programlisting">
assert (buf.len == 6);
assert (memcmp (buf.data, "hi mom", 6) == 0);
</pre><p>
To finish up, use:
</p><pre class="programlisting">
PROTOBUF_C_BUFFER_SIMPLE_CLEAR (&amp;buf);
</pre><p>
</p></div></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="id2479168"></a>protobuf-c: packing and unpacking messages</h2></div></div></div><p>
To pack messages one first computes their packed size,
then provide a buffer to pack into.
</p><pre class="programlisting">
size_t protobuf_c_message_get_packed_size
(ProtobufCMessage *message);
void protobuf_c_message_pack (ProtobufCMessage *message,
unsigned char *out);
</pre><p>
</p><p>
Or you can use the "streaming" approach:
</p><pre class="programlisting">
void protobuf_c_message_pack_to_buffer
(ProtobufCMessage *message,
ProtobufCBuffer *buffer);
</pre><p>
where <span class="type">ProtobufCBuffer</span> is a base object with an append metod.
See <a href="#buffers" title="Buffers">the section called &#8220;Buffers&#8221;</a>.
</p><p>
To unpack messages, you should simple call
</p><pre class="programlisting">
ProtobufCMessage *
protobuf_c_message_unpack (const ProtobufCMessageDescriptor *,
ProtobufCAllocator *allocator,
size_t len,
const unsigned char *data);
</pre><p>
If you pass NULL for <em class="parameter"><code>allocator</code></em>, then
the default allocator will be used.
</p><p>
You can cast the result to the type that matches
the descriptor.
</p><p>
The result of unpacking should be freed with protobuf_c_message_free().
</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="id2479561"></a>Author</h2></div></div></div><p>Dave Benson.</p></div></div></body></html>