Skip to content

Releases: protocolbuffers/protobuf

Protocol Buffers v3.0.0-alpha-2

26 Feb 09:49
Compare
Choose a tag to compare
Pre-release

Version 3.0.0-alpha-2 (C++/Java/Python/Ruby/JavaNano)

General

  • Introduced Protocol Buffers language version 3 (aka proto3).

    When protobuf was initially opensourced it implemented Protocol Buffers
    language version 2 (aka proto2), which is why the version number
    started from v2.0.0. From v3.0.0, a new language version (proto3) is
    introduced while the old version (proto2) will continue to be supported.

    The main intent of introducing proto3 is to clean up protobuf before
    pushing the language as the foundation of Google's new API platform.
    In proto3, the language is simplified, both for ease of use and to
    make it available in a wider range of programming languages. At the
    same time a few features are added to better support common idioms
    found in APIs.

    The following are the main new features in language version 3:

    1. Removal of field presence logic for primitive value fields, removal
      of required fields, and removal of default values. This makes proto3
      significantly easier to implement with open struct representations,
      as in languages like Android Java, Objective C, or Go.
    2. Removal of unknown fields.
    3. Removal of extensions, which are instead replaced by a new standard
      type called Any.
    4. Fix semantics for unknown enum values.
    5. Addition of maps.
    6. Addition of a small set of standard types for representation of time,
      dynamic data, etc.
    7. A well-defined encoding in JSON as an alternative to binary proto
      encoding.

    This release (v3.0.0-alpha-2) includes partial proto3 support for C++,
    Java, Python, Ruby and JavaNano. Items 6 (well-known types) and 7
    (JSON format) in the above feature list are not implemented.

    A new notion "syntax" is introduced to specify whether a .proto file
    uses proto2 or proto3:

    // foo.proto
    syntax = "proto3";
    message Bar {...}
    

    If omitted, the protocol compiler will generate a warning and "proto2" will
    be used as the default. This warning will be turned into an error in a
    future release.

    We recommend that new Protocol Buffers users use proto3. However, we do not
    generally recommend that existing users migrate from proto2 from proto3 due
    to API incompatibility, and we will continue to support proto2 for a long
    time.

  • Added support for map fields (implemented in proto2 and proto3 C++/Java/JavaNano and proto3 Ruby).

    Map fields can be declared using the following syntax:

    message Foo {
      map<string, string> values = 1;
    }
    

    Data of a map field will be stored in memory as an unordered map and it
    can be accessed through generated accessors.

C++

  • Added arena allocation support (for both proto2 and proto3).

    Profiling shows memory allocation and deallocation constitutes a significant
    fraction of CPU-time spent in protobuf code and arena allocation is a
    technique introduced to reduce this cost. With arena allocation, new
    objects will be allocated from a large piece of preallocated memory and
    deallocation of these objects is almost free. Early adoption shows 20% to
    50% improvement in some Google binaries.

    To enable arena support, add the following option to your .proto file:

    option cc_enable_arenas = true;
    

    Protocol compiler will generate additional code to make the generated
    message classes work with arenas. This does not change the existing API
    of protobuf messages and does not affect wire format. Your existing code
    should continue to work after adding this option. In the future we will
    make this option enabled by default.

    To actually take advantage of arena allocation, you need to use the arena
    APIs when creating messages. A quick example of using the arena API:

    {
      google::protobuf::Arena arena;
      // Allocate a protobuf message in the arena.
      MyMessage* message = Arena::CreateMessage<MyMessage>(&arena);
      // All submessages will be allocated in the same arena.
      if (!message->ParseFromString(data)) {
        // Deal with malformed input data.
      }
      // Must not delete the message here. It will be deleted automatically
      // when the arena is destroyed.
    }
    

    Currently arena does not work with map fields. Enabling arena in a .proto
    file containing map fields will result in compile errors in the generated
    code. This will be addressed in a future release.

Python

  • Python has received several updates, most notably support for proto3
    semantics in any .proto file that declares syntax="proto3".
    Messages declared in proto3 files no longer represent field presence
    for scalar fields (number, enums, booleans, or strings). You can
    no longer call HasField() for such fields, and they are serialized
    based on whether they have a non-zero/empty/false value.
  • One other notable change is in the C++-accelerated implementation.
    Descriptor objects (which describe the protobuf schema and allow
    reflection over it) are no longer duplicated between the Python
    and C++ layers. The Python descriptors are now simple wrappers
    around the C++ descriptors. This change should significantly
    reduce the memory usage of programs that use a lot of message
    types.

Ruby

  • We have added proto3 support for Ruby via a native C extension.

    The Ruby extension itself is included in the ruby/ directory, and details on
    building and installing the extension are in ruby/README.md. The extension
    will also be published as a Ruby gem. Code generator support is included as
    part of protoc with the --ruby_out flag.

    The Ruby extension implements a user-friendly DSL to define message types
    (also generated by the code generator from .proto files). Once a message
    type is defined, the user may create instances of the message that behave in
    ways idiomatic to Ruby. For example:

    • Message fields are present as ordinary Ruby properties (getter method
      foo and setter method foo=).
    • Repeated field elements are stored in a container that acts like a native
      Ruby array, and map elements are stored in a container that acts like a
      native Ruby hashmap.
    • The usual well-known methods, such as #to_s, #dup, and the like, are
      present.

    Unlike several existing third-party Ruby extensions for protobuf, this
    extension is built on a "strongly-typed" philosophy: message fields and
    array/map containers will throw exceptions eagerly when values of the
    incorrect type are inserted.

    See ruby/README.md for details.

JavaNano

  • JavaNano is a special code generator and runtime library designed especially
    for resource-restricted systems, like Android. It is very resource-friendly
    in both the amount of code and the runtime overhead. Here is an an overview
    of JavaNano features compared with the official Java protobuf:

    • No descriptors or message builders.
    • All messages are mutable; fields are public Java fields.
    • For optional fields only, encapsulation behind setter/getter/hazzer/
      clearer functions is opt-in, which provide proper 'has' state support.
    • For proto2, if not opted in, has state (field presence) is not available.
      Serialization outputs all fields not equal to their defaults.
      The behavior is consistent with proto3 semantics.
    • Required fields (proto2 only) are always serialized.
    • Enum constants are integers; protection against invalid values only
      when parsing from the wire.
    • Enum constants can be generated into container interfaces bearing
      the enum's name (so the referencing code is in Java style).
    • CodedInputByteBufferNano can only take byte[](not InputStream).
    • Similarly CodedOutputByteBufferNano can only write to byte[].
    • Repeated fields are in arrays, not ArrayList or Vector. Null array
      elements are allowed and silently ignored.
    • Full support for serializing/deserializing repeated packed fields.
    • Support extensions (in proto2).
    • Unset messages/groups are null, not an immutable empty default
      instance.
    • toByteArray(...) and mergeFrom(...) are now static functions of
      MessageNano.
    • The 'bytes' type translates to the Java type byte[].

    See javanano/README.txt for details.

Protocol Buffers v3.0.0-alpha-1

11 Dec 02:39
Compare
Choose a tag to compare
Pre-release

Version 3.0.0-alpha-1 (C++/Java)

General

  • Introduced Protocol Buffers language version 3 (aka proto3).

    When protobuf was initially opensourced it implemented Protocol Buffers
    language version 2 (aka proto2), which is why the version number
    started from v2.0.0. From v3.0.0, a new language version (proto3) is
    introduced while the old version (proto2) will continue to be supported.

    The main intent of introducing proto3 is to clean up protobuf before
    pushing the language as the foundation of Google's new API platform.
    In proto3, the language is simplified, both for ease of use and to
    make it available in a wider range of programming languages. At the
    same time a few features are added to better support common idioms
    found in APIs.

    The following are the main new features in language version 3:

    1. Removal of field presence logic for primitive value fields, removal
      of required fields, and removal of default values. This makes proto3
      significantly easier to implement with open struct representations,
      as in languages like Android Java, Objective C, or Go.
    2. Removal of unknown fields.
    3. Removal of extensions, which are instead replaced by a new standard
      type called Any.
    4. Fix semantics for unknown enum values.
    5. Addition of maps.
    6. Addition of a small set of standard types for representation of time,
      dynamic data, etc.
    7. A well-defined encoding in JSON as an alternative to binary proto
      encoding.

    This release (v3.0.0-alpha-1) includes partial proto3 support for C++ and
    Java. Items 6 (well-known types) and 7 (JSON format) in the above feature
    list are not impelmented.

    A new notion "syntax" is introduced to specify whether a .proto file
    uses proto2 or proto3:

    // foo.proto
    syntax = "proto3";
    message Bar {...}
    

    If omitted, the protocol compiler will generate a warning and "proto2" will
    be used as the default. This warning will be turned into an error in a
    future release.

    We recommend that new Protocol Buffers users use proto3. However, we do not
    generally recommend that existing users migrate from proto2 from proto3 due
    to API incompatibility, and we will continue to support proto2 for a long
    time.

  • Added support for map fields (implemented in C++/Java for both proto2 and
    proto3).

    Map fields can be declared using the following syntax:

    message Foo {
      map<string, string> values = 1;
    }
    

    Data of a map field will be stored in memory as an unordered map and it
    can be accessed through generated accessors.

C++

  • Added arena allocation support (for both proto2 and proto3).

    Profiling shows memory allocation and deallocation constitutes a significant
    fraction of CPU-time spent in protobuf code and arena allocation is a
    technique introduced to reduce this cost. With arena allocation, new
    objects will be allocated from a large piece of preallocated memory and
    deallocation of these objects is almost free. Early adoption shows 20% to
    50% improvement in some Google binaries.

    To enable arena support, add the following option to your .proto file:

    option cc_enable_arenas = true;
    

    Protocol compiler will generate additional code to make the generated
    message classes work with arenas. This does not change the existing API
    of protobuf messages and does not affect wire format. Your existing code
    should continue to work after adding this option. In the future we will
    make this option enabled by default.

    To actually take advantage of arena allocation, you need to use the arena
    APIs when creating messages. A quick example of using the arena API:

    {
      google::protobuf::Arena arena;
      // Allocate a protobuf message in the arena.
      MyMessage* message = Arena::CreateMessage<MyMessage>(&arena);
      // All submessages will be allocated in the same arena.
      if (!message->ParseFromString(data)) {
        // Deal with malformed input data.
      }
      // Must not delete the message here. It will be deleted automatically
      // when the arena is destroyed.
    }
    

    Currently arena does not work with map fields. Enabling arena in a .proto
    file containing map fields will result in compile errors in the generated
    code. This will be addressed in a future release.

Protocol Buffers v2.6.1

21 Oct 23:20
Compare
Choose a tag to compare

2014-10-20 version 2.6.1

C++

  • Added atomicops support for Solaris.
  • Released memory allocated by InitializeDefaultRepeatedFields() and GetEmptyString(). Some memory sanitizers reported them as memory leaks.

Java

  • Updated DynamicMessage.setField() to handle repeated enum values correctly.
  • Fixed a bug that caused NullPointerException to be thrown when converting manually constructed FileDescriptorProto to FileDescriptor.

Python

  • Fixed WhichOneof() to work with de-serialized protobuf messages.
  • Fixed a missing file problem of Python C++ implementation.

v2.6.0

28 Aug 00:03
Compare
Choose a tag to compare

2014-08-15 version 2.6.0

General

  • Added oneofs(unions) feature. Fields in the same oneof will share
    memory and at most one field can be set at the same time. Use the
    oneof keyword to define a oneof like:

    message SampleMessage {
      oneof test_oneof {
        string name = 4;
        YourMessage sub_message = 9;
      }
    }
    
  • Files, services, enums, messages, methods and enum values can be marked
    as deprecated now.

  • Added Support for list values, including lists of mesaages, when
    parsing text-formatted protos in C++ and Java.

    For example:  foo: [1, 2, 3]
    

C++

  • Enhanced customization on TestFormat printing.
  • Added SwapFields() in reflection API to swap a subset of fields.
    Added SetAllocatedMessage() in reflection API.
  • Repeated primitive extensions are now packable. The
    [packed=true] option only affects serializers. Therefore, it is
    possible to switch a repeated extension field to packed format
    without breaking backwards-compatibility.
  • Various speed optimizations.

Java

  • writeTo() method in ByteString can now write a substring to an
    output stream. Added endWith() method for ByteString.
  • ByteString and ByteBuffer are now supported in CodedInputStream
    and CodedOutputStream.
  • java_generate_equals_and_hash can now be used with the LITE_RUNTIME.

Python

  • A new C++-backed extension module (aka "cpp api v2") that replaces the
    old ("cpp api v1") one. Much faster than the pure Python code. This one
    resolves many bugs and is recommended for general use over the
    pure Python when possible.
  • Descriptors now have enum_types_by_name and extension_types_by_name dict
    attributes.
  • Support for Python 3.

Protocol Buffers v2.5.0

25 Mar 00:49
Compare
Choose a tag to compare

Version 2.5.0

General

  • New notion "import public" that allows a proto file to forward the content
    it imports to its importers. For example,

    // foo.proto
    import public "bar.proto";
    import "baz.proto";
    
    // qux.proto
    import "foo.proto";
    // Stuff defined in bar.proto may be used in this file, but stuff from
    // baz.proto may NOT be used without importing it explicitly.
    

    This is useful for moving proto files. To move a proto file, just leave
    a single "import public" in the old proto file.

  • New enum option "allow_alias" that specifies whether different symbols can
    be assigned the same numeric value. Default value is "true". Setting it to
    false causes the compiler to reject enum definitions where multiple symbols
    have the same numeric value.
    Note: We plan to flip the default value to "false" in a future release.
    Projects using enum aliases should set the option to "true" in their .proto
    files.

C++

  • New generated method set_allocated_foo(Type* foo) for message and string
    fields. This method allows you to set the field to a pre-allocated object
    and the containing message takes the ownership of that object.
  • Added SetAllocatedExtension() and ReleaseExtension() to extensions API.
  • Custom options are now formatted correctly when descriptors are printed in
    text format.
  • Various speed optimizations.

Java

  • Comments in proto files are now collected and put into generated code as
    comments for corresponding classes and data members.

  • Added Parser to parse directly into messages without a Builder. For
    example,

    Foo foo = Foo.PARSER.ParseFrom(input);
    

    Using Parser is ~25% faster than using Builder to parse messages.

  • Added getters/setters to access the underlying ByteString of a string field
    directly.

  • ByteString now supports more operations: substring(), prepend(), and
    append(). The implementation of ByteString uses a binary tree structure
    to support these operations efficiently.

  • New method findInitializationErrors() that lists all missing required
    fields.

  • Various code size and speed optimizations.

Python

  • Added support for dynamic message creation. DescriptorDatabase,
    DescriptorPool, and MessageFactory work like their C++ couterparts to
    simplify Descriptor construction from *DescriptorProtos, and MessageFactory
    provides a message instance from a Descriptor.
  • Added pickle support for protobuf messages.
  • Unknown fields are now preserved after parsing.
  • Fixed bug where custom options were not correctly populated. Custom
    options can be accessed now.
  • Added EnumTypeWrapper that provides better accessibility to enum types.
  • Added ParseMessage(descriptor, bytes) to generate a new Message instance
    from a descriptor and a byte string.

Protocol Buffers v2.4.1

25 Mar 00:49
Compare
Choose a tag to compare

Version 2.4.1

C++

  • Fixed the frendship problem for old compilers to make the library now gcc 3
    compatible again.
  • Fixed vcprojects/extract_includes.bat to extract compiler/plugin.h.

Java

  • Removed usages of JDK 1.6 only features to make the library now JDK 1.5
    compatible again.
  • Fixed a bug about negative enum values.
  • serialVersionUID is now defined in generated messages for java serializing.
  • Fixed protoc to use java.lang.Object, which makes "Object" now a valid
    message name again.

Python

  • Experimental C++ implementation now requires C++ protobuf library installed.
    See the README.txt in the python directory for details.