Skip to content
Permalink
Browse files

Merge branch '3.6.x' into merge-3-6-x

  • Loading branch information...
acozzette committed Jun 25, 2018
2 parents 35567c1 + f7ada12 commit 82d3d7d250645322f8a7343188e5ae6246a76414
Showing with 628 additions and 93 deletions.
  1. +88 −0 CHANGES.txt
  2. +1 −1 Protobuf.podspec
  3. +6 −0 appveyor.yml
  4. +1 −1 configure.ac
  5. +1 −1 csharp/Google.Protobuf.Tools.nuspec
  6. +1 −1 csharp/src/Google.Protobuf/Google.Protobuf.csproj
  7. +0 −4 csharp/src/Google.Protobuf/Reflection/ReflectionUtil.cs
  8. +1 −1 java/core/pom.xml
  9. +1 −1 java/pom.xml
  10. +1 −1 java/util/pom.xml
  11. +11 −4 js/map.js
  12. +39 −1 js/maps_test.js
  13. +1 −1 js/package.json
  14. +32 −0 js/testbinary.proto
  15. +14 −0 kokoro/release/linux/build_artifacts.sh
  16. +16 −0 kokoro/release/linux/prepare_build.sh
  17. +8 −0 kokoro/release/linux/release.cfg
  18. +15 −0 kokoro/release/linux/ruby/ruby_build.sh
  19. +8 −0 kokoro/release/linux/ruby/ruby_build_environment.sh
  20. +19 −0 kokoro/release/macos/build_artifacts.sh
  21. +8 −0 kokoro/release/macos/release.cfg
  22. +15 −0 kokoro/release/macos/ruby/ruby_build.sh
  23. +57 −0 kokoro/release/macos/ruby/ruby_build_environment.sh
  24. +23 −0 kokoro/release/protoc/macos/build.sh
  25. +8 −0 kokoro/release/protoc/macos/release.cfg
  26. +27 −0 kokoro/release/protoc/windows/build.bat
  27. +8 −0 kokoro/release/protoc/windows/release.cfg
  28. +19 −3 php/ext/google/protobuf/package.xml
  29. +1 −1 php/ext/google/protobuf/protobuf.h
  30. +14 −0 php/src/Google/Protobuf/Internal/Descriptor.php
  31. +2 −0 php/src/Google/Protobuf/Internal/DescriptorPool.php
  32. +14 −0 php/src/Google/Protobuf/Internal/EnumDescriptor.php
  33. +28 −4 php/src/Google/Protobuf/Internal/GPBUtil.php
  34. +24 −1 php/src/Google/Protobuf/Internal/MapField.php
  35. +22 −1 php/src/Google/Protobuf/Internal/RepeatedField.php
  36. +33 −16 php/tests/compatibility_test.sh
  37. +1 −1 post_process_dist.sh
  38. +1 −1 protoc-artifacts/pom.xml
  39. +1 −1 python/google/protobuf/__init__.py
  40. +2 −4 python/setup.py
  41. +3 −1 ruby/ext/google/protobuf_c/extconf.rb
  42. +1 −1 ruby/google-protobuf.gemspec
  43. +3 −3 src/Makefile.am
  44. +2 −2 src/google/protobuf/any.pb.h
  45. +2 −2 src/google/protobuf/api.pb.h
  46. +5 −1 src/google/protobuf/compiler/js/js_generator.cc
  47. +2 −2 src/google/protobuf/compiler/plugin.pb.h
  48. +6 −8 src/google/protobuf/descriptor.cc
  49. +2 −2 src/google/protobuf/descriptor.pb.h
  50. +2 −2 src/google/protobuf/duration.pb.h
  51. +2 −2 src/google/protobuf/empty.pb.h
  52. +2 −2 src/google/protobuf/field_mask.pb.h
  53. +9 −0 src/google/protobuf/generated_message_util.h
  54. +2 −2 src/google/protobuf/source_context.pb.h
  55. +2 −2 src/google/protobuf/struct.pb.h
  56. +5 −5 src/google/protobuf/stubs/common.h
  57. +2 −2 src/google/protobuf/timestamp.pb.h
  58. +2 −2 src/google/protobuf/type.pb.h
  59. +2 −2 src/google/protobuf/wrappers.pb.h
@@ -1,3 +1,91 @@
2018-06-01 version 3.6.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)

C++
* Starting from this release, we now require C++11. For those we cannot yet
upgrade to C++11, we will try to keep the 3.5.x branch updated with
critical bug fixes only. If you have any concerns about this, please
comment on issue #2780.
* Moved to C++11 types like std::atomic and std::unique_ptr and away from our
old custom-built equivalents.
* Added support for repeated message fields in lite protos using implicit
weak fields. This is an experimental feature that allows the linker to
strip out more unused messages than previously was possible.
* Fixed SourceCodeInfo for interpreted options and extension range options.
* Fixed always_print_enums_as_ints option for JSON serialization.
* Added support for ignoring unknown enum values when parsing JSON.
* Create std::string in Arena memory.
* Fixed ValidateDateTime to correctly check the day.
* Fixed bug in ZeroCopyStreamByteSink.
* Various other cleanups and fixes.

Java
* Dropped support for Java 6.
* Added a UTF-8 decoder that uses Unsafe to directly decode a byte buffer.
* Added deprecation annotations to generated code for deprecated oneof
fields.
* Fixed map field serialization in DynamicMessage.
* Cleanup and documentation for Java Lite runtime.
* Various other fixes and cleanups
* Fixed unboxed arraylists to handle an edge case
* Improved performance for copying between unboxed arraylists
* Fixed lite protobuf to avoid Java compiler warnings
* Improved test coverage for lite runtime
* Performance improvements for lite runtime

Python
* Fixed bytes/string map key incompatibility between C++ and pure-Python
implementations (issue #4029)
* Added __init__.py files to compiler and util subpackages
* Use /MT for all Windows versions
* Fixed an issue affecting the Python-C++ implementation when used with
Cython (issue #2896)
* Various text format fixes
* Various fixes to resolve behavior differences between the pure-Python and
Python-C++ implementations

PHP
* Added php_metadata_namespace to control the file path of generated metadata
file.
* Changed generated classes of nested message/enum. E.g., Foo.Bar, which
previously generates Foo_Bar, now generates Foo/Bar
* Added array constructor. When creating a message, users can pass a php
array whose content is field name to value pairs into constructor. The
created message will be initialized according to the array. Note that
message field should use a message value instead of a sub-array.
* Various bug fixes.

Objective-C
* We removed some helper class methods from GPBDictionary to shrink the size
of the library, the functionary is still there, but you may need to do some
specific +alloc / -init… methods instead.
* Minor improvements in the performance of object field getters/setters by
avoiding some memory management overhead.
* Fix a memory leak during the raising of some errors.
* Make header importing completely order independent.
* Small code improvements for things the undefined behaviors compiler option
was flagging.

Ruby
* Added ruby_package file option to control the module of generated class.
* Various bug fixes.

Javascript
* Allow setting string to int64 field.

Csharp
* Unknown fields are now parsed and then sent back on the wire. They can be
discarded at parse time via a CodedInputStream option.
* Movement towards working with .NET 3.5 and Unity
* Expression trees are no longer used
* AOT generics issues in Unity/il2cpp have a workaround (see this commit for
details)
* Floating point values are now compared bitwise (affects NaN value
comparisons)
* The default size limit when parsing is now 2GB rather than 64MB
* MessageParser now supports parsing from a slice of a byte array
* JSON list parsing now accepts null values where the underlying proto
representation does

2017-12-20 version 3.5.1 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Planned Future Changes
* Make C++ implementation C++11 only: we plan to require C++11 to build
@@ -5,7 +5,7 @@
# dependent projects use the :git notation to refer to the library.
Pod::Spec.new do |s|
s.name = 'Protobuf'
s.version = '3.5.2'
s.version = '3.6.0'
s.summary = 'Protocol Buffers v.3 runtime library for Objective-C.'
s.homepage = 'https://github.com/google/protobuf'
s.license = '3-Clause BSD License'
@@ -16,6 +16,12 @@ environment:
BUILD_DLL: ON
UNICODE: ON

- platform: Win64
language: cpp
image: Visual Studio 2017
BUILD_DLL: OFF
UNICODE: ON

- platform: Win64
language: csharp
image: Visual Studio 2017
@@ -17,7 +17,7 @@ AC_PREREQ(2.59)
# In the SVN trunk, the version should always be the next anticipated release
# version with the "-pre" suffix. (We used to use "-SNAPSHOT" but this pushed
# the size of one file name in the dist tarfile over the 99-char limit.)
AC_INIT([Protocol Buffers],[3.5.2],[protobuf@googlegroups.com],[protobuf])
AC_INIT([Protocol Buffers],[3.6.0],[protobuf@googlegroups.com],[protobuf])

AM_MAINTAINER_MODE([enable])

@@ -5,7 +5,7 @@
<title>Google Protocol Buffers tools</title>
<summary>Tools for Protocol Buffers - Google's data interchange format.</summary>
<description>See project site for more info.</description>
<version>3.5.2</version>
<version>3.6.0</version>
<authors>Google Inc.</authors>
<owners>protobuf-packages</owners>
<licenseUrl>https://github.com/google/protobuf/blob/master/LICENSE</licenseUrl>
@@ -4,7 +4,7 @@
<Description>C# runtime library for Protocol Buffers - Google's data interchange format.</Description>
<Copyright>Copyright 2015, Google Inc.</Copyright>
<AssemblyTitle>Google Protocol Buffers</AssemblyTitle>
<VersionPrefix>3.5.2</VersionPrefix>
<VersionPrefix>3.6.0</VersionPrefix>
<Authors>Google Inc.</Authors>
<TargetFrameworks>netstandard1.0;net45</TargetFrameworks>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
@@ -34,10 +34,6 @@
using System;
using System.Reflection;

#if NET35
using Google.Protobuf.Compatibility;
#endif

namespace Google.Protobuf.Reflection
{
/// <summary>
@@ -6,7 +6,7 @@
<parent>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-parent</artifactId>
<version>3.5.2</version>
<version>3.6.0</version>
</parent>

<artifactId>protobuf-java</artifactId>
@@ -11,7 +11,7 @@

<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-parent</artifactId>
<version>3.5.2</version>
<version>3.6.0</version>
<packaging>pom</packaging>

<name>Protocol Buffers [Parent]</name>
@@ -6,7 +6,7 @@
<parent>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-parent</artifactId>
<version>3.5.2</version>
<version>3.6.0</version>
</parent>

<artifactId>protobuf-java-util</artifactId>
@@ -443,7 +443,8 @@ jspb.Map.prototype.serializeBinary = function(
/**
* Read one key/value message from the given BinaryReader. Compatible as the
* `reader` callback parameter to jspb.BinaryReader.readMessage, to be called
* when a key/value pair submessage is encountered.
* when a key/value pair submessage is encountered. If the Key is undefined,
* we should default it to 0.
* @template K, V
* @param {!jspb.Map} map
* @param {!jspb.BinaryReader} reader
@@ -457,19 +458,25 @@ jspb.Map.prototype.serializeBinary = function(
* readMessage, in which case the second callback arg form is used.
*
* @param {?function(V,!jspb.BinaryReader)=} opt_valueReaderCallback
* The BinaryReader parsing callback for type V, if V is a message type.
* The BinaryReader parsing callback for type V, if V is a message type
*
* @param {K=} opt_defaultKey
* The default value for the type of map keys. Accepting map
* entries with unset keys is required for maps to be backwards compatible
* with the repeated message representation described here: goo.gl/zuoLAC
*
*/
jspb.Map.deserializeBinary = function(map, reader, keyReaderFn, valueReaderFn,
opt_valueReaderCallback) {
var key = undefined;
opt_valueReaderCallback, opt_defaultKey) {
var key = opt_defaultKey;
var value = undefined;

while (reader.nextField()) {
if (reader.isEndGroup()) {
break;
}
var field = reader.getFieldNumber();

if (field == 1) {
// Key.
key = keyReaderFn.call(reader);
@@ -35,6 +35,11 @@ goog.require('goog.userAgent');
goog.require('proto.jspb.test.MapValueEnum');
goog.require('proto.jspb.test.MapValueMessage');
goog.require('proto.jspb.test.TestMapFields');
goog.require('proto.jspb.test.TestMapFieldsOptionalKeys');
goog.require('proto.jspb.test.MapEntryOptionalKeysStringKey');
goog.require('proto.jspb.test.MapEntryOptionalKeysInt32Key');
goog.require('proto.jspb.test.MapEntryOptionalKeysInt64Key');
goog.require('proto.jspb.test.MapEntryOptionalKeysBoolKey');

// CommonJS-LoadFromFile: test_pb proto.jspb.test
goog.require('proto.jspb.test.MapValueMessageNoBinary');
@@ -76,7 +81,7 @@ function toArray(iter) {
* Helper: generate test methods for this TestMapFields class.
* @param {?} msgInfo
* @param {?} submessageCtor
* @param {!string} suffix
* @param {string} suffix
*/
function makeTests(msgInfo, submessageCtor, suffix) {
/**
@@ -260,6 +265,39 @@ function makeTests(msgInfo, submessageCtor, suffix) {
var decoded = msgInfo.deserializeBinary(serialized);
checkMapFields(decoded);
});
/**
* Tests deserialization of undefined map keys go to default values in binary format.
*/
it('testMapDeserializationForUndefinedKeys', function() {
var testMessageOptionalKeys = new proto.jspb.test.TestMapFieldsOptionalKeys();
var mapEntryStringKey = new proto.jspb.test.MapEntryOptionalKeysStringKey();
mapEntryStringKey.setValue("a");
testMessageOptionalKeys.setMapStringString(mapEntryStringKey);
var mapEntryInt32Key = new proto.jspb.test.MapEntryOptionalKeysInt32Key();
mapEntryInt32Key.setValue("b");
testMessageOptionalKeys.setMapInt32String(mapEntryInt32Key);
var mapEntryInt64Key = new proto.jspb.test.MapEntryOptionalKeysInt64Key();
mapEntryInt64Key.setValue("c");
testMessageOptionalKeys.setMapInt64String(mapEntryInt64Key);
var mapEntryBoolKey = new proto.jspb.test.MapEntryOptionalKeysBoolKey();
mapEntryBoolKey.setValue("d");
testMessageOptionalKeys.setMapBoolString(mapEntryBoolKey);
var deserializedMessage = msgInfo.deserializeBinary(
testMessageOptionalKeys.serializeBinary()
);
checkMapEquals(deserializedMessage.getMapStringStringMap(), [
['', 'a']
]);
checkMapEquals(deserializedMessage.getMapInt32StringMap(), [
[0, 'b']
]);
checkMapEquals(deserializedMessage.getMapInt64StringMap(), [
[0, 'c']
]);
checkMapEquals(deserializedMessage.getMapBoolStringMap(), [
[false, 'd']
]);
});
}


@@ -1,6 +1,6 @@
{
"name": "google-protobuf",
"version": "3.5.2",
"version": "3.6.0",
"description": "Protocol Buffers for JavaScript",
"main": "google-protobuf.js",
"files": [
@@ -201,6 +201,38 @@ message TestMapFields {
map<string, TestMapFields> map_string_testmapfields = 12;
}

// These proto are 'mock map' entries to test the above map deserializing with
// undefined keys. Make sure TestMapFieldsOptionalKeys is written to be
// deserialized by TestMapFields
message MapEntryOptionalKeysStringKey {
optional string key = 1;
optional string value = 2;
}

message MapEntryOptionalKeysInt32Key {
optional int32 key = 1;
optional string value = 2;
}

message MapEntryOptionalKeysInt64Key {
optional int64 key = 1;
optional string value = 2;
}

message MapEntryOptionalKeysBoolKey {
optional bool key = 1;
optional string value = 2;
}

message TestMapFieldsOptionalKeys {
optional MapEntryOptionalKeysStringKey map_string_string = 1;
optional MapEntryOptionalKeysInt32Key map_int32_string= 8;
optional MapEntryOptionalKeysInt64Key map_int64_string = 9;
optional MapEntryOptionalKeysBoolKey map_bool_string = 10;
}

// End mock-map entries

enum MapValueEnum {
MAP_VALUE_FOO = 0;
MAP_VALUE_BAR = 1;
@@ -0,0 +1,14 @@
#!/bin/bash

set -ex

# change to repo root
cd $(dirname $0)/../../..

source kokoro/release/linux/prepare_build.sh

# ruby environment
source kokoro/release/linux/ruby/ruby_build_environment.sh

# build artifacts
bash kokoro/release/linux/ruby/ruby_build.sh
@@ -0,0 +1,16 @@
#!/bin/bash

# Move docker's storage location to scratch disk so we don't run out of space.
echo 'DOCKER_OPTS="${DOCKER_OPTS} --graph=/tmpfs/docker"' | sudo tee --append /etc/default/docker
# Use container registry mirror for pulling docker images (should make downloads faster)
# See https://cloud.google.com/container-registry/docs/using-dockerhub-mirroring
echo 'DOCKER_OPTS="${DOCKER_OPTS} --registry-mirror=https://mirror.gcr.io"' | sudo tee --append /etc/default/docker
sudo service docker restart

# Download Docker images from DockerHub
export DOCKERHUB_ORGANIZATION=grpctesting

# All artifacts come here
mkdir artifacts
export ARTIFACT_DIR=$(pwd)/artifacts

@@ -0,0 +1,8 @@
# Configuration for Linux release builds
build_file: "protobuf/kokoro/release/linux/build_artifacts.sh"

action {
define_artifacts {
regex: "github/protobuf/artifacts/**"
}
}
@@ -0,0 +1,15 @@
#!/bin/bash

# Build protoc
if test ! -e src/protoc; then
./autogen.sh
./configure
make -j4
fi

umask 0022
pushd ruby
bundle install && bundle exec rake gem:native
ls pkg
mv pkg/* $ARTIFACT_DIR
popd
@@ -0,0 +1,8 @@
#!/bin/bash

set +ex
[[ -s /etc/profile.d/rvm.sh ]] && . /etc/profile.d/rvm.sh
set -e # rvm commands are very verbose
rvm --default use ruby-2.4.1
gem install bundler --update
set -ex

0 comments on commit 82d3d7d

Please sign in to comment.
You can’t perform that action at this time.