forked from protocolbuffers/protobuf
-
Notifications
You must be signed in to change notification settings - Fork 0
/
CHANGES.txt
2070 lines (1797 loc) · 93.5 KB
/
CHANGES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
2019-04-29 version 3.8.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
C++
* Use std::atomic<int32> in case of myriad2 platform
* Always declare enums to be int-sized
* Added DebugString() and ShortDebugString() methods on MessageLite
* Specialized different parse loop control flows
* Make hasbits potentially in register. The or's start forming an obstacle because it's a read modify store on the same mem address on each iteration.
* Move to an internal MACRO for parser validity checks.
* Improve map parsing performance.
* Make MergePartialFromCodedStream non virtual. This allows direct calls, potential inlining and is also a code health improvement
* Add an overall limit to parse_context to prevent reading past it. This allows to remove a annoying level of indirection.
* Fix a mistake, we shouldn't verify map key/value strings for utf8 in opt mode for proto2.
* Further improvements to cut binary size.
* Prepare to make MergePartialFromCodedStream non-virtual.
* A report on some interesting behavior change in python (caused by b/27494216) made me realize there is a check that needs to be done in case the parse ended on a end group tag.
* Add a note of caution to the comments around skip in CodedOutputStream.
* Simplify end check.
* Add overload for ParseMessage for MessageLite/Message types. If the explicit type is not known inlining won't help de-virtualizing the virtual call.
* Reduce linker input. It turns out that ParseMessage is not inlined, producing template instantiations that are used only once and save nothing but cost more.
* Improve the parser.
* [c++17] Changed proto2::RepeatedPtrField iterators to no longer derive from the deprecated std::iterator class.
* Change the default value of case_insensitive_enum_parsing to false for JsonStringToMessage.
* Add a warning if a field name doesn't match the style guide.
* Fix TextFormat not round-trip correctly when float value is max float.
* Added locationed info for some errors at compiler
* Python reserved keywords are now working with getattr()/setattr() for most descriptors.
* Added AllowUnknownField() in text_format
* Append '_' to C++ reserved keywords for message, enum, extension
* Fix MSVC warning C4244 in protobuf's parse_context.h.
* Updating Iterators to be compatible with C++17 in MSVC.
* Use capability annotation in mutex.h
* Fix "UndefinedBehaviorSanitizer: cfi-bad-type"
* CriticalSectionLock class as a lightweight replacement for std::mutex on Windows platforms.
* Removed vestigial wire_format_lite_inl.h
C#
* Added System.Memory dependency.
Java
* Make Java protoc code generator ignore optimize_for LITE_RUNTIME. Users should instead use the Java lite protoc plugin.
* Change Extension getMessageDefaultInstance() to return Message instead of MessageLite.
* Prevent malicious input streams from leaking buffers for ByteString or ByteBuffer parsing.
* Release new Javalite runtime.
* Show warning in case potential file name conflict.
* Allow Java reserved keywords to be used in extensions.
* Added setAllowUnknownFields() in text format
* Add memoization to ExtensionRegistryLite.getEmptyRegistry()
* Improve performance of CodedOutputStream.writeUInt32NoTag
* Add an optimized mismatch-finding algorithm to UnsafeUtil.
* When serializing uint32 varints, check that we have MAX_VARINT32_SIZE bytes left, not just MAX_VARINT_SIZE.
* Minor optimization to RopeByteString.PieceIterator
JavaScript
* Simplify generated toObject code when the default value is used.
Python
* Changes implementation of Name() for enums that allow aliases in proto2 in Python to be in line with claims in C++ implementation (to return first value).
* Added double_format option in text format printer.
* Added iter and __contains__ to extension dict
* Added allow_unknown_field option in python text format parser
* Fixed Timestamp.ToDatetime() loses precision issue
* Support unknown field in text format printer.
* Float field will be convert to inf if bigger than struct.unpack('f', b'\xff\xff\x7f\x7f')[0] which is about 3.4028234664e+38,
convert to -inf if smaller than -3.4028234664e+38
* Allowed casting str->bytes in Message.__setstate__
Ruby
* Helper methods to get enum name for Ruby.
2019-01-24 version 3.7.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
C++
* Introduced new MOMI (maybe-outside-memory-interval) parser.
* Add an option to json_util to parse enum as case-insensitive. In the future, enum parsing in json_util will become case-sensitive.
* Added conformance test for enum aliases
* Added support for --cpp_out=speed:...
* Added use of C++ override keyword where appropriate
* Many other cleanups and fixes.
Java
* Fix illegal reflective access warning in JDK 9+
* Add BOM
Python
* Added Python 3.7 compatibility.
* Modified ParseFromString to return bytes parsed .
* Introduce Proto C API.
* FindFileContainingSymbol in descriptor pool is now able to find field and enum values.
* reflection.MakeClass() and reflection.ParseMessage() are deprecated.
* Added DescriptorPool.FindMethodByName() method in pure python (c extension alreay has it)
* Flipped proto3 to preserve unknown fields by default.
* Added support for memoryview in python3 proto message parsing.
* Added MergeFrom for repeated scalar fields in c extension (pure python already has it)
* Surrogates are now rejected at setters in python3.
* Added public unknown field API.
* RecursionLimit is also set to max if allow_oversize_protos is enabled.
* Disallow duplicate scalars in proto3 text_format parse.
* Fix some segment faults for c extension map field.
PHP
* Most issues for json encoding/decoding in the c extension have been fixed. There are still some edge cases not fixed. For more details, check conformance/failure_list_php_c.txt.
* Supports php 7.3
* Added helper methods to convert between enum values and names.
* Allow setting/getting wrapper message fields using primitive values.
* Various bug fixes.
Ruby
* Ruby 2.6 support.
* Drops support for ruby < 2.3.
* Most issues for json encoding/decoding in the c extension have been fixed. There are still some edge cases not fixed. For more details, check conformance/failure_list_ruby.txt.
* Json parsing can specify an option to ignore unknown fields: msg.decode_json(data, {ignore_unknown_fields: true}).
* Added support for proto2 syntax (partially).
* Various bug fixes.
Csharp
* More support for FieldMask include merge, intersect and more.
* Increasing the default recursion limit to 100.
* Support loading FileDescriptors dynamically.
* Provide access to comments from descriptors.
* Added Any.Is method.
* Compatible with C# 6
* Added IComparable and comparison operators on Timestamp.
Objective C
* Add ability to introspect list of enum values (#4678)
* Copy the value when setting message/data fields (#5215)
* Support suppressing the objc package prefix checks on a list of files (#5309)
* More complete keyword and NSObject method (via categories) checks for field names, can result in more fields being rename, but avoids the collisions at runtime (#5289)
* Small fixes to TextFormat generation for extensions (#5362)
* Provide more details/context in deprecation messages (#5412)
* Array/Dictionary enumeration blocks NS_NOESCAPE annotation for Swift (#5421)
* Properly annotate extensions for ARC when their names imply behaviors (#5427)
* Enum alias name collision improvements (#5480)
2018-07-27 version 3.6.1 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
C++
* Introduced workaround for Windows issue with std::atomic and std::once_flag
initialization (#4777, #4773).
PHP
* Added compatibility with PHP 7.3 (#4898).
Ruby
* Fixed Ruby crash involving Any encoding (#4718).
2018-06-01 version 3.6.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
C++
* Starting from this release, we now require C++11. For those we cannot yet
upgrade to C++11, we will try to keep the 3.5.x branch updated with
critical bug fixes only. If you have any concerns about this, please
comment on issue #2780.
* Moved to C++11 types like std::atomic and std::unique_ptr and away from our
old custom-built equivalents.
* Added support for repeated message fields in lite protos using implicit
weak fields. This is an experimental feature that allows the linker to
strip out more unused messages than previously was possible.
* Fixed SourceCodeInfo for interpreted options and extension range options.
* Fixed always_print_enums_as_ints option for JSON serialization.
* Added support for ignoring unknown enum values when parsing JSON.
* Create std::string in Arena memory.
* Fixed ValidateDateTime to correctly check the day.
* Fixed bug in ZeroCopyStreamByteSink.
* Various other cleanups and fixes.
Java
* Dropped support for Java 6.
* Added a UTF-8 decoder that uses Unsafe to directly decode a byte buffer.
* Added deprecation annotations to generated code for deprecated oneof
fields.
* Fixed map field serialization in DynamicMessage.
* Cleanup and documentation for Java Lite runtime.
* Various other fixes and cleanups
* Fixed unboxed arraylists to handle an edge case
* Improved performance for copying between unboxed arraylists
* Fixed lite protobuf to avoid Java compiler warnings
* Improved test coverage for lite runtime
* Performance improvements for lite runtime
Python
* Fixed bytes/string map key incompatibility between C++ and pure-Python
implementations (issue #4029)
* Added __init__.py files to compiler and util subpackages
* Use /MT for all Windows versions
* Fixed an issue affecting the Python-C++ implementation when used with
Cython (issue #2896)
* Various text format fixes
* Various fixes to resolve behavior differences between the pure-Python and
Python-C++ implementations
PHP
* Added php_metadata_namespace to control the file path of generated metadata
file.
* Changed generated classes of nested message/enum. E.g., Foo.Bar, which
previously generates Foo_Bar, now generates Foo/Bar
* Added array constructor. When creating a message, users can pass a php
array whose content is field name to value pairs into constructor. The
created message will be initialized according to the array. Note that
message field should use a message value instead of a sub-array.
* Various bug fixes.
Objective-C
* We removed some helper class methods from GPBDictionary to shrink the size
of the library, the functionary is still there, but you may need to do some
specific +alloc / -init… methods instead.
* Minor improvements in the performance of object field getters/setters by
avoiding some memory management overhead.
* Fix a memory leak during the raising of some errors.
* Make header importing completely order independent.
* Small code improvements for things the undefined behaviors compiler option
was flagging.
Ruby
* Added ruby_package file option to control the module of generated class.
* Various bug fixes.
Javascript
* Allow setting string to int64 field.
Csharp
* Unknown fields are now parsed and then sent back on the wire. They can be
discarded at parse time via a CodedInputStream option.
* Movement towards working with .NET 3.5 and Unity
* Expression trees are no longer used
* AOT generics issues in Unity/il2cpp have a workaround (see this commit for
details)
* Floating point values are now compared bitwise (affects NaN value
comparisons)
* The default size limit when parsing is now 2GB rather than 64MB
* MessageParser now supports parsing from a slice of a byte array
* JSON list parsing now accepts null values where the underlying proto
representation does
2017-12-20 version 3.5.1 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Planned Future Changes
* Make C++ implementation C++11 only: we plan to require C++11 to build
protobuf code starting from 3.6.0 release. Please join this github issue:
https://github.com/protocolbuffers/protobuf/issues/2780 to provide your feedback.
protoc
* Fixed a bug introduced in 3.5.0 and protoc in Windows now accepts non-ascii
characters in paths again.
C++
* Removed several usages of C++11 features in the code base.
* Fixed some compiler warnings.
PHP
* Fixed memory leak in C-extension implementation.
* Added discardUnknokwnFields API.
* Removed duplicatd typedef in C-extension headers.
* Avoided calling private php methods (timelib_update_ts).
* Fixed Any.php to use fully-qualified name for DescriptorPool.
Ruby
* Added Google_Protobuf_discard_unknown for discarding unknown fields in
messages.
C#
* Unknown fields are now preserved by default.
* Floating point values are now bitwise compared, affecting message equality
check and Contains() API in map and repeated fields.
2017-11-13 version 3.5.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Planned Future Changes
* Make C++ implementation C++11 only: we plan to require C++11 to build
protobuf code starting from 3.6.0 release. Please join this github issue:
https://github.com/protocolbuffers/protobuf/issues/2780 to provide your feedback.
General
* Unknown fields are now preserved in proto3 for most of the language
implementations for proto3 by default. See the per-language section for
details.
* reserve keyword are now supported in enums
C++
* Proto3 messages are now preserving unknown fields by default. If you rely on
unknowns fields being dropped. Please use DiscardUnknownFields() explicitly.
* Deprecated the unsafe_arena_release_* and unsafe_arena_add_allocated_*
methods for string fields.
* Added move constructor and move assignment to RepeatedField,
RepeatedPtrField and google::protobuf::Any.
* Added perfect forwarding in Arena::CreateMessage
* In-progress experimental support for implicit weak fields with lite protos.
This feature allows the linker to strip out more unused messages and reduce
binary size.
* Various performance optimizations.
Java
* Proto3 messages are now preserving unknown fields by default. If you’d like
to drop unknown fields, please use the DiscardUnknownFieldsParser API. For
example:
Parser<Foo> parser = DiscardUnknownFieldsParser.wrap(Foo.parser());
Foo foo = parser.parseFrom(input);
* Added a new CodedInputStream decoder for Iterable<ByteBuffer> with direct
ByteBuffers.
* TextFormat now prints unknown length-delimited fields as messages if
possible.
* FieldMaskUtil.merge() no longer creates unnecessary empty messages when a
message field is unset in both source message and destination message.
* Various performance optimizations.
Python
* Proto3 messages are now preserving unknown fields by default. Use
message.DiscardUnknownFields() to drop unknown fields.
* Add FieldDescriptor.file in generated code.
* Add descriptor pool FindOneofByName in pure python.
* Change unknown enum values into unknown field set .
* Add more Python dict/list compatibility for Struct/ListValue.
* Add utf-8 support for text_format.Merge()/Parse().
* Support numeric unknown enum values for proto3 JSON format.
* Add warning for Unexpected end-group tag in cpp extension.
PHP
* Proto3 messages are now preserving unknown fields.
* Provide well known type messages in runtime.
* Add prefix ‘PB’ to generated class of reserved names.
* Fixed all conformance tests for encode/decode json in php runtime. C
extension needs more work.
Objective-C
* Fixed some issues around copying of messages with unknown fields and then
mutating the unknown fields in the copy.
C#
* Added unknown field support in JsonParser.
* Fixed oneof message field merge.
* Simplify parsing messages from array slices.
Ruby
* Unknown fields are now preserved by default.
* Fixed several bugs for segment fault.
Javascript
* Decoder can handle both paced and unpacked data no matter how the proto is
defined.
* Decoder now accept long varint for 32 bit integers.
2017-08-14 version 3.4.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Planned Future Changes
* There are some changes that are not included in this release but are planned
for the near future
- Preserve unknown fields in proto3: We are going to bring unknown fields
back into proto3. In this release, some languages start to support
preserving unknown fields in proto3, controlled by flags/options. Some
languages also introduce explicit APIs to drop unknown fields for
migration. Please read the change log sections by languages for details.
For general timeline and plan:
https://docs.google.com/document/d/1KMRX-G91Aa-Y2FkEaHeeviLRRNblgIahbsk4wA14gRk/view
For issues and discussions:
https://github.com/protocolbuffers/protobuf/issues/272
- Make C++ implementation C++11 only: we plan to require C++11 to build
protobuf code starting from 3.5.0 or 3.6.0 release, after unknown fields
semantic changes are finished. Please join this
github issue:
https://github.com/protocolbuffers/protobuf/issues/2780
to provide your feedback.
General
* Extension ranges now accept options and are customizable.
* "reserve" keyword now supports “max” in field number ranges,
e.g. reserve 1000 to max;
C++
* Proto3 messages are now able to preserve unknown fields. The default
behavior is still to drop unknowns, which will be flipped in a future
release. If you rely on unknowns fields being dropped. Please use
Message::DiscardUnknownFields() explicitly.
* Packable proto3 fields are now packed by default in serialization.
* Following C++11 features are introduced when C++11 is available:
- move-constructor and move-assignment are introduced to messages
- Repeated fields constructor now takes std::initializer_list
- rvalue setters are introduced for string fields
* Experimental Table-Driven parsing and serialization available to test. To
enable it, pass in table_driven_parsing table_driven_serialization protoc
generator flags for C++
$ protoc --cpp_out=table_driven_parsing,table_driven_serialization:./ \
test.proto
* lite generator parameter supported by the generator. Once set, all generated
files, use lite runtime regardless of the optimizer_for setting in the
.proto file.
* Various optimizations to make C++ code more performant on PowerPC platform
* Fixed maps data corruption when the maps are modified by both reflection API
and generated API.
* Deterministic serialization on maps reflection now uses stable sort.
* file() accessors are introduced to various *Descriptor classes to make
writing template function easier.
* ByteSize() and SpaceUsed() are deprecated.Use ByteSizeLong() and
SpaceUsedLong() instead
* Consistent hash function is used for maps in DEBUG and NDEBUG build.
* "using namespace std" is removed from stubs/common.h
* Various performance optimizations and bug fixes
Java
* Introduced new parser API DiscardUnknownFieldsParser in preparation of
proto3 unknown fields preservation change. Users who want to drop unknown
fields should migrate to use this new parser API. For example:
Parser<Foo> parser = DiscardUnknownFieldsParser.wrap(Foo.parser());
Foo foo = parser.parseFrom(input);
* Introduced new TextFormat API printUnicodeFieldValue() that prints field
value without escaping unicode characters.
* Added Durations.compare(Duration, Duration) and
Timestamps.compare(Timestamp, Timestamp).
* JsonFormat now accepts base64url encoded bytes fields.
* Optimized CodedInputStream to do less copies when parsing large bytes
fields.
* Optimized TextFormat to allocate less memory when printing.
Python
* SerializeToString API is changed to SerializeToString(self, **kwargs),
deterministic parameter is accepted for deterministic serialization.
* Added sort_keys parameter in json format to make the output deterministic.
* Added indent parameter in json format.
* Added extension support in json format.
* Added __repr__ support for repeated field in cpp implementation.
* Added file in FieldDescriptor.
* Added pretty-print filter to text format.
* Services and method descriptors are always printed even if generic_service
option is turned off.
* Note: AppEngine 2.5 is deprecated on June 2017 that AppEngine 2.5 will
never update protobuf runtime. Users who depend on AppEngine 2.5 should use
old protoc.
PHP
* Support PHP generic services. Specify file option php_generic_service=true
to enable generating service interface.
* Message, repeated and map fields setters take value instead of reference.
* Added map iterator in c extension.
* Support json encode/decode.
* Added more type info in getter/setter phpdoc
* Fixed the problem that c extension and php implementation cannot be used
together.
* Added file option php_namespace to use custom php namespace instead of
package.
* Added fluent setter.
* Added descriptor API in runtime for custom encode/decode.
* Various bug fixes.
Objective-C
* Fix for GPBExtensionRegistry copying and add tests.
* Optimize GPBDictionary.m codegen to reduce size of overall library by 46K
per architecture.
* Fix some cases of reading of 64bit map values.
* Properly error on a tag with field number zero.
* Preserve unknown fields in proto3 syntax files.
* Document the exceptions on some of the writing apis.
C#
* Implemented IReadOnlyDictionary<K,V> in MapField<K,V>
* Added TryUnpack method for Any message in addition to Unpack.
* Converted C# projects to MSBuild (csproj) format.
Ruby
* Several bug fixes.
Javascript
* Added support of field option js_type. Now one can specify the JS type of a
64-bit integer field to be string in the generated code by adding option
[jstype = JS_STRING] on the field.
2017-04-05 version 3.3.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Planned Future Changes
* There are some changes that are not included in this release but are
planned for the near future:
- Preserve unknown fields in proto3: please read this doc:
https://docs.google.com/document/d/1KMRX-G91Aa-Y2FkEaHeeviLRRNblgIahbsk4wA14gRk/view
for the timeline and follow up this github issue:
https://github.com/protocolbuffers/protobuf/issues/272
for discussion.
- Make C++ implementation C++11 only: we plan to require C++11 to build
protobuf code starting from 3.4.0 or 3.5.0 release. Please join this
github issue:
https://github.com/protocolbuffers/protobuf/issues/2780
to provide your feedback.
C++
* Fixed map fields serialization of DynamicMessage to correctly serialize
both key and value regardless of their presence.
* Parser now rejects field number 0 correctly.
* New API Message::SpaceUsedLong() that’s equivalent to
Message::SpaceUsed() but returns the value in size_t.
* JSON support
- New flag always_print_enums_as_ints in JsonPrintOptions.
- New flag preserve_proto_field_names in JsonPrintOptions. It will instruct
the JSON printer to use the original field name declared in the .proto
file instead of converting them to lowerCamelCase when printing JSON.
- JsonPrintOptions.always_print_primtive_fields now works for oneof message
fields.
- Fixed a bug that doesn’t allow different fields to set the same json_name
value.
- Fixed a performance bug that causes excessive memory copy when printing
large messages.
* Various performance optimizations.
Java
* Map field setters eagerly validate inputs and throw NullPointerExceptions
as appropriate.
* Added ByteBuffer overloads to the generated parsing methods and the Parser
interface.
* proto3 enum's getNumber() method now throws on UNRECOGNIZED values.
* Output of JsonFormat is now locale independent.
Python
* Added FindServiceByName() in the pure-Python DescriptorPool. This works only
for descriptors added with DescriptorPool.Add(). Generated descriptor_pool
does not support this yet.
* Added a descriptor_pool parameter for parsing Any in text_format.Parse().
* descriptor_pool.FindFileContainingSymbol() now is able to find nested
extensions.
* Extending empty [] to repeated field now sets parent message presence.
PHP
* Added file option php_class_prefix. The prefix will be prepended to all
generated classes defined in the file.
* When encoding, negative int32 values are sign-extended to int64.
* Repeated/Map field setter accepts a regular PHP array. Type checking is
done on the array elements.
* encode/decode are renamed to serializeToString/mergeFromString.
* Added mergeFrom, clear method on Message.
* Fixed a bug that oneof accessor didn’t return the field name that is
actually set.
* C extension now works with php7.
* This is the first GA release of PHP. We guarantee that old generated code
can always work with new runtime and new generated code.
Objective-C
* Fixed help for GPBTimestamp for dates before the epoch that contain
fractional seconds.
* Added GPBMessageDropUnknownFieldsRecursively() to remove unknowns from a
message and any sub messages.
* Addressed a threading race in extension registration/lookup.
* Increased the max message parsing depth to 100 to match the other languages.
* Removed some use of dispatch_once in favor of atomic compare/set since it
needs to be heap based.
* Fixes for new Xcode 8.3 warnings.
C#
* Fixed MapField.Values.CopyTo, which would throw an exception unnecessarily
if provided exactly the right size of array to copy to.
* Fixed enum JSON formatting when multiple names mapped to the same numeric
value.
* Added JSON formatting option to format enums as integers.
* Modified RepeatedField<T> to implement IReadOnlyList<T>.
* Introduced the start of custom option handling; it's not as pleasant as it
might be, but the information is at least present. We expect to extend code
generation to improve this in the future.
* Introduced ByteString.FromStream and ByteString.FromStreamAsync to
efficiently create a ByteString from a stream.
* Added whole-message deprecation, which decorates the class with [Obsolete].
Ruby
* Fixed Message#to_h for messages with map fields.
* Fixed memcpy() in binary gems to work for old glibc, without breaking the
build for non-glibc libc’s like musl.
Javascript
* Added compatibility tests for version 3.0.0.
* Added conformance tests.
* Fixed serialization of extensions: we need to emit a value even if it is
falsy (like the number 0).
* Use closurebuilder.py in favor of calcdeps.py for compiling JavaScript.
2017-01-23 version 3.2.0 (C++/Java/Python/PHP/Ruby/Objective-C/C#/JavaScript/Lite)
General
* Added protoc version number to protoc plugin protocol. It can be used by
protoc plugin to detect which version of protoc is used with the plugin and
mitigate known problems in certain version of protoc.
C++
* The default parsing byte size limit has been raised from 64MB to 2GB.
* Added rvalue setters for non-arena string fields.
* Enabled debug logging for Android.
* Fixed a double-free problem when using Reflection::SetAllocatedMessage()
with extension fields.
* Fixed several deterministic serialization bugs:
* MessageLite::SerializeAsString() now respects the global deterministic
serialization flag.
* Extension fields are serialized deterministically as well. Fixed protocol
compiler to correctly report importing-self as an error.
* Fixed FileDescriptor::DebugString() to print custom options correctly.
* Various performance/codesize optimizations and cleanups.
Java
* The default parsing byte size limit has been raised from 64MB to 2GB.
* Added recursion limit when parsing JSON.
* Fixed a bug that enumType.getDescriptor().getOptions() doesn't have custom
options.
* Fixed generated code to support field numbers up to 2^29-1.
Python
* You can now assign NumPy scalars/arrays (np.int32, np.int64) to protobuf
fields, and assigning other numeric types has been optimized for
performance.
* Pure-Python: message types are now garbage-collectable.
* Python/C++: a lot of internal cleanup/refactoring.
PHP (Alpha)
* For 64-bit integers type (int64/uint64/sfixed64/fixed64/sint64), use PHP
integer on 64-bit environment and PHP string on 32-bit environment.
* PHP generated code also conforms to PSR-4 now.
* Fixed ZTS build for c extension.
* Fixed c extension build on Mac.
* Fixed c extension build on 32-bit linux.
* Fixed the bug that message without namespace is not found in the descriptor
pool. (#2240)
* Fixed the bug that repeated field is not iterable in c extension.
* Message names Empty will be converted to GPBEmpty in generated code.
* Added phpdoc in generated files.
* The released API is almost stable. Unless there is large problem, we won't
change it. See
https://developers.google.com/protocol-buffers/docs/reference/php-generated
for more details.
Objective-C
* Added support for push/pop of the stream limit on CodedInputStream for
anyone doing manual parsing.
C#
* No changes.
Ruby
* Message objects now support #respond_to? for field getters/setters.
* You can now compare “message == non_message_object” and it will return false
instead of throwing an exception.
* JRuby: fixed #hashCode to properly reflect the values in the message.
Javascript
* Deserialization of repeated fields no longer has quadratic performance
behavior.
* UTF-8 encoding/decoding now properly supports high codepoints.
* Added convenience methods for some well-known types: Any, Struct, and
Timestamp. These make it easier to convert data between native JavaScript
types and the well-known protobuf types.
2016-09-23 version 3.1.0 (C++/Java/Python/PHP/Ruby/Objective-C/C#/JavaScript/Lite)
General
* Proto3 support in PHP (alpha).
* Various bug fixes.
C++
* Added MessageLite::ByteSizeLong() that’s equivalent to
MessageLite::ByteSize() but returns the value in size_t. Useful to check
whether a message is over the 2G size limit that protobuf can support.
* Moved default_instances to global variables. This allows default_instance
addresses to be known at compile time.
* Adding missing generic gcc 64-bit atomicops.
* Restore New*Callback into google::protobuf namespace since these are used
by the service stubs code
* JSON support.
* Fixed some conformance issues.
* Fixed a JSON serialization bug for bytes fields.
Java
* Fixed a bug in TextFormat that doesn’t accept empty repeated fields (i.e.,
“field: [ ]”).
* JSON support
* Fixed JsonFormat to do correct snake_case-to-camelCase conversion for
non-style-conforming field names.
* Fixed JsonFormat to parse empty Any message correctly.
* Added an option to JsonFormat.Parser to ignore unknown fields.
* Experimental API
* Added UnsafeByteOperations.unsafeWrap(byte[]) to wrap a byte array into
ByteString without copy.
Python
* JSON support
* Fixed some conformance issues.
PHP (Alpha)
* We have added the proto3 support for PHP via both a pure PHP package and a
native c extension. The pure PHP package is intended to provide usability
to wider range of PHP platforms, while the c extension is intended to
provide higher performance. Both implementations provide the same runtime
APIs and share the same generated code. Users don’t need to re-generate
code for the same proto definition when they want to switch the
implementation later. The pure PHP package is included in the php/src
directory, and the c extension is included in the php/ext directory.
Both implementations provide idiomatic PHP APIs:
* All messages and enums are defined as PHP classes.
* All message fields can only be accessed via getter/setter.
* Both repeated field elements and map elements are stored in containers
that act like a normal PHP array.
Unlike several existing third-party PHP implementations for protobuf, our
implementations are built on a "strongly-typed" philosophy: message fields
and array/map containers will throw exceptions eagerly when values of the
incorrect type (not including those that can be type converted, e.g.,
double <-> integer <-> numeric string) are inserted.
Currently, pure PHP runtime supports php5.5, 5.6 and 7 on linux. C
extension runtime supports php5.5 and 5.6 on linux.
See php/README.md for more details about installment. See
https://developers.google.com/protocol-buffers/docs/phptutorial for more
details about APIs.
Objective-C
* Helpers are now provided for working the Any well known type (see
GPBWellKnownTypes.h for the api additions).
* Some improvements in startup code (especially when extensions aren’t used).
Javascript
* Fixed missing import of jspb.Map
* Fixed valueWriterFn variable name
Ruby
* Fixed hash computation for JRuby's RubyMessage
* Make sure map parsing frames are GC-rooted.
* Added API support for well-known types.
C#
* Removed check on dependency in the C# reflection API.
2016-09-06 version 3.0.2 (C++/Java/Python/Ruby/Objective-C/C#/JavaScript/Lite)
General
* Various bug fixes.
Objective C
* Fix for oneofs in proto3 syntax files where fields were set to the zero
value.
* Fix for embedded null character in strings.
* CocoaDocs support
Ruby
* Fixed memory corruption bug in parsing that could occur under GC pressure.
Javascript
* jspb.Map is now properly exported to CommonJS modules.
C#
* Removed legacy_enum_values flag.
2016-07-27 version 3.0.0 (C++/Java/Python/Ruby/Objective-C/C#/JavaScript/Lite)
General
* This log only contains changes since the beta-4 release. Summarized change
log since the last stable release (v2.6.1) can be found in the github
release page.
Compatibility Notice
* v3.0.0 is the first API stable release of the v3.x series. We do not expect
any future API breaking changes.
* For C++, Java Lite and Objective-C, source level compatibility is
guaranteed. Upgrading from v3.0.0 to newer minor version releases will be
source compatible. For example, if your code compiles against protobuf
v3.0.0, it will continue to compile after you upgrade protobuf library to
v3.1.0.
* For other languages, both source level compatibility and binary level
compatibility are guaranteed. For example, if you have a Java binary built
against protobuf v3.0.0. After switching the protobuf runtime binary to
v3.1.0, your built binary should continue to work.
* Compatibility is only guaranteed for documented API and documented
behaviors. If you are using undocumented API (e.g., use anything in the C++
internal namespace), it can be broken by minor version releases in an
undetermined manner.
Ruby
* When you assign a string field `a.string_field = "X"`, we now call
#encode(UTF-8) on the string and freeze the copy. This saves you from
needing to ensure the string is already encoded as UTF-8. It also prevents
you from mutating the string after it has been assigned (this is how we
ensure it stays valid UTF-8).
* The generated file for `foo.proto` is now `foo_pb.rb` instead of just
`foo.rb`. This makes it easier to see which imports/requires are from
protobuf generated code, and also prevents conflicts with any `foo.rb` file
you might have written directly in Ruby. It is a backward-incompatible
change: you will need to update all of your `require` statements.
* For package names like `foo_bar`, we now translate this to the Ruby module
`FooBar`. This is more idiomatic Ruby than what we used to do (`Foo_bar`).
JavaScript
* Scalar fields like numbers and boolean now return defaults instead of
`undefined` or `null` when they are unset. You can test for presence
explicitly by calling `hasFoo()`, which we now generate for scalar fields.
Java Lite
* Java Lite is now implemented as a separate plugin, maintained in the
`javalite` branch. Both lite runtime and protoc artifacts will be available
in Maven.
C#
* Target platforms now .NET 4.5, selected portable subsets and .NET Core.
* legacy_enum_values option is no longer supported.
2016-07-15 version 3.0.0-beta-4 (C++/Java/Python/Ruby/Objective-C/C#/JavaScript)
General
* Added a deterministic serialization API for C++. The deterministic
serialization guarantees that given a binary, equal messages will be
serialized to the same bytes. This allows applications like MapReduce to
group equal messages based on the serialized bytes. The deterministic
serialization is, however, NOT canonical across languages; it is also
unstable across different builds with schema changes due to unknown fields.
Users who need canonical serialization, e.g. persistent storage in a
canonical form, fingerprinting, etc, should define their own
canonicalization specification and implement the serializer using reflection
APIs rather than relying on this API.
* Added OneofOptions. You can now define custom options for oneof groups.
import "google/protobuf/descriptor.proto";
extend google.protobuf.OneofOptions {
optional int32 my_oneof_extension = 12345;
}
message Foo {
oneof oneof_group {
(my_oneof_extension) = 54321;
...
}
}
C++ (beta)
* Introduced a deterministic serialization API in
CodedOutputStream::SetSerializationDeterministic(bool). See the notes about
deterministic serialization in the General section.
* Added google::protobuf::Map::swap() to swap two map fields.
* Fixed a memory leak when calling Reflection::ReleaseMessage() on a message
allocated on arena.
* Improved error reporting when parsing text format protos.
* JSON
- Added a new parser option to ignore unknown fields when parsing JSON.
- Added convenient methods for message to/from JSON conversion.
* Various performance optimizations.
Java (beta)
* File option "java_generate_equals_and_hash" is now deprecated. equals() and
hashCode() methods are generated by default.
* Added a new JSON printer option "omittingInsignificantWhitespace" to produce
a more compact JSON output. The printer will pretty-print by default.
* Updated Java runtime to be compatible with 2.5.0/2.6.1 generated protos.
Python (beta)
* Added support to pretty print Any messages in text format.
* Added a flag to ignore unknown fields when parsing JSON.
* Bugfix: "@type" field of a JSON Any message is now correctly put before
other fields.
Objective-C (beta)
* Updated the code to support compiling with more compiler warnings
enabled. (Issue 1616)
* Exposing more detailed errors for parsing failures. (PR 1623)
* Small (breaking) change to the naming of some methods on the support classes
for map<>. There were collisions with the system provided KVO support, so
the names were changed to avoid those issues. (PR 1699)
* Fixed for proper Swift bridging of error handling during parsing. (PR 1712)
* Complete support for generating sources that will go into a Framework and
depend on generated sources from other Frameworks. (Issue 1457)
C# (beta)
* RepeatedField optimizations.
* Support for .NET Core.
* Minor bug fixes.
* Ability to format a single value in JsonFormatter (advanced usage only).
* Modifications to attributes applied to generated code.
Javascript (alpha)
* Maps now have a real map API instead of being treated as repeated fields.
* Well-known types are now provided in the google-protobuf package, and the
code generator knows to require() them from that package.
* Bugfix: non-canonical varints are correctly decoded.
Ruby (alpha)
* Accessors for oneof fields now return default values instead of nil.
Java Lite
* Java lite support is removed from protocol compiler. It will be supported
as a protocol compiler plugin in a separate code branch.
2016-05-16 version 3.0.0-beta-3 (C++/Java/Python/Ruby/Nano/Objective-C/C#/JavaScript)
General
* Supported Proto3 lite-runtime in C++/Java for mobile platforms.
* Any type now supports APIs to specify prefixes other than
type.googleapis.com
* Removed javanano_use_deprecated_package option; Nano will always has its own
".nano" package.
C++ (Beta)
* Improved hash maps.
- Improved hash maps comments. In particular, please note that equal hash
maps will not necessarily have the same iteration order and
serialization.
- Added a new hash maps implementation that will become the default in a
later release.
* Arenas
- Several inlined methods in Arena were moved to out-of-line to improve
build performance and code size.
- Added SpaceAllocatedAndUsed() to report both space used and allocated
- Added convenient class UnsafeArenaAllocatedRepeatedPtrFieldBackInserter
* Any
- Allow custom type URL prefixes in Any packing.
- TextFormat now expand the Any type rather than printing bytes.
* Performance optimizations and various bug fixes.
Java (Beta)
* Introduced an ExperimentalApi annotation. Annotated APIs are experimental
and are subject to change in a backward incompatible way in future releases.
* Introduced zero-copy serialization as an ExperimentalApi
- Introduction of the `ByteOutput` interface. This is similar to
`OutputStream` but provides semantics for lazy writing (i.e. no
immediate copy required) of fields that are considered to be immutable.
- `ByteString` now supports writing to a `ByteOutput`, which will directly
expose the internals of the `ByteString` (i.e. `byte[]` or `ByteBuffer`)
to the `ByteOutput` without copying.
- `CodedOutputStream` now supports writing to a `ByteOutput`. `ByteString`
instances that are too large to fit in the internal buffer will be
(lazily) written to the `ByteOutput` directly.
- This allows applications using large `ByteString` fields to avoid
duplication of these fields entirely. Such an application can supply a
`ByteOutput` that chains together the chunks received from
`CodedOutputStream` before forwarding them onto the IO system.
* Other related changes to `CodedOutputStream`
- Additional use of `sun.misc.Unsafe` where possible to perform fast
access to `byte[]` and `ByteBuffer` values and avoiding unnecessary
range checking.
- `ByteBuffer`-backed `CodedOutputStream` now writes directly to the
`ByteBuffer` rather than to an intermediate array.
* Improved lite-runtime.
- Lite protos now implement deep equals/hashCode/toString
- Significantly improved the performance of Builder#mergeFrom() and
Builder#mergeDelimitedFrom()
* Various bug fixes and small feature enhancement.
- Fixed stack overflow when in hashCode() for infinite recursive oneofs.
- Fixed the lazy field parsing in lite to merge rather than overwrite.
- TextFormat now supports reporting line/column numbers on errors.
- Updated to add appropriate @Override for better compiler errors.
Python (Beta)
* Added JSON format for Any, Struct, Value and ListValue
* [ ] is now accepted for both repeated scalar fields and repeated message
fields in text format parser.
* Numerical field name is now supported in text format.
* Added DiscardUnknownFields API for python protobuf message.
Objective-C (Beta)
* Proto comments now come over as HeaderDoc comments in the generated sources
so Xcode can pick them up and display them.
* The library headers have been updated to use HeaderDoc comments so Xcode can
pick them up and display them.
* The per message and per field overhead in both generated code and runtime
object sizes was reduced.
* Generated code now include deprecated annotations when the proto file
included them.
C# (Beta)
In general: some changes are breaking, which require regenerating messages.
Most user-written code will not be impacted *except* for the renaming of enum
values.
* Allow custom type URL prefixes in `Any` packing, and ignore them when
unpacking
* `protoc` is now in a separate NuGet package (Google.Protobuf.Tools)
* New option: `internal_access` to generate internal classes
* Enum values are now PascalCased, and if there's a prefix which matches the
name of the enum, that is removed (so an enum `COLOR` with a value
`COLOR_BLUE` would generate a value of just `Blue`). An option
(`legacy_enum_values`) is temporarily available to disable this, but the
option will be removed for GA.
* `json_name` option is now honored
* If group tags are encountered when parsing, they are validated more
thoroughly (although we don't support actual groups)
* NuGet dependencies are better specified
* Breaking: `Preconditions` is renamed to `ProtoPreconditions`
* Breaking: `GeneratedCodeInfo` is renamed to `GeneratedClrTypeInfo`
* `JsonFormatter` now allows writing to a `TextWriter`
* New interface, `ICustomDiagnosticMessage` to allow more compact
representations from `ToString`
* `CodedInputStream` and `CodedOutputStream` now implement `IDisposable`,
which simply disposes of the streams they were constructed with
* Map fields no longer support null values (in line with other languages)
* Improvements in JSON formatting and parsing
Javascript (Alpha)
* Better support for "bytes" fields: bytes fields can be read as either a
base64 string or UInt8Array (in environments where TypedArray is supported).
* New support for CommonJS imports. This should make it easier to use the
JavaScript support in Node.js and tools like WebPack. See js/README.md for
more information.
* Some significant internal refactoring to simplify and modularize the code.
Ruby (Alpha)