Skip to content

Releases

v1.9.0

v1.9.0 Pre-release
Pre-release

Choose a tag to compare

@TFMV TFMV released this 03 Feb 17:41
73291c6

What's Changed

Full Changelog: v1.8.0...v1.9.0

v1.8.0

v1.8.0 Pre-release
Pre-release

Choose a tag to compare

@TFMV TFMV released this 02 Feb 15:30
8d05e9d

What's Changed

  • use snake case for descriptors by @TFMV in #19

Full Changelog: v1.7.0...v1.8.0

v1.7.0

v1.7.0 Pre-release
Pre-release

Choose a tag to compare

@TFMV TFMV released this 02 Feb 15:09
dae86d5

What's Changed

Full Changelog: v1.6.0...v1.7.0

v1.6.0

v1.6.0 Pre-release
Pre-release

Choose a tag to compare

@TFMV TFMV released this 02 Feb 14:45
d7299e5

What's Changed

Full Changelog: v1.5.0...v1.6.0

v1.5.0

v1.5.0 Pre-release
Pre-release

Choose a tag to compare

@TFMV TFMV released this 02 Feb 07:58
29c64da

What's Changed

Full Changelog: v1.4.0...v1.5.0

v1.4.0

v1.4.0 Pre-release
Pre-release

Choose a tag to compare

@TFMV TFMV released this 01 Feb 23:22
8910d4e

What's Changed

Full Changelog: v1.3.0...v1.4.0

v1.3.0

v1.3.0 Pre-release
Pre-release

Choose a tag to compare

@TFMV TFMV released this 01 Feb 23:05
4d7c51d

What's Changed

  • Additional Data Type Support by @TFMV in #12

Full Changelog: v1.2.0...v1.3.0

v1.2.0

v1.2.0 Pre-release
Pre-release

Choose a tag to compare

@TFMV TFMV released this 01 Feb 22:51
215f5d4

What's Changed

  • Add Fuzz Testing. Bug Fixes by @TFMV in #9
  • Add Go Reference Badge by @TFMV in #10
  • Add Examples Link by @TFMV in #11

Full Changelog: v1.1.0...v1.2.0

v1.1.0

v1.1.0 Pre-release
Pre-release

Choose a tag to compare

@TFMV TFMV released this 01 Feb 14:03
88d2139

This release of arrowpb represents a substantial enhancement to the existing Arrow-to-Protobuf integration code. It provides greater flexibility, improved performance, and better support for advanced use cases like well-known types (google.protobuf.Timestamp, wrapper types), dictionary-encoded columns (mapped to enums), and parallel record conversion. Below is a high-level summary of the key changes and features:

  1. Well-Known Types (WKT) Support
    Automatic Integration with google.protobuf.Timestamp and google.protobuf.*Value (wrappers) by embedding the necessary proto definitions into a global descriptor set.
    Runtime-Generated Descriptors now resolve references to these WKT files (e.g., timestamp.proto, wrappers.proto) when compiling via protodesc.NewFiles.
  2. Dictionary-Encoded Columns => Enums
    Optional handling of arrow.DictionaryType as a proto enum.
    When enabled (MapDictionariesToEnums = true in ConvertConfig), dictionary-encoded columns generate a nested enum in the containing message, allowing cleaner, more strongly typed representations in Protobuf.
  3. Struct and List Support
    Deeply-Nested Structs: arrow.StructType fields become nested proto messages, recursively.
    Repeated Fields: arrow.ListType, arrow.FixedSizeListType, and arrow.LargeListType become repeated fields in Protobuf.
    The code automatically handles multi-level nesting (lists within structs, structs within lists, etc.).
  4. Configurable Presence & Syntax
    Proto2 vs. Proto3: If UseProto2Syntax = true, the generated .proto file descriptor is marked syntax = "proto2".
    Wrapper Types (google.protobuf.*Value): Set UseWrapperTypes = true to map scalars like int64 to a wrapper message (Int64Value), thereby preserving presence bits even under proto3.
  5. Parallel Conversion & Descriptor Caching
    ConvertInParallel function allows distributing row conversion across multiple goroutines, improving throughput on large Arrow records.
    Descriptor Caching (with DescriptorCache sync.Map) avoids recompiling the same schema repeatedly.
  6. Logging and Diagnostics
    Uses a zap.Logger for detailed logging of:
    Descriptor compilation steps
    Duration of record-to-proto conversions
    Error paths and fallback strategies
  7. Example Usage & Utilities
    CreateArrowRecord: A sample function to build a simple Arrow record.
    FormatArrowJSON: Renders Arrow data as pretty-printed JSON for debugging.
    ArrowReaderToProtos: Streams over an array.RecordReader and produces a slice of Protobuf messages (one per row).

What's Changed

  • Enhancements. V0.2 by @TFMV in #5
  • Add Zap Logging by @TFMV in #6
  • resolve WKT and dictionary enum references by @TFMV in #7
  • Improve Test Coverage by @TFMV in #8

Full Changelog: v0.2.0...v1.1.0

v0.2.0

v0.2.0 Pre-release
Pre-release

Choose a tag to compare

@TFMV TFMV released this 31 Jan 16:18
c08a28e

What's Changed

  • Expose Core Functionality by @TFMV in #4

Full Changelog: v0.1...v0.2.0