Skip to content

Make (msg *Message) totalSize() public #1254

Open
@taguhoiya

Description

@taguhoiya

Suggestion

  • I would like to make totalSize() public to get to know the exact size of kakfa.Message
  • now the method of knowing the message size is the package-private one and too complicated to implement on the client side.
  • (I actually used totalSize() in my repository and had to copy and paste many methods all the way)

can we make totalSize() public?

reference

here are all the methods that I personally implemented and I copied and pasted all methods literally from the https://github.com/segmentio/kafka-go/blob/main/message.go

source code
package kafkas

import (
  "github.com/segmentio/kafka-go"
)

// This file is all implemented by https://github.com/segmentio/kafka-go/blob/main/message.go
const timestampSize = 8

func varIntLen(i int64) int {
  u := uint64((i << 1) ^ (i >> 63)) // zig-zag encoding
  n := 0

  for u >= 0x80 {
  	u >>= 7
  	n++
  }

  return n + 1
}

func varBytesLen(b []byte) int {
  return varIntLen(int64(len(b))) + len(b)
}

func varStringLen(s string) int {
  return varIntLen(int64(len(s))) + len(s)
}

func varArrayLen(n int, f func(int) int) int {
  size := varIntLen(int64(n))
  for i := 0; i < n; i++ {
  	size += f(i)
  }
  return size
}

func sizeofBytes(b []byte) int32 {
  return 4 + int32(len(b))
}

func size(msg *kafka.Message) int32 {
  return 4 + 1 + 1 + sizeofBytes(msg.Key) + sizeofBytes(msg.Value) + timestampSize
}

func headerSize(msg *kafka.Message) int {
  return varArrayLen(len(msg.Headers), func(i int) int {
  	h := &msg.Headers[i]
  	return varStringLen(h.Key) + varBytesLen(h.Value)
  })
}

// Make it public for external packages
func TotalSize(msg *kafka.Message) int32 {
  return int32(headerSize(msg)) + size(msg)
}

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @taguhoiya

      Issue actions

        Make `(msg *Message) totalSize()` public · Issue #1254 · segmentio/kafka-go