There is a way to work around to see if it fits your need. Here is how we did it to read from any offset:

1. Get the first and last offsets on a topic/partition:

    ...
    conn, err = dialer.DialLeader(ctx, "tcp", broker, k.Topic, k.Partition) // we use kafka.DefaultDialer
    if err != nil {
         // should retry, or another broker
    }
    first, last, err = conn.ReadOffsets()
   ...

2. use the above offsets to set your desired starting offset k.Offset between first and last (inclusive):

        k.client = kafka.NewReader(config)
	_ = k.client.SetOffset(k.Offset)
        ...
        m, err := k.client.ReadMessage(k.context)

Yeah this seems reasonable and was probably a mistake in the initial implementation, can you submit a PR with the change?

I'll try to find an implementation to avoid breaking changes in the initial behavior.