Nil pointer

[Davincible]

I'm getting a nil pointer on streams.go:176. Not sure why. Maybe invalid stream/subject

[ripienaar]

What version?

please show the stack trace.

[Davincible]

I'm sorry, didn't save the full stack trace & can't find it anymore. Moved on to using the new nats-io/jetstream package.

Version was latest, I checked and it was this line;

jsm.go/streams.go

Line 176 in ef62e6a

stream.cfg = &info.Config

. Info must have been nil since stream is accessed earlier

Code I used

func (s *StreamCacheStorage[T]) listStreamMessages(subj string) ([]T, error) {
	if len(subj) == 0 {
		return []T{}, nil
	}

	stream, err := s.jsmM.LoadStream(s.streamName)
	if err != nil {
		return nil, fmt.Errorf("load stream: %w", err)
	}

	pager, err := stream.PageContents(
		jsm.PagerFilterSubject(subj),
		jsm.PagerSize(batchLimit),
	)
	if err != nil {
		return nil, fmt.Errorf("page content: %w", err)
	}

	var msgs []*nats.Msg

	hasMore := true
	for hasMore {
		msg, last, err := pager.NextMsg(context.Background())
		if !last && err != nil {
			return nil, fmt.Errorf("fetch next stream page: %w", err)
		}

		msgs = append(msgs, msg)

		hasMore = !last
	}

	return unmarshalMsgs[T](msgs)
}

Since the consumers were not being deleted, I ended up with 20k something consumers, that might have had something to do with it. Only showed up at scale in prod system. Local testing went fine

[ripienaar]

Why were consumers not deleted? Consumers are set to auto remove after a hour of idle time, are you running a particularly old server?

jsm.go/stream_pager.go

Line 169 in a21621a

InactiveThreshold(time.Hour),

[ripienaar]

Apart from that, I would not use stream pager for what you are doing. Make a consumer and just fetch the messages. This repo is mainly orientated to needs of the CLI/Terraform/GHA features, you should use nats.go instead.

[Davincible]

It was a new server. Initially used this lib as I couldn't get what I wanted with main repo, and looked at the CLI for an example. I do have it working now with the new jetstream package though, not using this code anymore.