Store and Groups

Store and Groups

A Store holds groups, articles and posted articles (store.go). fakeserver.Options{Store: st} serves it with the built-in reader (NNTP Commands). All methods are safe for concurrent use, so a test can change the store while a server under test is connected. Every returned value is a copy. NewStore() and &Store{} are the same.

Groups

type Group struct {
	Name, Description string
	Low, High, Count  int64
	Posting           bool
	Created           time.Time
}

• A new group has Low 1, High 0, Count 0.
• High is a high-water mark, as in INN: the highest number ever given out. Removing articles never lowers it, and numbers are not reused.
• Low is the lowest existing number. A group whose articles were all removed has Low = High+1 and Count 0 (RFC 3977 6.1.1.2).
• Count is the number of articles; gaps are not counted.

Method	Effect
AddGroup(name, description)	create a group (nothing happens if it exists); Posting true, Created now
AddGroupAt(name, description, created)	the same with a creation time, for NEWGROUPS
SetPosting(group, bool)	the y/n flag of LIST ACTIVE
Group(name), Groups()	look up one group, or all sorted by name

Articles

type Article struct {
	Number    int64
	MessageID string
	Headers   []string // "Name: value", without CRLF
	Body      []string // without CRLF and without dot-stuffing
	Date      time.Time
	Arrived   time.Time // when it reached the server, for NEWNEWS
}

AddArticle(group, a) files a copy of a and returns its number:

• a.Number when it is above 0 (it must be free), else High+1. Big numbers work: binary groups above 6×10^10 can be set up directly.
• A missing Message-ID is taken from the header, else generated as <number.group@fakeserver.invalid>.
• A zero Date becomes now, and so does a zero Arrived. Set Arrived to date an article for NEWNEWS; it is independent of Date and of the Date header, as INN's arrival time is. Each crossposted copy has its own.
• A group that doesn't exist is created.
• To crosspost, call AddArticle once per group.
• Test-setup mistakes panic: an empty group name, a negative or taken number, a MessageID that differs from the Message-ID header, a generated group (see below). For an overview record whose Message-ID differs from the article's, use StoreOptions.OverviewFields.

When served, an article gets Message-ID, Date and Newsgroups (every group it is filed in) headers if it lacks them. a.WithDefaults(group) returns that version.

st := fakeserver.NewStore()
st.AddGroup("alt.binaries.test", "big numbers")
n := st.AddArticle("alt.binaries.test", fakeserver.Article{
	Number:    60000000000,
	MessageID: "<big@example.invalid>",
	Headers:   []string{"From: a@example.invalid", "Subject: big"},
})
st.AddArticle("alt.binaries.test", fakeserver.Article{}) // 60000000001, Message-ID generated
st.RemoveArticle("alt.binaries.test", n)
// Group: Low 60000000001, High 60000000001, Count 1

Method	Returns
Article(group, n)	one article
ByMessageID(id)	the first article stored under id, and every group and number it is in
Numbers(group)	all existing numbers, ascending
Overview(group, from, to)	the articles in [from, to]; to ≤ 0 means up to High
Next(group, n), Last(group, n)	the next existing number above or below n
a.Header(name)	a header value: case-insensitive, unfolded, "" when absent

Expiry and cancels

Method	Effect
RemoveArticle(group, n)	remove one article; High stays, Low and Count follow, and ByMessageID forgets this location
Expire(group, below)	remove every article numbered below below; returns how many
Cancel(id)	remove the article from every group it is in; returns how many

Generated groups

Since v0.5.0 a group can be generated: its articles are made when a client asks for them and are never stored, so a group of 10^7 articles or more costs no memory. This is how a provider's big binary group looks.

st := fakeserver.NewStore()
st.AddGenerated("alt.binaries.huge", "ten million articles",
	fakeserver.Generator{Low: 1, High: 10_000_000})
s := fakeserver.New(t, fakeserver.Options{Store: st})

c := client.Dial(t, s.Addr())
c.Cmd("GROUP alt.binaries.huge") // 211 10000000 1 10000000 alt.binaries.huge
c.Cmd("XOVER 9999901-10000000")  // 224 and 100 records
over := c.ReadDotLines()
code, line := c.Cmd("XOVER 1-") // 503: the range spans more than 100000 numbers

st.SetGeneratedHigh("alt.binaries.huge", 10_000_050) // 50 new articles
st.Expire("alt.binaries.huge", 1000)                 // Low is now 1000

Generator field	Effect
Low, High	the numbers; every number between them exists unless Article says it is a hole
Article	func(group, n) (Article, bool) makes article n; false makes n a hole. nil: GeneratedArticle
Lookup	func(id) (int64, bool) maps a Message-ID to its number, for ARTICLE <id>, OVER <id> and ByMessageID. nil: IDs of the form <n.group@fakeserver.invalid>
Limit	the most numbers one OVER, XOVER or LISTGROUP may span (holes count); 0 = DefaultGenLimit (100000), below 0 = no limit

• GeneratedArticle(group, n) is the default article: From poster@generated.invalid, Subject generated article n in group, Message-ID <n.group@fakeserver.invalid>, Date 2020-01-01 plus n seconds, one body line. The same n always gives the same article. A generated article without an Arrived arrived at its Date.
• Count is High-Low+1, holes included, like the estimate of a real server.
• A request that spans more numbers than Limit gets 503 fakeserver: ... instead of a reply the test can't hold. A client that asks for the whole of a huge group shows up as an error. NEWNEWS looks at a matching generated group whole, so it gets the 503 too when the group spans more than Limit, whatever the time asked for.
• GROUP, LIST, ARTICLE by number or Message-ID, NEXT, LAST and OVER, XOVER and LISTGROUP with a range work as for stored articles.
• Expire raises Low; SetGeneratedHigh(group, high) moves High (new articles arrive, or a lower value removes the newest).
• NumbersIn(group, from, to) returns the existing numbers in a range; Numbers panics when the group spans more than the limit. GenLimit(group) returns the limit (0 for a stored group).
• AddArticle panics on a generated group; RemoveArticle and Cancel find nothing there.
• AddGenerated panics for an empty name, a group that already has stored articles, Low below 1 or High below Low-1.

A generator with holes and its own limit:

st.AddGenerated("alt.holes", "", fakeserver.Generator{
	Low: 1, High: 1_000_000,
	Article: func(group string, n int64) (fakeserver.Article, bool) {
		if n%10 == 0 {
			return fakeserver.Article{}, false // every 10th number is a hole
		}
		return fakeserver.GeneratedArticle(group, n)
	},
	Limit: 5000,
})

Posted articles

POST checks the article (Newsgroups, Subject and From required, a new Message-ID in <...>), records it with AddPosted, and files it into each group of its Newsgroups header. Posted() returns what was posted; Articles() on the server has the raw wire lines. A posted article's Arrived is the time it was posted, whatever its Date header says.

NEWNEWS

NEWNEWS wildmat date time [GMT] lists the Message-IDs of the articles in the groups the wildmat matches whose Arrived is at or after the time, as nntpget asks for them:

st := fakeserver.NewStore()
st.AddArticle("alt.test", fakeserver.Article{
	MessageID: "<old@example.invalid>",
	Arrived:   time.Date(2025, 1, 1, 0, 0, 0, 0, time.UTC),
})
st.AddArticle("alt.test", fakeserver.Article{MessageID: "<new@example.invalid>"}) // arrives now
// NEWNEWS alt.* 20260101 000000 GMT → 230, <new@example.invalid>, .

• Each Message-ID is listed once (a crossposted article where it is first found), in group name order and by number within a group.
• The time is always UTC, with or without GMT.
• Many servers turn NEWNEWS off: StoreOptions.NoNewnews makes it an unknown command (500, not in CAPABILITIES or HELP). INN answers 502 NEWNEWS command disabled by administrator instead; use Faults.Override "NEWNEWS" for that.

StoreOptions

Options.StoreOpts changes how the Store server answers.

Field	Effect
User, Pass	require AUTHINFO USER/PASS; 480 before it. A 480 written to a logged-in connection (also one from a fault) ends the login
ReaderOnly	greet with 201; POST gets 440
OverviewFmt	the LIST OVERVIEW.FMT answer. It doesn't change the OVER lines
GroupNumbers	func(conn int, g Group) Group: the numbers GROUP, LISTGROUP and LIST ACTIVE report, per connection. The articles served stay the stored ones
ListgroupRange	"reject": 501 for LISTGROUP group range; "ignore": the whole group whatever the range
NoOver, NoXover	that command is unknown (500, not in HELP); NoOver also drops OVER MSGID from CAPABILITIES
NoNewnews	NEWNEWS is unknown (500) and not in CAPABILITIES or HELP
EmptyOverReply	the reply to an OVER/XOVER range without articles, instead of 423
MaxOverRange	the largest OVER/XOVER range; an open range ends at High
OverRangeTooLarge	the reply to a larger range; when nil, only the records of its first MaxOverRange numbers
OverviewFields	func(conn int, group string, fields []string) []string changes the fields of every OVER/XOVER line before they are joined (since v0.5.0); see below
Compress	offer COMPRESS DEFLATE (RFC 8054); see Fake Server
CompressOutputOnly	offer it, but compress only the server's output and read the client's commands uncompressed: a half implementation, for client tests

st.HighestExisting is a ready-made GroupNumbers: it reports the highest existing number as High, like servers whose last number goes down when the newest articles are cancelled. Real World Server Behaviours shows how to use these options.

Overview lines

OVER and XOVER lines always have these fields, in this order (the default LIST OVERVIEW.FMT): Subject, From, Date, Message-ID, References, :bytes, :lines, Xref:full.

• Tabs, CR, LF and NUL in header values become spaces.
• :bytes is head and body with CRLF line ends plus the blank line between them; :lines the body lines.
• Xref is the stored Xref header, else Xref: fakeserver group:number ... over every group the article is in.

StoreOptions.OverviewFields gets the fields of each line (fields[0] is the article number, then the fields above; conn is the connection's ID) and returns the fields to send. It may change, drop or add fields. Use it for overview data out of step with the articles, such as a Message-ID that isn't the one in the article's header:

StoreOpts: fakeserver.StoreOptions{
	OverviewFields: func(conn int, group string, f []string) []string {
		if f[0] == "2" {
			f[4] = "<stale@example.invalid>" // the Message-ID field
		}
		return f
	},
},

The articles themselves stay as stored.