Real World Server Behaviours

Real World Server Behaviours

Real news servers don't all follow RFC 3977 to the letter. A client or cache that works against the plain Store server can still fail against them. The options below make the fake server behave like such servers. Each one has a test in realworld_test.go, and NewsCache runs its server against them in tests/e2e/upstream_test.go.

Real-world behaviour	How to get it
Expired and cancelled articles leave gaps; numbers are never reused	RemoveArticle, Expire, Cancel (the high-water mark stays)
A group whose articles all expired	remove them all: Low = High+1, or report 211 0 0 0 with GroupNumbers
The last number goes down (newest articles cancelled)	GroupNumbers: st.HighestExisting
Frontends of a server farm that lag behind each other	GroupNumbers with a lower High on some connections
Big article numbers (binary groups above 2^31, above 6×10^10)	Article.Number, or a generated group with a high Low
Very large groups (10^7 articles and more)	Store.AddGenerated (see Store and Groups)
An overview database out of step with the spool (a record's Message-ID isn't the article's)	StoreOptions.OverviewFields
Several upstream servers with their own numbers for the same group	two servers, each with its own Store (example below)
Old LISTGROUP without ranges (RFC 977/2980)	ListgroupRange: "reject" (501) or "ignore" (whole group)
Only OVER, or only XOVER	NoXover, NoOver
No NEWNEWS	NoNewnews (500), or INN's 502 with Faults.Override "NEWNEWS"
A transit peer without streaming (innfeed and innxmit fall back to IHAVE)	PeerOptions.NoStream (see Fake Server)
A peer that refuses articles after reading them (a content filter)	PeerOptions.Reject (TAKETHIS 439, IHAVE 437)
A peer that is busy for a moment	PeerOptions.Defer431Once (CHECK 431), Defer436Once (IHAVE 436)
420, or an empty 224, for a range without articles	EmptyOverReply
OVER replies capped at N records, whatever the range	MaxOverRange
An error for too large a range	MaxOverRange with OverRangeTooLarge
A connection limit	Options.MaxConns (502 Too many connections, or MaxConnsGreeting such as 400 Too many users)
A login that times out mid-session	StoreOpts.User/Pass with an Override 480 and OverrideFrom/OverrideTimes
A slow server	Delay, LineDelay, Drip
Connections cut in the middle of a reply	DropInMultiline, CloseAfter
One failing article in a pipeline	OverrideLine ("ARTICLE 2")
Overlong lines, replies that never end	LongLine, Endless
Another LIST OVERVIEW.FMT	OverviewFmt (the OVER lines keep the default order; OverviewFields changes them)

Example

A server farm whose even-numbered connections lag five articles behind, with old LISTGROUP, no OVER, 420 for empty ranges, at most 10 records per XOVER and two connections:

st := fakeserver.NewStore()
for i := 0; i < 20; i++ {
	st.AddArticle("alt.test", fakeserver.Article{})
}
lag := func(conn int, g fakeserver.Group) fakeserver.Group {
	if conn%2 == 0 { // every second connection lags 5 articles behind
		g.High -= 5
		g.Count -= 5
	}
	return g
}
s := fakeserver.New(t, fakeserver.Options{
	Store: st,
	StoreOpts: fakeserver.StoreOptions{
		GroupNumbers:   lag,
		ListgroupRange: "reject",
		NoOver:         true,
		EmptyOverReply: []string{"420 No article(s) selected"},
		MaxOverRange:   10,
	},
	MaxConns: 2,
})
// connection 1: "211 20 1 20 alt.test", connection 2: "211 15 1 15 alt.test",
// "LISTGROUP alt.test 1-5": 501, "XOVER 1-20": 10 records, "XOVER 30-40": 420,
// a third connection: "502 Too many connections"

The last number going down:

s := fakeserver.New(t, fakeserver.Options{
	Store:     st,
	StoreOpts: fakeserver.StoreOptions{GroupNumbers: st.HighestExisting},
})
st.Cancel("<5.alt.test@fakeserver.invalid>") // GROUP now says "211 4 1 4 alt.test"

Several upstream servers that carry the same group with their own numbers:

a := fakeserver.NewStore()
b := fakeserver.NewStore()
for i := int64(1); i <= 4; i++ {
	id := fmt.Sprintf("<%d@example.invalid>", i)
	a.AddArticle("test.beta", fakeserver.Article{Number: i, MessageID: id})
	b.AddArticle("test.beta", fakeserver.Article{Number: 100 + i, MessageID: id})
}
srvA := fakeserver.New(t, fakeserver.Options{Store: a}) // "211 4 1 4 test.beta"
srvB := fakeserver.New(t, fakeserver.Options{Store: b}) // "211 4 101 104 test.beta"

NewsCache's upstreams_test.go uses generated groups, OverviewFields and two servers like this.

Failures of the server under test

A full disk, or a process without file descriptors left, happens on the side of the server under test, not in the fake server. proc.Options.Ulimit sets such limits for a server binary (see Running Server Binaries).

Not covered yet (v0.5.0)

• OVER lines in another field order without an OverviewFields function: OverviewFmt only changes the LIST OVERVIEW.FMT answer.
• STARTTLS, COMPRESS, XHDR, HDR, IHAVE in the Store server (IHAVE is answered by PeerHandler).

When a test needs a behaviour that isn't here, add it to the kit (see Development).