Troubleshooting

Troubleshooting

Where the log is

Build	Log
default (syslog)	syslog, facility news: journalctl -u newscache, or the file your syslog writes news.* to
--without-syslog	LogDirectory/newscache.log (and newscache_newscacheclean.log, newscache_updatenews.log)
container image	the container's output: docker logs newscache

--enable-debug adds much more detail. newscache -o shows how a binary was built.

With --without-syslog, the log file must be a regular file with one link. A symbolic or hard link is refused with an error on stderr (a link to the program's own stderr, such as /dev/stderr, is fine). The daemon started without -n or -d points its standard input, output and error at /dev/null, so such messages are lost then; start it with -n to see them. Errors in Username or Groupname go to the log.

Control bytes in a log line (ESC, CR, DEL and the other bytes below 0x20) are written as ?: commands, user names, group names and host names come from clients and DNS. The same holds for .headerlog (--enable-headerlog), except tabs and line ends.

Checking the configuration

newscache -c /etc/newscache.conf -p

prints the configuration as NewsCache parsed it, or the line of the first error. A file that can't be opened is reported as cannot open configuration file <path>: <reason> (up to 1.3.22: unexpected EOF in <path>, whatever the reason). Upstream passwords are shown as # Password set, not shown, so the output can be pasted into a bug report.

A numeric option that isn't a plain number in its range (ClientTimeout 20m, Retries -1) and an AccessList prefix length that is empty, too large or a netmask with gaps are errors: the program doesn't start, and a running daemon keeps its old configuration at a reload.

A client with the debug right can check its own view:

XDEBUG dump authorization
XDEBUG dump configuration

To talk to NewsCache by hand:

printf 'CAPABILITIES\r\nQUIT\r\n' | nc localhost 119

Common problems

502 You have no permission to talk. Goodbye.

No AccessList entry allows the client. Nothing is allowed by default, not even localhost. Add a Client entry for its address or network before Default, and reload. Remember:

• the first matching entry wins;
• a plain address (Client 10.1.2.3) is exactly that host;
• behind Docker/Podman port forwarding, clients on the host itself arrive from the container network (172.17.0.1, 10.88.0.1);
• behind a TLS proxy, NewsCache sees the proxy's address;
• in inetd mode the client is called stdin.

See Access and Authentication.

480 Authentication required / the newsreader asks for a password

The client's entry allows the command only after a login. If logins should be optional, give the entry its own rights (allow read post); if they are wanted, check the user file. 502 access denied after a login means the login worked but doesn't grant that group or right. A failed login gets 481.

Logins always fail

• The user file: check its format (user:hash[:list:read:postto:allow]) and that the user NewsCache runs as (news) can read it.
• Local accounts (Authentication unix, PAM with pam_unix) can't be checked as news; the log says "cannot read the shadow password" or shows PAM's error with a hint. Use a user file or a PAM module that works without root. See Access and Authentication.
• The delay: each failed login waits AuthFailureDelay (at most 1 s), and after three failures the connection is closed.
• MD5 hashes on big-endian hosts (s390x, ppc64, ...) never matched up to 1.3.22; fixed after 1.3.22.

The upstream server can't be reached over TLS

The log has TLS handshake with <host> failed: ... and a TLS: line with the reason. Usual causes: the certificate has expired, its name doesn't match the Server line, or it comes from a private CA (use TLSCAFile). openssl s_client -connect host:563 -servername host shows the certificate. With starttls, a server that doesn't offer STARTTLS makes the connection fail; NewsCache never falls back to plain text. See Upstream Servers and TLS.

TLS clients can't connect

The log has TLS handshake failed: <reason> for each client that couldn't connect. Check the certificate with openssl s_client -connect host:563 -servername host: an expired certificate (NewsCache warns at start), a missing intermediate certificate (use the full chain in TLSCertFile), or a client that speaks plain NNTP to the TLS port. After a certificate renewal, SIGHUP reloads it only if Username can read the key; the log says so if it can't. See TLS for Clients.

Can't resolve the news server (in a container)

The container has no working DNS, typically because the host uses a local resolver (nameserver 127.0.0.1). Build with docker build --network host and run with --dns <resolver>. See the README.

LIST shows groups that should be gone, or misses new ones

The active list is cached for the first Timeouts value. After changing the servers' Read lists, remove SpoolDirectory/.active64 or run updatenews -a.

Upgrading from 1.3.22

These changes are on main and will be in the next release (see the Unreleased section of NEWS):

• Reply codes: a failed login answers 481 instead of 502. The greeting and MODE READER answer 201 instead of 200 when the client may not post at that moment. Upstream failures and internal faults answer 403 instead of 503. After a POSTed article only 240 or 441 come. See NNTP Commands and Access and Authentication.
• Configuration: numeric options must be plain numbers in their range, the last line counts even without a newline, and bad prefix lengths in the AccessList are errors. Check the file with newscache -p before restarting.
• NiceClean: newscacheclean now uses it (default 3) instead of NiceServer.
• New limits: ReadTimeout (300 s per read or write on an upstream connection), MaxArticleSize and MaxPostSize (1 MiB each). Raise the size limits if you need larger articles. Active-list and range replies have separate whole-reply limits; see Upstream Servers and TLS.
• Log and pid file: only a regular file with one link is written. A link there is refused (the log) or left alone with a warning (the pid file).
• LIST ACTIVE with a wildmat and LIST NEWSGROUPS show only the groups of the client's List.
• updatenews -w is accepted and ignored.
• PAM: NewsCache now gives PAM the client's address (PAM_RHOST), so host-based PAM rules (pam_access, pam_succeed_if rhost) apply. Check such rules in the newscache PAM service before upgrading.
• Logins: after a failed AUTHINFO PASS, send AUTHINFO USER again (a second PASS gets 482). A command the client may not use, where a login can't help, gets 502 (440 for POST) instead of 500.
• Failed logins wait at most 1 s, the same each time (AuthFailureDelay above 1 counts as 1; it was 15, 30, 45 s).
• Building: a plain ./configure uses /usr/local/etc instead of NONE/etc; --without-pam, --with-md5auto and --with-uniquename work as documented.
• TLS for clients is new and off unless TLSListenTo is set (see TLS for Clients). ListenTo [::1] now works on hosts without a global IPv6 address, and the listening sockets queue up to 128 connections instead of 4.
• Crossposts: when a posting goes to multiple upstream servers, NewsCache adds missing Date and Injection-Date along with a shared Message-ID before forwarding or queuing copies.
• Log lines: control bytes (below 0x20, and DEL) are written as ?.
• SIGTERM: a session ends within about a second, also while it waits for the upstream server, and doesn't reconnect after the signal.
• Big article files are written as .art<number>.tmp and renamed; newscacheclean removes a leftover .tmp by its age.
• Locks: a session no longer keeps a group locked while it writes to a client or waits for the upstream server, and a LIST ACTIVE refresh no longer blocks other sessions' GROUP. Two lock files appear in the spool, .active.refresh and .fetch.lock (no data). While another session refreshes the active list, a group that is new upstream answers 411 until the refresh is done. On a new spool, the first fetch of the active list makes other sessions wait for it, also with several upstream servers, instead of serving part of the list.
• Posting flags: a GROUP no longer turns a moderated (m) or read-only (n) group into y in the active list.
• updatenews -l reserves each range of 10000 numbers instead of locking the whole group: clients keep reading the group meanwhile.

Upgrading from 1.3.21 or older

• Active list: at the first start NewsCache converts .active to .active64 and renames the old file to .active.v1 (the log says so). The old file can then be deleted.
• Group databases: they stay valid. Old ones may be large; they never shrink, so remove a group's directory to get the space back (see Caching and Large Groups).
• Message-ID history: it is off unless you add History on.
• CachePort: it works now, but is ignored when a ListenTo line is set. Prefer ListenTo DEFAULT:port.
• ListenTo DEFAULT: it now listens on IPv6 too. Use ListenTo 0.0.0.0 for IPv4 only.
• Pid file: the systemd and container configs no longer set PidFile.

Going back to 1.3.21 works, but it drops each group's cached articles the first time it uses the group (see Caching and Large Groups).

After make install, restart the daemon: a running daemon keeps using the old binary, and reload doesn't change that.

A damaged spool file

NewsCache checks its spool files. A damaged .db or .active64 is moved to name.bad, logged once, and rebuilt from the upstream server; newscacheclean removes the .bad files. If this keeps happening, check the disk, and that only one NewsCache installation uses the spool. See doc/newscache-corrupt-db-files.txt.

A big article's file (.art<number>) that a crash cut off under an older version (its last line has no line end, or there is no empty line after the header) is fetched again from the upstream server and written anew. If that server no longer has the article, the stored copy is served as it is and the log says so. A file cut right after a line end in the body can't be told from a whole one.

The spool's disk is full

NewsCache keeps answering: articles and overview records it can't store are passed on from the upstream server, and the log says what couldn't be stored ("cannot store article ...", "cannot grow the file"). Nothing is left half written: a big article's file is written under a temporary name and renamed only when it is complete, so a failed write keeps the old file. Up to 1.3.21 a full spool crashed sessions (SIGBUS) and could leave articles cut off. Free space with newscacheclean, and keep SpoolSize well below the file system's size. The databases take the disk space of their full size (the history's 16 files about 8 MB).

The spool grows without limit

Run newscacheclean regularly (cron or a systemd timer), and check SpoolSize (in KB). With History on, the history in .history counts too; newscacheclean removes it when it alone takes more than half of SpoolSize.

Further notes in doc/

Most of these date from 2004 (Herbert Straub); details may have changed.

File	Topic
newscache-debugging.txt	Debugging with newscache -i and gdb
newscache-coredump.txt	Getting a core dump
newscache-corrupt-db-files.txt	Damaged spool databases
newscache-pam.txt	PAM
newscache-stunnel-ssl.txt	TLS for clients with stunnel (needed in 1.3.22 and older; from 1.3.23 see TLS for Clients)

Reporting a bug

Open an issue at https://github.com/go-while/NewsCache/issues with:

• the version (newscache -v) and how it was built (newscache -o);
• your system;
• the output of newscache -p (passwords are hidden, but check for other things you don't want to publish);
• the relevant log lines, and the NNTP commands that show the problem.