A simple null file http and https server originally written using Go 1.5.
Go has evolved considerably since 1.5 but I've tried to keep
the code as close to this version. I also haven't used any third-party
packages to minimize dependency build issues. If your platform has
Go then nullserv
will likely run on it.
Because you're running a DNS ad blocker and you want a server that returns minimal valid files for each based on file suffixes in the URL.
nullserv
reduces web page layout problems when blocking ads. It also
fixes layout and "good" ad detection for moble apps because the responses
from nullserv
are valid images.
The modern web as moved on since I started this project with almost all
websites using encrypted https traffic for ads and trackers. The good
news is nullserv
also listens on https port 443 and aborts all SSL/TLS
connections early. It claims the client's SSL certificate of authority (CA)
is invalid. This side-steps the requirement for proxies or self-generated
certificate authorities needed to run a real https server.
Pull the repo, install Google Go and run
make
. I use make
instead of go build
to dynamically
generate buildinfo.go
as well as file2gobyte
.
If you prefer to manually build nullserv
run:
$ cd /path/to/cloned/repo/nullserv
$ cc -Wall -O3 -o file2gobyte file2gobyte.c # OPTIONAL
$ sh mkbuildinfo.sh
$ go build -o nullserv *.go
The optional file2gobyte
utility emits Go's []byte{...}
array syntax
on stdout similar to xxd -i filename
for C. It's not necessary to
run nullserv
and only used to assist adding file data to files.go
.
There's also a Dockerfile
to build and run nullserv
inside a Docker
instance.
The emitted nullserv
binary is self-contained and relies on no external
files (except maybe a JSON config file). You may run it from within the
repo or copy it to somewere common like /usr/local/bin
.
It's recommended to run daemons as a non-root user whenever possible.
Unfortunately nullserv
bind to TCP ports 80 (http) and 443 (https)
by default. Most Unix operating systems forbid anyone but root to bind to
privileged ports lower than 1024. The Go language also has
problems with Setuid/Setgid
across all threads which prevents handling the issue inside nullserv
.
The simplest way to run nullserv
as non-root is to temporarily disable
privileged ports checking at the OS level, run nullserv
as a daemon,
and re-enable privileged port checking. Here's a Linux eample:
# (as user root or via sudo)
# Disable unprivileged port binding on 80 and 443 to allow nullserv
# to run as user nobody. After the daemon starts re-enable on < 1024.
# Even though this is in the net.ipv4 namespace it also allows binding
# of IPv6 ports.
sysctl -q net.ipv4.ip_unprivileged_port_start=80
# start-stop-daemon if found in any Debian, or openrc, based distro.
# Other distros have comperable dameonizing tools. Red Hat based
# distros can use a combination of 'nohup' and 'runner'.
start-stop-daemon --start --background \
--exec "/path/to/binary/nullserv" \
--stdout "$LOG/nullserv.log" --stderr "$LOG/nullserv.log" \
--user nobody
# re-enable low port checking
sysctl -q net.ipv4.ip_unprivileged_port_start=1024
Usage of ./nullserv:
-A string
https address (default '' = all)
-P int
https port (default 443)
-a string
http address (default '' = all)
-c string
JSON config file
-m int
content cache age in secs (default 31536000)
-p int
http port (default 80)
-v int
verbose level
nullserv
supports JSON configuration files as an alternative to command
line arguments. See the example_confs/
subdirectory for a few use-cases.
The contents of the file takes precedence over command line arguments and
any missing parameters use nullserv
defaults.
For example if you only wish to change the max_age
parameter to use
no-store
and never have clients cache responses:
$ cat max_age.conf
{ "max_age" : -1 }
$ nullserv -c max_age.conf
The ConfFile
struct in conf.go
lists all the valid JSON parameters
names as does the JSON file example_confs/all.json
.
nullserv
interprets certain file suffixes as requests for internal state.
Any URL ending with .ver
or .version
will return version information in
JSON:
$ curl http://127.0.0.1/.ver
{
"build_date": "Wed, 10 Jun 2020 09:44:58 -0500",
"commit_date": "Sat, 30 May 2020 17:33:56 -0500",
"reset_date": "Wed, 10 Jun 2020 12:45:34 -0500",
"sha": "56596c6",
"version": "1.3.0"
}
Any URL ending with .stat
or .stats
will return statistics:
$ curl http://127.0.0.1/.stat
{
"http_1.1": 6,
"https_tls_1.0": 9,
"stats_ok": 1,
"stats_version": 3,
"suffix_" : 1,
"suffix_stats": 3,
"suffix_version": 2
}
Keys starting with http_
and https_
track the client protocol requests
counts. In the example above we've seen 6 http 1.1 and 9 https 1.0
connections.
The stats_
keys are meta-data. The stats_ok
value returns 1
if
the .stats
structure is valid, correct data (0
otherwise). The
stats_version
is a monotonically increasing version number which will
be incremented whenever the layout of this JSON response is altered.
The suffix_*
keys show counts for each of the file suffixes encountered
since the last .reset
event. The above data has 1 request for a URL
with no file suffix, 3 for file suffix .stats
, and 2 for file
suffix .version
.
Any URL ending with .res
or .reset
will reset all internal statistics.
The response is identical to .stat
after resetting all counts. This also
updates reset_date
in the .ver
output to the current time on the host
running nullserv
.
$ curl http://127.0.0.1/.res
{
"stats_ok": 1,
"stats_version": 3
}
The idea for nullserv came from another GitHub project,
pixelserv. I wanted a real project
to learn how to use Google Go. The running binary on a 64-bit x86 host
consumes about 8MB of real memory (mostly Go libraries). It's not as
small as pixelserv is but I think it's a lot easier to extend and
maintain. Go's low-level http
library and go routines make scalability
a breeze.