Xymon hosts.cfg format

[bonomani]

List of all info about the hosts file format

The Xymon hosts.cfg format is based on a file that exist in every computer.

Goal of this discussion:

• Collect all information about hosts file format, evolution trends, implementation...
• Compare format
• Resume

Resume

"host" file format (in general)

• Inline comments are allowed
  • Not really clear if this is working on all OS, but it seems so (Windows OK)
  • Line that start with the # are OK on all systems
• Hostname can have multiple ip addresses
  • At least 2, one per protocol (for IPv6 and V4)
  • Previous host.conf file allow a "multi" keyword, which do not restrict them to 2
  • DNSmasq use the hosts files and accept multiple ip addresses and resolved the in round robin manner
  • What If the same IP address is entered 2 times? DNSmasq? Weight Round Robin?
• Localhost is suggested to be an alias
  • This seems not to be the case in most hosts files!
• Overriding IP addresses
  • A main use capability
  • Developer use it a lot to simulate remote server on their own computer
  • Hostname was not allowed to be an IP address (this info is probably just for me), but
    • Condition "They must begin with an alphabetic character" seems relaxed in RFC1123
    • Top Level domain are not numeric, (but internal top level can be!)
    • But IP address should not be resolved, hostname should.
      • Even now it can be entered in an hostname field, a test is generaly done to see if the format match an IP. If not the resolution is done: this speed up things.
        • This would say that we can overload and ip with another one? (need test)

Security concerns

• Using the host file at an application level seems a bad idea
  • The hosts file may present an attack vector for malicious software. (wikipedia)
    • If a malicious hacker can change it, if affect not only the application that normally use it but also all the OS and the application installed on the OS, which increase the attack surface
  • In the host.conf file, some mechanism were implemented to prevent
    • Spoofing: file and DNS values are compared, if they don’t match, it is a spoofing attack and an alert can be made (syslog)
    • In switch.conf, which was the successor or host.conf, this do not exist anymore
• Some app use it to:
  • Block traffic, overriding the IP address of malicious web site with 127.0.0.1

Hosts file format (general in computer)

• RFC 952, updated by RFC 1123
• (Obsolete RFC 810)

Extract from Linux man:

From each host a single line should be present with the following information:

IP_address canonical_hostname [aliases...]

• The IP address can conform to either IPv4 or IPv6.
• Fields of the entry are separated by any number of blanks and/or tab characters.
• Text from a "#" character until the end of the line is a comment, and is ignored.
• Host names may contain only alphanumeric characters, minus signs ("-"), and periods (".").
• They must begin with an alphabetic character and end with an alphanumeric character.
• Optional aliases provide for name changes, alternate spellings, shorter hostnames, or generic hostnames (for example, - localhost).
• If required, a host may have two separate entries in this file; one for each version of the Internet Protocol (IPv4 and IPv6).

Extract for The Linux Documentation Project

The host.conf File

• https://tldp.org/LDP/nag/node82.html
• Obsolete, replace by nsswitch.conf file
• This is not the "hosts" file!
• It resides in /etc and tells the resolver which services to use, and in what order.
• Fields may be separated by white space (spaces or tabs)
• A hash sign (#) introduces a comment that extends to the next newline.

# /etc/host.conf
# We have named running, but no NIS (yet)
order   bind hosts
# Allow multiple addrs
multi   on
# Guard against spoof attempts
nospoof on
# Trim local domain (not really necessary).
trim    vbrew.com.

order: This determines the order in which the resolving services are tried. Valid options are :

• bind for querying the name server
• hosts for lookups in /etc/hosts
• nis for NIS lookups

multi: Takes on or off as options. This determines if a host in /etc/hosts is allowed to have several IP addresses, which is usually referred to as being ``multi-homed''. This flag has no effect on DNS or NIS queries.

nospoof: As explained in the previous chapter, DNS allows you to find the hostname belonging to an IP address by using the in- addr.arpa domain. Attempts by name servers to supply a false hostname are called ``spoofing''. To guard against this, the resolver may be configured to check if the original IP address is in fact associated with the hostname obtained. If not, the name is rejected and an error returned. This behavior is turned on by setting nospoof on.

alert: This option takes on or off as arguments. If it is turned on, any spoof attempts (see above) will cause the resolver to log a message to the syslog facility.

trim: This option takes a domain name as an argument, which will be removed from hostnames before lookup. This is useful for hosts entries, where you might only want to specify hostnames with- out local domain. A lookup of a host with the local domain name appended will have this removed, thus allowing the lookup in /etc/hosts to succeed. trim options accumulate, making it possible to consider your host as being local to several domains.

The nsswitch.conf File

The nsswitch.conf file allows the system administrator to configure a wide variety of different databases. We'll limit our discussion to options that relate to host and network IP address resolution. You can easily find more information about the other features by reading the GNU standard library documentation.

• Fields may be separated by whitespace (spaces or tabs)
• A hash sign (#) introduces a comment that extends to the next newline
• Each line describes a particular service; hostname resolution is one of these.

# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# Information about this file is available in the `libc6-doc' package.

hosts:          dns [!UNAVAIL=return] files
networks:       files

dns: Use the Domain Name System (DNS) service to resolve the address. This makes sense only for host address resolution, not network address resolution. This mechanism uses the /etc/resolv.conf file.

files: Search a local file for the host or network name and its corresponding address. This option uses the traditional /etc/hosts and /etc/network files.

nis or nisplus: Use the Network Information System (NIS) to resolve the host or network address. NIS and NIS+ are discussed in detail in Chapter 13.

You are able to control the lookup behavior more precisely using “action items” that describe what action to take given the result of the previous lookup attempt. Action items appear between service specifications, and are enclosed within square brackets, [ ]. The general syntax of the action statement is:
[ [!] status = action ... ]
There are two possible actions:

• return: Controls returns to the program that attempted the name resolution. If a lookup attempt was successful, the resolver will return with the details, otherwise it will return a zero result.

• continue:
  The resolver will move on to the next service in the list and attempt resolution using it.

The optional (!) character specifies that the status value should be inverted before testing; that is, it means “not.”
The available status values on which we can act are:

• success: The requested entry was found without error. The default action for this status is return.

• notfound: There was no error in the lookup, but the target host or network could not be found. The default action for this status is continue.

• unavail The service queried was unavailable. This could mean that the hosts or networks file was unreadable for the files service or that a name server or NIS server did not respond for the dns or nis services. The default action for this status is continue.

• tryagain: This status means the service is temporarily unavailable. For the files files service, this would usually indicate that the relevant file was locked by some process. For other services, it may mean the server was temporarily unable to accept connections. The default action for this status is continue.

The resolv.conf File

# /etc/resolv.conf
# Our domain
domain         vbrew.com
#
# We use vlager as central name server:
name server     172.16.1.1

When resolving the name vale, the resolver looks up vale and, failing this, vale.vbrew.com.

Debian

# The following lines are desirable for IPv4 capable hosts
127.0.0.1       localhost

# 127.0.1.1 is often used for the FQDN of the machine
127.0.1.1       thishost.mydomain.org  thishost
192.168.1.10    foo.mydomain.org       foo
192.168.1.13    bar.mydomain.org       bar
146.82.138.7    master.debian.org      master
209.237.226.90  www.opensource.org

# The following lines are desirable for IPv6 capable hosts
::1             localhost ip6-localhost ip6-loopback
ff02::1         ip6-allnodes
ff02::2         ip6-allrouters

Comments:

• 127.0.1.1 differ with other system that just use 127.0.0.1

Centos

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
1.2.3.4     thishost.mydomain.org  thishost

Comments:

• Centos (and RHEL OS Family)
• 127.0.1.1 Don’t use that
• Hostname is bound to the main ip address
  • Fine for server that use a fixed ip address
  • What about DHCP, is this really a problem

Windows

# Copyright (c) 1993-2009 Microsoft Corp.
#
# This is a sample HOSTS file used by Microsoft TCP/IP for Windows.
#
# This file contains the mappings of IP addresses to host names. Each
# entry should be kept on an individual line. The IP address should
# be placed in the first column followed by the corresponding host name.
# The IP address and the host name should be separated by at least one
# space.
#
# Additionally, comments (such as these) may be inserted on individual
# lines or following the machine name denoted by a '#' symbol.
#
# For example:
#
#      102.54.94.97     rhino.acme.com          # source server
#       38.25.63.10     x.acme.com              # x client host

# localhost name resolution is handled within DNS itself.
#	127.0.0.1       localhost
#	::1             localhost

Comments:

• The file is "empty" by default

MacOS (Probably the same on other BSD)

##
# Host Database
#
#
# localhost is used to configure the lookback interface
# when the system is booting. Do not change this entry.
##
127.0.0.1 localhost
255.255.255.255 broadcasthost
::1 localhost

Comments:

• Very Simple
• Needed

Xymon

Entry format (main format)

IP-address hostname # tag1 tag2 ...

Comments:

• Space (or whitespace?) needed "strictly" in the same format as above (not really clear in the doc, mainly after the # symbol)

• Main format compliant with host RFC

• More restrictive

  • No alias (could aliases have some sense? Is it a problem, to have aliases)?

• Default Host

  • ".default." as hostname
  • Should be listed before all hosts, first in the file, why?
  • Not compliant with host file RFC, not start and end with alphanumeric char (probably voluntarily)
  • Ip address seem 0.0.0.0
  • Do not support to put tests, but default "conn", why?
  • Negativ test example !ssh is the reverse of "ssh", but:
    • Reversing a test seems probematic as red become green and vice versa OK, what about yellow, blue, gray?
    • !ssh is naturally "not ssh", which can have more clear sense if ".default." would accept tests.
      -.default. seems an arbitrary choice. Other possibilities could be
  • Equivalency with domaine name "." (dot is the top level domain" (but should not compliant with hosts file RFC?)
  • Working with IP and Netmask or IP list, range (or all of this)
    • Netmask is not compliant with hosts file RFC (could be expanded in hosts only...)
  • Working with (sub)domain anme, hostname, wildcard, substring...
  • Working with IP both (IP and Hostname)

• Broken line

  • Long lines can be broken up by putting a backslash at the end of the line and continuing the entry on the next line.
  • Not RFC compliant (?)

• 0.0.0.0 IP address

  • Dynamic DNS resolution,
  • Use the "c-ares" fast C lib by default
  • Possible to have the default OS resolution
    • Slow
  • "Compliant with host RFC format, but not the same behavior;
    • In host file it will be resolve as 0.0.0.0 (see Special IPv4 addresses)
    • In Xymon it is by the DNS....

• Multiple IP address

  • Some tags allow multiple IP addresses

• Includes: Allow to include other files

  • filename = "path name + file name" (relative path (from /$XYMONHOME/etc/) or absolute path, file name with extension)
  • Xymon use the term of "tag" for it,
    • Another name would be less confusing as the term "tag" is used for the tests after the inline comment (see ENtry format below)
    • "keywords" ?
  • "include filename"
    • Can be nested
  • dispinclude filename
    • Only for the xymongen tool. Can be used e.g. to put a group of hosts on multiple sub-pages, without having to repeat the host definitions.
  • netinclude filename
    • Only for the xymonnet tool. (sample wiuld be helpfull)
  • directory directoryname
    • Files in folder are included in alphabetical order. support for nested sub-directories.
    • Ignored
      • If start with "."
      • If end with "~", ",v", ".rpmsave", ".rpmnew", dpkg-new", ".dpkg-orig"
      • All special files (devices, sockets, pipes etc).
  • Optional include/directory (not disp/net?). reven an error message being logged if not present

• Others (Observed Behavior)

  • Duplicate line
    • Not removed, no warning, even they are the same
  • Duplicate Hostname (with different ip)
    • Merged: The best IP is selected

• Auto match

  • A hostname (not FQDN) can match automatically a FQDN.
  • Is the domain name that is setup (OS, Level) involve/check in the process?

Comments

• Features of the hosts.cfg
  • A list of hosts and theire service to be monitored
  • A configuration file (includes)
  • A format tool (page, report,...)
• ".default."
  • Seems not accoumplished enough
• "Tag" term could be clearer
  • "Inline tags" (tests) vs "at the beginning of a line" tags (directives)
  • In the doc, all seems at the same level
• Includes:
  • Title "TAGS RECOGNIZED BY ALL TOOLS", but dispinclude and netinclude are not recognized by all tools
  • The "directory" seems already powerfull feature and a good candidate for improvement
• Scaling
  • Having all feature in one file is very convenient for small site, less for bigger
  • "Includes" can help to split into many piece. "Directory" offer a quite dynamic way of rejoining all files
    • hosts.d folder can be used (conf.d, conf.hosts.d ?)
  • For bigger site, it could be convenient to split by feature, not only by host :
    • A simple structure can be (just ideas)
      • A formating hosts.cfg file (with format directivive/tag) and also with "includes", but without any "ip hostname" test
      • "ip hostname" files without format or include directive, completely compliant with hosts file RFC
• Integration with other tools
  • API: mainly proprietary tools ....
  • Interexchange format: hosts file format is a standard, but direct integration are limited (mainly some DNS can use it directly. (OS can also, but for security concern it is not recommended)
  • Room for improvement

Devmon

Devmon use the Xymon file format

Comments

• Look for tag "DEVMON"
  • Xymon default functionanlity ".default." probably not supported
  • Broken line: same as Xymon
  • Duplicate Hostname: Detected and rejected. First is best
  • 0.0.0.0 IP address, quite the same as Xymon, resolved by the OS
  • Multiple IP addresses?

DNS File

$ORIGIN example.com.     ; designates the start of this zone file in the namespace
$TTL 3600                ; default expiration time (in seconds) of all RRs without their own TTL value
example.com.  IN  SOA   ns.example.com. username.example.com. ( 2020091025 7200 3600 1209600 3600 )
example.com.  IN  NS    ns                    ; ns.example.com is a nameserver for example.com
example.com.  IN  NS    ns.somewhere.example. ; ns.somewhere.example is a backup nameserver for example.com
example.com.  IN  MX    10 mail.example.com.  ; mail.example.com is the mailserver for example.com
@             IN  MX    20 mail2.example.com. ; equivalent to above line, "@" represents zone origin
@             IN  MX    50 mail3              ; equivalent to above line, but using a relative host name
example.com.  IN  A     192.0.2.1             ; IPv4 address for example.com
              IN  AAAA  2001:db8:10::1        ; IPv6 address for example.com
ns            IN  A     192.0.2.2             ; IPv4 address for ns.example.com
              IN  AAAA  2001:db8:10::2        ; IPv6 address for ns.example.com
www           IN  CNAME example.com.          ; www.example.com is an alias for example.com
wwwtest       IN  CNAME www                   ; wwwtest.example.com is another alias for www.example.com
mail          IN  A     192.0.2.3             ; IPv4 address for mail.example.com
mail2         IN  A     192.0.2.4             ; IPv4 address for mail2.example.com
mail3         IN  A     192.0.2.5             ; IPv4 address for mail3.example.com

Comments:
-Very similar to hosts file...

• Comment symbol is “;” and differ from hosts RFC
• Type A and AAAA of particular interest at it is the mapping Hostname -> IP
• Hostname first, IP second.
• 0.0.0.0 is supported and mean any ip address
  • but then ? it is not really resolved!
  • Better a negative answer!
  • Why 0.0.0.0 ? (and why not 255.255.255.255?) could it be used for other purpose?

POWER DNS

Power DNS can export the content of the host file into Power DNS
Functionality:

• Can auto add a domain name to a hosts if hostname do not contains a "."
  Comment as it is injected in the DNS, the DNS rule apply
• Duplicate line?
• (Not investigated, but we can expect to have a working resolution
• Duplicate hostname?
• Round robin resolution? could/should? (I don’t know, the current behavior)
• PTR Available
• Alias make PTR resolution in a round robin fashion
• "lbnamed" a load balancing dns server in Perl, no EDNS compliant was migrated into PowerDNS with coprossessing script.

DNSmasq

• DNSmasq seems to support multipe IP in hosts file
  • Round robin
  • Resolver?
  • This is not a recommanded way of doing redundancy
    • It won't work in most case as most apps/protocols will only try to connect with the first ip
    • Apps or protocols that work: ? (I think telnet....)
  • This can be use for load balancing pool of redundant

Other Info of interest

Special IPv4 addresses
https://man7.org/linux/man-pages/man7/ip.7.html

127.0.0.1       INADDR_LOOPBACK, always refers to the local host via the loopback device; 
0.0.0.0         INADDR_ANY, means any address for binding; 
255.255.255.255 INADDR_BROADCAST, means any host and has the same effect on bind as INADDR_ANY for historical reasons.

Special IPv6 addresses
https://man7.org/linux/man-pages/man7/ipv6.7.html

::              IN6ADDR_ANY, mean IPv6 0000:0000:0000:0000:0000:0000:0000:0000 than also be written as 0:0:0:0:0:0:0:0 
::0             Same as :: (dont know exaclty why this form exist, not completly compressed, a parsing problem?) 
::1             IN6ADDR_LOOPBACK

Comments:

• The ANY bind Address is interesting as it is used in Xymon.
  • IPv4, the ANY bind address is the "any address for binding" and it work as the 255.255.255.255 broadcast address
  • The broadcast is always "active" on any interface, so the interface will always accept broadcast packet (255.255.255.255 or 0.0.0.0)
  • As routers do not route broadcast packets, the interface will mainly (some exception: helper) receive broadcast packet from other computers on the same (V)LAN.
• Some use:
  • Computer can use the IP ANY (0.0.0.0 or ::) as a source IP, which indicate that they don’t have any address for now. (Like a DHCP process)
  • As a default gateway= “:./0” (similar IPv4 0.0.0.0/0)
  • The null IP v6 "::" address :is a valid ipv6 address

mDNS

• Aka multicast DNS resolution, part of the zero config initiative from Apple. Also known as "Bonjour"
• Goal: Discover service automatically
  • Each device use "announce" their service to multi cast IP address
  • Multicast IP address
• 224.0.0.251 or IPv6 address ff02::fb and UDP port 5353
  • Multicast network 224.0.0.0/24 is a special range of multicast address the "Local subnetwork"
  • Sent to all hosts in the lan, without any configuration
• No routed between LANs
  • Need a mDNS gateway to pass information between (v)lan
  • Cisco implement a mDNS gateway in modern router & L3 switch

DNS Resolution

• Xymon uses the “c-ares” C lib to speed up DNS resolution, but can also us the OS resolution
• Some app use a real DNS speed up server DNS resolution
  • DNSmasq seems to be a good candidate