Internal Domain Monitoring Developer Guide

Internal Domain Monitoring - Developer Guide

How an internal domain is checked beyond "does it resolve": expected DNS records, the optional HTTP check on an HTTPS endpoint, and the REST API for endpoints, expected records and trust profiles. This is slice 3 of #203, built under #243 for 4.9.0 (changelog #2726, #2727, #2728).

For what it does from a user's point of view, see Domains - Internal domains, Internal websites and Expected records. For why it is built this way, see Internal monitoring in Domains - design. The foundations - scope, the domainMayLeaveNetwork() gate, health and its two dates, HTTPS endpoints and trust profiles - are in Domains - Developer Guide § 18 and § 18.1; this page builds on them and repeats only what it needs.

---

The shape of it

File	Job
includes/domains/scope.php	DomainRecordType, domainRecordValues() (the one normalised form), and domainRunInternalChecks() - which now asks the expected records too
includes/domains/dns.php	SOA in domainDnsSystem() and the DoH parser, normalised to the primary name server
includes/domains/endpoints.php	domainCheckEndpoint() (the HTTP check sits inside it), domainHttpStatus(), domainHttpExpect()
includes/services/domains.php	expectedReady(), expectedRecords(), saveExpectedRecord(), deleteExpectedRecord(), httpCheckReady(); saveEndpoint() takes http_path / http_expect; runInternalChecks() passes the expected records in
api/domains/endpoints.php	the screen's adapter: save_expected / delete_expected actions, and expected + expected_ready in the GET
assets/js/domains-view.js, domains/view.php	expectedHtml(), openExpected(), saveExpected(), the mExpected dialog; epPath / epExpect in the endpoint dialog
lang/en/domains.php	expected.*, check.int_expected_*, check.ep_http_*, endpoint.detail.http_*, endpoint.http_line
api/v1/resources/domain_monitoring.php	the 12 REST routes
api/v1/lib/routes.php, api/v1/lib/permissions.php	the routes, and the domains.internal key permission
api/v1/spec.json, api/v1/lib/openapi_schemas.php	the contract: 12 entries, schemas DomainEndpoint, DomainExpectedRecord, DomainTrustProfile
database/freeitsm.sql, includes/db_verify_schema.php, includes/db_verify_indexes.php	domain_expected_records and its unique key; http_path, http_expect on domain_endpoints
includes/analyst_lifecycle.php	domain_expected_records.created_by in the analyst map (history)
tests/domains-expected-records.php, tests/domains-internal-endpoints.php (part 2b), tests/api-domain-monitoring.php	the proof - see Tests

Everything is still one check run. Nothing in this slice added a new scheduler step, a new health state or a new table of results. Expected records and the HTTP check produce findings and an endpoint status, and those feed the health that slice 1 already stores and slice 2 already extends.

---

Where it sits

DomainsService::runChecks()                       (screen Check now, REST /check, the scheduler)
  └─ internal domain → runInternalChecks($conn, $row, $id, $probe)
        ├─ domainRunInternalChecks($row, self::expectedRecords($conn, $id))
        │     ├─ A / AAAA for the domain and its ssl_hosts          (slice 1)
        │     └─ one question per expected record                   (slice 3a)
        └─ each endpoint, when $probe:
              domainCheckEndpoint($ep, ...)
                 host / port / network rules → resolve → TLS        (slice 2)
                 └─ http_path set → domainHttpStatus()              (slice 3b)
              storeEndpointResult()
     → health_status, health_completed_datetime, check_results (findings)

Every new question goes through this server's resolver only and every new connection goes to the address already resolved and permitted. Neither slice 3 feature gives an internal domain a new route out of the network.

---

1. What slice 2 already guarantees

The HTTP check runs inside the endpoint check, so it inherits all of this. The full account is in § 18.1; in short, domainCheckEndpoint() returns before connecting to anything when:

1. the host is not the domain or a name under it, or is an address (re-checked at check time, not only on save);
2. the port is not in domain_internal_ports;
3. no network is permitted (domain_internal_networks is empty by default);
4. the A query failed (an empty A answer is a real answer - the Windows rule);
5. any resolved address is outside the permitted networks.

Then it connects to the address with the host name as peer_name, and judges the certificate: no answer, expired, wrong name, untrusted - in that order. A refusal is not completed (the known status stays, incomplete is set); a judged problem is a completed result. Endpoints and trust profiles are written only by somebody holding Cap::DOMAINS_INTERNAL, and Check now probes endpoints only for such a person (or the scheduler).

---

2. Expected records (3a, #2726)

The table

CREATE TABLE IF NOT EXISTS `domain_expected_records` (
    `id`               INT NOT NULL AUTO_INCREMENT,
    `domain_id`        INT NOT NULL,
    `host`             VARCHAR(253) NOT NULL,
    `record_type`      VARCHAR(5) NOT NULL,                -- DomainRecordType: A | AAAA | CNAME | NS | SOA
    `expected_values`  TEXT NULL,                          -- one per line; NULL = only that the type exists
    `created_by`       INT NULL,
    `created_datetime` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
    `updated_datetime` DATETIME NULL,
    PRIMARY KEY (`id`),
    UNIQUE KEY `uq_domain_expected_record` (`domain_id`, `host`, `record_type`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

• One row per name and type. The unique key is what makes "A for app.example.local" a single expectation with a set of values, rather than one row per address. A second row for the same pair is a 23000 that saveExpectedRecord() turns into a duplicate validation error.
• expected_values is stored already normalised, one value per line, or NULL. NULL is a meaning, not a gap: the type only has to exist. That is the setting for an address that changes.
• No result columns. Unlike domain_endpoints, the outcome of an expected record is not stored on the row. It is a finding in the domain's check_results, like every other DNS answer. The screen and the API's /findings read it from there.
• No foreign key - like the other Domains child tables, rows are tied to the domain by domain_id alone (see the trap below).

The types

enum DomainRecordType: string
{
    case A     = 'A';
    case AAAA  = 'AAAA';
    case CNAME = 'CNAME';
    case NS    = 'NS';
    case SOA   = 'SOA';
}

The enum is the whole list. The save validates with DomainRecordType::tryFrom(strtoupper(...)), the check skips a stored row whose type tryFrom() does not know, the screen's <select> offers the same five, and the OpenAPI schema enumerates them. NS and SOA are for a name that is a zone - slice 1 deliberately asks no NS question unless one is added here.

One normalised form

Every comparison - what was typed, what was stored, what DNS answered - goes through one function:

function domainRecordValues(DomainRecordType $type, array $values): array
{
    $out = [];
    foreach ($values as $v) {
        $v = strtolower(rtrim(trim((string)$v), '.'));
        if ($v === '') continue;
        if ($type === DomainRecordType::A) {
            if (filter_var($v, FILTER_VALIDATE_IP, FILTER_FLAG_IPV4) === false) continue;
        } elseif ($type === DomainRecordType::AAAA) {
            if (filter_var($v, FILTER_VALIDATE_IP, FILTER_FLAG_IPV6) === false) continue;
            $v = inet_ntop(inet_pton($v));
        } elseif (!preg_match('/^(?=.{1,253}$)([a-z0-9_]([a-z0-9_-]{0,61}[a-z0-9])?)(\.[a-z0-9_]([a-z0-9_-]{0,61}[a-z0-9])?)*$/', $v)) {
            continue;
        }
        $out[] = $v;
    }
    $out = array_values(array_unique($out));
    sort($out);
    return $out;
}

Why each step is there:

Step	Why
trim, strtolower, rtrim('.')	App.Example.Local. and app.example.local are the same name. DNS answers come back lower-case with no dot already (domainDns() normalises); typed values must match that
inet_ntop(inet_pton($v))	2001:DB8:0:0:0:0:0:1 and 2001:db8::1 are one address. Without this, IPv6 spelling alone would be a mismatch
invalid values dropped	the comparison can only ever be between valid values. A typo such as 10.0.0.999 can never be compared - and so can never make a comparison pass. The save refuses it by name before it is stored (below)
array_unique + sort	order alone is never a change - the design's rule. Two sets are equal when the sorted lists are ===

The host-name pattern (CNAME, NS, SOA) allows _ because service names such as _ldap._tcp use it.

Saving

DomainsService::saveExpectedRecord($conn, $ctx, $in) is the only writer, for the screen and the API alike. After the readiness check, loadForActor() (company scope - a domain the caller cannot reach is not found) and a refusal for a public domain, it checks the name and the values:

        $type = DomainRecordType::tryFrom(strtoupper(trim((string)($in['record_type'] ?? ''))));
        if ($type === null) throw new ServiceError('validation', 'record_type', 'Choose A, AAAA, CNAME, NS or SOA.');
        $typed = array_values(array_filter(array_map('trim', preg_split('/[\r\n,]+/', (string)($in['expected_values'] ?? ''))), 'strlen'));
        foreach ($typed as $v) {
            if (!domainRecordValues($type, [$v])) {
                throw new ServiceError('validation', 'expected_values', "\"$v\" is not a valid {$type->value} value. Enter one per line.");
            }
        }
        $values = domainRecordValues($type, $typed);
        if (in_array($type, [DomainRecordType::CNAME, DomainRecordType::SOA], true) && count($values) > 1) {
            throw new ServiceError('validation', 'expected_values', "A name has only one {$type->value}: enter one value.");
        }
        $stored = $values ? implode("\n", $values) : null;

• Each value is validated on its own (domainRecordValues($type, [$v]) returns [] for an invalid one), so the error names the bad value instead of saying "something is wrong". Running the whole list through once would silently drop it - right for the comparison, wrong for a save.
• Commas are accepted as well as new lines, so 10.0.0.5, 10.0.0.6 pasted on one line works.
• CNAME and SOA hold one value. A name cannot have two CNAMEs, and an SOA is compared by its single primary name server.
• The host goes through domainNormaliseHost() and must pass domainHostBelongsTo($host, $name) - the domain or a name under it, the same rule as endpoints.

History. Every add, change and removal writes a domain_audit row with field_name = 'expected_record', source app or api. The value is written as it reads - app.example.local A: 10.0.0.5, 10.0.0.6 - so the history is still readable a year later:

                self::audit($conn, (int)$domain['id'], $ctx->actorId ?: null, 'expected_record',
                    "{$old['host']} {$old['record_type']}" . ($old['expected_values'] ? ': ' . str_replace("\n", ', ', $old['expected_values']) : ''),
                    "$host {$type->value}" . ($stored ? ': ' . str_replace("\n", ', ', $stored) : ''), $src);

Readiness. expectedReady() probes the table once per request (a static). Before Database Verification, expectedRecords() returns [] (so a check runs exactly as slice 1), and saveExpectedRecord() / deleteExpectedRecord() throw not_ready.

The check

runInternalChecks() reads the rows and hands them in:

        $res = domainRunInternalChecks($row, self::expectedRecords($conn, $id));

domainRunInternalChecks() takes them, and an optional resolver:

function domainRunInternalChecks(array $d, array $expected = [], ?callable $resolve = null): array
{
    // TRAP: always system_only here, whatever the install-wide resolver says -
    // 'system' falls back to Google, and 'auto' IS Google on Windows. A test may
    // pass its own resolver; production never does.
    $resolve = $resolve ?? fn(string $h, string $t) => domainDns($h, $t, 'system_only');

🔴 TRAP: the $resolve parameter is for tests only. Production passes nothing and gets system_only, which never sends a name to DNS-over-HTTPS or falls back to Google. Do not thread the install-wide domain_dns_resolver setting into this call - auto is Google on Windows, and an internal name would leave the network. The only caller, runInternalChecks(), passes two arguments.

After the slice 1 address checks, one question per expected record:

    foreach ($expected as $x) {
        $host = (string)$x['host'];
        $type = DomainRecordType::tryFrom(strtoupper((string)$x['record_type']));
        if ($type === null) continue;
        $want = domainRecordValues($type, preg_split('/\r\n|\r|\n/', (string)($x['expected_values'] ?? '')));
        $r = $resolve($host, $type->value);
        $p = ['host' => $host, 'type' => $type->value];
        if (!$r['ok']) {
            $failed = true;
            $add('int_expected_not_checked', 'info', $p + ['error' => (string)$r['error']]);
            continue;
        }
        $got = domainRecordValues($type, $r['records']);
        if (!$got) {
            $problem = true;
            $add('int_expected_missing', 'fail', $p);
        } elseif ($want && $want !== $got) {
            $problem = true;
            $add('int_expected_mismatch', 'fail', $p + ['expected' => implode(', ', $want), 'found' => implode(', ', array_slice($got, 0, 8))]);
        } else {
            $add('int_expected_ok', 'pass', $p + ['found' => implode(', ', array_slice($got, 0, 8))]);
        }
    }

The same three outcomes as every other internal question:

DNS said	Finding	Effect on the run
could not be asked (ok: false)	int_expected_not_checked (info)	$failed - the run is not completed unless a problem is confirmed elsewhere
answered, no records of the type	int_expected_missing (fail)	$problem - a confirmed problem, a fresh observation
answered, values expected and the sets differ	int_expected_mismatch (fail), naming both sets	$problem
answered, no values expected, or the sets are equal	int_expected_ok (pass)	none

and the run's result is unchanged in shape:

    return ['findings' => $f, 'completed' => $problem || !$failed, 'problem' => $problem];

completed: $problem || !$failed is slice 1's rule: a confirmed problem is a fresh observation even when another question could not be asked. So a missing record moves health_completed_datetime and sets health_status = 'problem' through the existing UPDATE in runInternalChecks(), with no new code there. All findings have area = 'health' and weight = 0: an internal domain has no grade to subtract from.

The answer is normalised again with domainRecordValues() before comparing, even though domainDns() already lower-cases and sorts. That is what makes IPv6 spelling and duplicates irrelevant on the DNS side too, and drops anything in the answer that is not a valid value of the type.

Expected records are DNS only, so they run on every Check now - including one by an analyst without Cap::DOMAINS_INTERNAL, whose run passes $probe = false and re-uses the endpoints' stored results. Asking this server's resolver is not a connection to an internal address, so it needs no permission.

SOA in the DNS helper

SOA was not a type domainDns() answered before. It was added to both resolvers, normalised to the primary name server (the SOA's MNAME), because that is the one part of an SOA that says which server is authoritative and does not change on every zone edit (the serial does).

The system resolver, in domainDnsSystem():

    $map = ['A' => DNS_A, 'AAAA' => DNS_AAAA, 'MX' => DNS_MX, 'TXT' => DNS_TXT, 'NS' => DNS_NS, 'CNAME' => DNS_CNAME, 'SOA' => DNS_SOA];

            case 'SOA':   if (!empty($r['mname'])) $out[] = strtolower(rtrim($r['mname'], '.')); break;

And the DoH parser, which gets the SOA as one string (mname rname serial refresh retry expire minimum) and keeps the first field:

            case 'SOA': $out[] = strtolower(rtrim((string)(preg_split('/\s+/', trim($d))[0] ?? ''), '.')); break;   // its primary name server

An internal domain only ever asks the system resolver; the DoH branch gives the same shape, so domainDns(..., 'SOA') answers alike whichever resolver is asked.

On screen

api/domains/endpoints.php GET returns expected (the rows) and expected_ready beside the endpoints, and the save and delete are two more POST actions on the same adapter, behind the same domainRequireCap($conn, Cap::DOMAINS_INTERNAL):

        case 'save_expected':
            domainApiOk(['expected' => DomainsService::saveExpectedRecord($conn, $ctx, $in)]);
        case 'delete_expected':
            DomainsService::deleteExpectedRecord($conn, $ctx, (int)($in['id'] ?? 0));
            domainApiOk();

expectedHtml() in assets/js/domains-view.js draws the table under the endpoints on the Health tab. The result column is not stored on the row - it is found among the last check's findings by host and type:

        const result = x => {
            const f = (D.findings || []).find(y => /^int_expected_/.test(y.key) && y.params && y.params.host === x.host && y.params.type === x.record_type);
            if (!f) return '<span class="dom-pill grey">' + esc(T('health.not_checked')) + '</span>';
            const cls = { pass: 'green', fail: 'red' }[f.level] || 'grey';
            const label = { pass: T('expected.ok'), fail: T('expected.problem') }[f.level] || T('health.not_checked');
            return '<span class="dom-pill ' + cls + '">' + esc(label) + '</span>' + (f.params.found ? '<div class="dom-sub">' + esc(T('expected.found', { found: f.params.found })) + '</div>' : '');
        };

So a record added since the last check shows Not checked until the next run - never a pass. The findings also appear in the Health area below with their full advice (check.int_expected_*).

TRAP: an edited record shows the previous result until the next check. The match is by host and type, not by values, so changing a record's values leaves the old int_expected_ok or _mismatch beside the new values until a check runs. saveExpected() reloads the page; it does not run a check.

Trap: deleting a domain must delete its expected records

DomainsService::deleteDomain() removes children by hand, from a list:

            foreach (['domain_cmdb_objects', 'domain_status_services', 'ticket_domains', 'domain_knowledge_articles', 'domain_status_incidents', 'domain_manual_certificates', 'domain_endpoints', 'domain_expected_records'] as $t) {

There is no foreign key to cascade, so a child table missing from this list is left behind with no domain. domain_expected_records was missing when slice 3a shipped, and the tests hid it: both deleted the rows by hand in their finally before calling deleteDomain(). It was added in changelog #2732, and tests/domains-expected-records.php now deletes a domain and asserts its records went with it - a check that fails on the old list. A new child table goes in this list, and gets the same assertion.

---

3. The HTTP check (3b, #2727)

The columns

    `http_path`               VARCHAR(500) NULL,           -- GH #243: an optional HTTP check - the path to GET
    `http_expect`             VARCHAR(100) NULL,           -- its expected status codes, e.g. "200, 204"; empty = 200

http_path NULL means no HTTP check: the endpoint is exactly a slice 2 certificate check. http_expect NULL means 200.

Readiness. Every read and write of the two columns is guarded by DomainsService::httpCheckReady() (one probe per request). saveEndpoint() leaves them out of its INSERT / UPDATE when they are missing, and api/domains/endpoints.php leaves them out of its SELECT:

        $httpCols = DomainsService::httpCheckReady($conn) ? 'e.http_path, e.http_expect, ' : '';

So saving an endpoint keeps working on an install that has not run Database Verification since upgrading. domainCheckEndpoint() needs no guard: it reads $ep['http_path'] ?? '' from a SELECT e.* row.

Where it runs

At the end of domainCheckEndpoint(), inside the try that owns the trust profile's temporary CA file:

    // The CA file lives until the optional HTTP check below has used it too.
    try {
        $c = domainTlsCertificate($host, $port, 8, $address, $cafile);
        $cert = $c['ok'] ? array_intersect_key($c, array_flip(['subject', 'issuer', 'valid_from', 'valid_to', 'days_left', 'sans', 'hostname_match', 'chain_valid'])) : null;
        $label = $host . ':' . $port;

        if (!$c['ok']) return $out(true, 'problem', 'no_answer', ['endpoint' => $label, 'error' => (string)$c['error']], $address);
        if ($c['days_left'] !== null && $c['days_left'] < 0) return $out(true, 'problem', 'expired', ['endpoint' => $label, 'date' => (string)$c['valid_to']], $address, $cert);
        if ($c['hostname_match'] === false) return $out(true, 'problem', 'wrong_name', ['endpoint' => $label, 'host' => $host], $address, $cert);
        if ($c['chain_valid'] === false) return $out(true, 'problem', 'untrusted', ['endpoint' => $label, 'issuer' => (string)$c['issuer']], $address, $cert);

        // The optional HTTP check (GH #243, slice 3) - only once the certificate itself is
        // good, over a verified connection to the same permitted address.
        $path = trim((string)($ep['http_path'] ?? ''));
        if ($path !== '') {
            $expect = domainHttpExpect($ep['http_expect'] ?? null);
            $h = domainHttpStatus($host, $port, $address, $path, $cafile);
            $where = ['endpoint' => $label, 'path' => $path];
            if (!$h['ok']) return $out(true, 'problem', 'http_no_answer', $where + ['error' => (string)$h['error']], $address, $cert);
            if (!in_array($h['status'], $expect, true)) {
                return $out(true, 'problem', 'http_status', $where + ['status' => $h['status'], 'expected' => implode(', ', $expect)], $address, $cert);
            }
        }
    } finally {
        if ($cafile !== null) @unlink($cafile);
    }

Three things follow from where it is placed:

1. It runs only after the certificate passes. An expired, wrong-name or untrusted certificate returns first, so a certificate problem is always reported as itself, and an application is never sent a request over a connection that would not be trusted. The test proves this from the trace: an untrusted certificate with a path set makes no https:// entry at all.
2. It uses the $address already resolved and permitted by the slice 2 rules above it. There is no second DNS lookup, so the HTTP check cannot be pointed somewhere the certificate check was not.
3. The try ... finally moved. In slice 2 the temporary CA file could be deleted straight after the certificate check. Now the HTTP check needs it too, so the whole block is one try and the file goes in finally, on every return path.

Both HTTP outcomes are completed problems ($out(true, 'problem', ...)): the endpoint answered, and the answer was wrong. The certificate is still returned ($cert), so the Certificates tab and ssl_expiry_date keep working when the page is down.

An endpoint whose certificate is inside its warning window and whose page answers correctly is still expiring: the window check is after the try, so it runs only when the HTTP check passed or was not asked.

The request

function domainHttpStatus(string $host, int $port, string $address, string $path, ?string $cafile, int $timeout = 8): array
{
    if (isset($GLOBALS['DOMAIN_HTTP_TRACE']) && is_array($GLOBALS['DOMAIN_HTTP_TRACE'])) {
        $GLOBALS['DOMAIN_HTTP_TRACE'][] = "https://$host:$port$path@$address";
    }
    // The path goes into the request line, so it is checked again here, not only on
    // save - a row written some other way must not be able to add a header.
    if (!str_starts_with($path, '/') || strlen($path) > 500 || !preg_match('/^[!-~]+$/', $path)) {
        return ['ok' => false, 'status' => null, 'error' => 'The path is not valid'];
    }
    $ssl = ['verify_peer' => true, 'verify_peer_name' => true, 'peer_name' => $host, 'SNI_enabled' => true];
    // TRAP: the SAME bundle as domainTlsCertificate(), or the certificate could pass and
    // the page check then fail on an install where PHP has no CA store of its own
    // (stock WAMP - see includes/ssl.php).
    $bundle = $cafile ?? (defined('SSL_CA_BUNDLE') ? SSL_CA_BUNDLE : (function_exists('sslResolveCaBundle') ? sslResolveCaBundle() : ''));
    if ($bundle) $ssl['cafile'] = $bundle;
    $target = str_contains($address, ':') ? "[$address]" : $address;
    $s = @stream_socket_client("ssl://$target:$port", $errno, $errstr, $timeout, STREAM_CLIENT_CONNECT, stream_context_create(['ssl' => $ssl]));
    if (!$s) return ['ok' => false, 'status' => null, 'error' => $errstr !== '' ? $errstr : 'No answer on port ' . $port];
    stream_set_timeout($s, $timeout);
    $hostHeader = $port === 443 ? $host : "$host:$port";
    fwrite($s, "GET $path HTTP/1.1\r\nHost: $hostHeader\r\nUser-Agent: FreeITSM-monitor\r\nAccept: */*\r\nConnection: close\r\n\r\n");
    $line = (string)fgets($s, 1024);
    fclose($s);
    if (!preg_match('#^HTTP/\d(?:\.\d)?\s+(\d{3})\b#', $line, $m)) {
        return ['ok' => false, 'status' => null, 'error' => $line === '' ? 'No HTTP answer' : 'Not an HTTP answer'];
    }
    return ['ok' => true, 'status' => (int)$m[1], 'error' => null];
}

It is deliberately not curl. Domains' shared HTTP client, domainHttpGet() in includes/domains/lookup.php, is built for public lookups and follows redirects and resolves names itself:

        CURLOPT_FOLLOWLOCATION => true,
        CURLOPT_MAXREDIRS      => 3,

Using it here would have meant switching those off, pinning the address with CURLOPT_RESOLVE, and trusting every future change to that shared handle not to switch them back on. A raw TLS stream to an address cannot follow a redirect or look a name up at all, so the guarantees are structural, not settings:

Option or line	What it guarantees
"ssl://$target:$port" with $target the resolved address	the connection goes to the address the network rules permitted, never to whatever the name resolves to at that moment. IPv6 is bracketed
'peer_name' => $host, 'SNI_enabled' => true	the server is asked for the host's certificate (SNI), and the name check is against the host, not the address
'verify_peer' => true, 'verify_peer_name' => true	validation is on. A certificate that slipped past the first check could not carry a request
the path check at the top	the path goes into the request line, so it is re-checked here, not trusted from the row - printable ASCII, no space, so it cannot add a header
'cafile' => $bundle - the trust profile's file, else SSL_CA_BUNDLE	exactly the same CAs as the certificate check
Host: $hostHeader (with :port off 443)	a name-based virtual host answers for the right site
Connection: close, one fgets($s, 1024), fclose($s)	only the status line is read. No body, no headers parsed, no Location acted on
no credentials, cookies or second request	no login and no page scripting - the design's scope

🔴 TRAP: a redirect is the answer, never followed. A 301 or 302 is compared against http_expect like any other code. That is the design's "no redirect unless the target is also permitted" at its strictest: following one would mean judging a new target against the network rules mid-check. If a 3xx is correct for an endpoint, the administrator lists it. tests/domains-internal-endpoints.php proves the Location (an outside address) is never asked for.

TRAP: both handshakes must trust the same CAs. domainTlsCertificate() verifies against $cafile ?? SSL_CA_BUNDLE. When the HTTP check shipped it set cafile only for a trust profile, so with no profile it fell back to PHP's own openssl.cafile - which stock WAMP does not configure. A certificate the first check trusted could then fail here as http_no_answer. Both now resolve the bundle with the same expression (changelog #2732). Keep them identical if either changes. The tests still run the HTTP check only with a trust profile, so this one is held by the code, not by a test.

The trace entry https://host:port/path@address is how the tests see the request: what was asked for, and at which address.

The expected codes

function domainHttpExpect(?string $raw): array
{
    $codes = [];
    foreach (preg_split('/[\s,]+/', (string)$raw) as $c) {
        if (preg_match('/^[1-5]\d\d$/', $c)) $codes[] = (int)$c;
    }
    $codes = array_values(array_unique($codes));
    sort($codes);
    return $codes ?: [200];
}

Spaces, commas or both separate codes; the result is unique, sorted integers, and [200] when nothing usable is there. The check compares with in_array($h['status'], $expect, true) - strict, both sides int.

Saving

saveEndpoint() validates both fields and stores them tidied:

        $http = self::httpCheckReady($conn);
        $path = $http ? trim((string)($in['http_path'] ?? '')) : '';
        $expect = $http ? trim((string)($in['http_expect'] ?? '')) : '';
        if ($path !== '' && (!str_starts_with($path, '/') || strlen($path) > 500 || !preg_match('/^[\x21-\x7e]+$/', $path))) {
            throw new ServiceError('validation', 'http_path', 'The path starts with / and has no spaces, for example /health.');
        }
        foreach (preg_split('/[\s,]+/', $expect, -1, PREG_SPLIT_NO_EMPTY) as $code) {
            if (!preg_match('/^[1-5]\d\d$/', $code)) throw new ServiceError('validation', 'http_expect', "\"$code\" is not an HTTP status code. Enter codes such as 200, 204.");
        }
        $path = $path !== '' ? $path : null;
        $expect = ($path !== null && $expect !== '') ? implode(', ', domainHttpExpect($expect)) : null;

• [\x21-\x7e]+ is a safety rule, not tidiness. The path is written straight into the request line (GET $path HTTP/1.1\r\n). Printable ASCII with no space means it can never carry a \r\n and add a header or a second request.
• The path must start with /: it is a path on this endpoint, never a URL that could name another host.
• Codes are refused one by one, naming the bad one - where domainHttpExpect() alone would silently drop it.
• Codes without a path store nothing: an http_expect with no check to apply would only mislead.

A changed check is a new target. The slice 2 rule that a changed host, port or trust profile clears the endpoint's results now includes the HTTP fields:

                $moved = $old['host'] !== $host || (int)$old['port'] !== $port || (int)$old['trust_profile_id'] !== (int)$profile
                      || ($http && (($old['http_path'] ?? null) !== $path || ($old['http_expect'] ?? null) !== $expect));

Comparing against the stored, tidied $expect means retyping 200,204 as 204 200 is not a change and keeps the results.

The endpoint history row now includes the path - app.example.local:443 /health - so a history reader can see when an HTTP check was added or changed.

What comes out

detail_key	Finding	endpoint.detail.* text
http_status	ep_http_status (fail)	{path} answered {status}, not {expected}.
http_no_answer	ep_http_no_answer (fail)	{path} did not answer over HTTP ({error}).

runInternalChecks() turns every endpoint's stored result into a finding keyed 'ep_' . detail_key, so the two new codes needed only the lang entries - no change to the service loop. The endpoint table shows the configured check under the host with endpoint.http_line (Checks /health for 200, 204).

---

4. The REST API (3c, #2728)

The routes

From api/v1/lib/routes.php:

    // Internal monitoring (GH #243): changes need domains.internal AND the analyst's Internal monitoring capability.
    ['GET',    '#^/domains/(\d+)/endpoints$#',                  ['domains', 'read'],     'apiDomainEndpointsList'],
    ['POST',   '#^/domains/(\d+)/endpoints$#',                  ['domains', 'internal'], 'apiDomainEndpointsCreate'],
    ['PATCH',  '#^/domains/(\d+)/endpoints/(\d+)$#',           ['domains', 'internal'], 'apiDomainEndpointsUpdate'],
    ['DELETE', '#^/domains/(\d+)/endpoints/(\d+)$#',           ['domains', 'internal'], 'apiDomainEndpointsDelete'],
    ['GET',    '#^/domains/(\d+)/expected-records$#',           ['domains', 'read'],     'apiDomainExpectedList'],
    ['POST',   '#^/domains/(\d+)/expected-records$#',           ['domains', 'internal'], 'apiDomainExpectedCreate'],
    ['PATCH',  '#^/domains/(\d+)/expected-records/(\d+)$#',    ['domains', 'internal'], 'apiDomainExpectedUpdate'],
    ['DELETE', '#^/domains/(\d+)/expected-records/(\d+)$#',    ['domains', 'internal'], 'apiDomainExpectedDelete'],
    ['GET',    '#^/domain-trust-profiles$#',                     ['domains', 'internal'], 'apiDomainTrustProfilesList'],
    ['POST',   '#^/domain-trust-profiles$#',                     ['domains', 'internal'], 'apiDomainTrustProfilesCreate'],
    ['PATCH',  '#^/domain-trust-profiles/(\d+)$#',              ['domains', 'internal'], 'apiDomainTrustProfilesUpdate'],
    ['DELETE', '#^/domain-trust-profiles/(\d+)$#',              ['domains', 'internal'], 'apiDomainTrustProfilesDelete'],

Reading a domain's endpoints and expected records needs only domains.read - the screen's rule that everybody who can see the domain sees the results. Trust profiles need domains.internal even to list, as on screen (?profiles=1 requires the capability).

Two gates, not one

domains.internal is a new action in api/v1/lib/permissions.php. The router checks it on the key. Every handler that changes anything then checks the analyst the key acts as:

/** 403 unless the key's analyst holds the Internal monitoring capability (the screen's rule). */
function apiDomainRequireInternal(PDO $conn, array $apiKey): void {
    if (!analystHasCapability($conn, (int)$apiKey['analyst_id'], Cap::DOMAINS_INTERNAL)) {
        apiError(403, 'forbidden', 'The analyst this key acts as does not have the Internal monitoring permission.');
    }
}

This is the first resource in api/v1/resources/ to check an analyst capability. Key permissions are granted by whoever creates the key; endpoints decide which internal addresses the server connects to, so a key must never reach further than its analyst could on screen. The capability is read on every request, so an analyst who loses the Internal monitoring role loses it through every key that acts as them, at once, without anybody editing the keys.

The order in each handler matters: apiLoadDomain() first (a domain outside the key's companies is a 404, never a 403, so ids cannot be probed), then the capability, then readiness.

Trust profiles serve every company

domain_trust_profiles has no tenant_id: one list for the whole install. A key limited to some companies could otherwise change the trust of another company's endpoints, so writes also need an unlimited key:

/** Trust profiles serve every company, so only a key with no company limit may change them. */
function apiDomainRequireAllCompanies(PDO $conn, array $apiKey): void {
    if (isMultiTenant($conn) && $apiKey['company_scope'] !== null) {
        apiError(403, 'forbidden', 'Trust profiles are shared by every company; only a key that is not limited to some companies may change them.');
    }
}

On a single-company install the check is skipped - there is nobody to protect.

The PEM never leaves. apiSerializeTrustProfile() returns id, name, summary (each CA's name and expiry, worked out on save), used_by and updated_at - never ca_pem. The screen's endpoint dialog likewise gets profile names only.

PATCH is partial

Each PATCH loads the stored row, takes each field from the body only when the key is present, and runs the full save rules on the merge:

    $pick = fn(string $k) => array_key_exists($k, $body) ? $body[$k] : ($cur[$k] ?? null);

            'http_path' => $pick('http_path') ?? '', 'http_expect' => $pick('http_expect') ?? '',

array_key_exists, not isset: {"http_path": null} is present and clears the check, while leaving http_path out keeps it. Because the merge goes through saveEndpoint() with an id, a PATCH that changes the host, port, trust or HTTP check clears the endpoint's results exactly as the screen does. {"warn_days": 30} changes only the window and keeps the results.

Expected records do the same, field by field:

            'expected_values' => array_key_exists('expected_values', $body) ? apiExpectedValuesText($body['expected_values']) : (string)$cur['expected_values'],

A trust profile PATCH with no ca_pem is a rename that keeps its certificates - saveTrustProfile()'s existing rule for an empty certificate box.

Values as a list

expected_values is a JSON list in both directions, and plain text one value per line is accepted too:

/** expected_values may arrive as a list or as text, one per line. */
function apiExpectedValuesText($v): string {
    return is_array($v) ? implode("\n", array_map('strval', $v)) : (string)($v ?? '');
}

        'expected_values' => $r['expected_values'] !== null && $r['expected_values'] !== '' ? explode("\n", $r['expected_values']) : [],

An empty list means "the type only has to exist". Because the service stores values normalised, the response is the sorted, tidied list - ["10.0.0.6", "10.0.0.5"] comes back as ["10.0.0.5", "10.0.0.6"].

The same service, the same errors

Every write calls the DomainsService method the screen calls, with ActorContext::fromApiKey($apiKey), and turns a ServiceError into the API's error with apiFailFromService(). So the API cannot accept anything the screen refuses, and the history rows say source = 'api'.

HTTP	code	When
403	forbidden	the key lacks domains.internal; the key's analyst lacks Internal monitoring; a company-limited key changing a trust profile
404	not_found	the domain is missing or outside the key's companies; the endpoint or record is not this domain's; no such trust profile
422	not_ready	Database Verification has not made the tables
422	not_internal	an endpoint or expected record on a public domain
422	the field name - host, port, warn_days, trust_profile, http_path, http_expect, record_type, expected_values, name, ca_pem	a value the save rules refuse, with the screen's message
422	duplicate	the same host and port, or the same name and type, twice
422	in_use	deleting a trust profile an endpoint uses

The endpoint serialiser maps incomplete to not_checked_since (the latest attempt could not run; status is the last known result) and returns detail as the bare key (http_status, network_refused ...). The human sentence for each result is in GET /domains/{id}/findings, which already fills in titles and advice for int_expected_* and ep_* from the English lang file.

The contract

12 entries in api/v1/spec.json, under the Domains section, and three schemas in api/v1/lib/openapi_schemas.php: DomainEndpoint, DomainExpectedRecord (with record_type enumerated and expected_values an array) and DomainTrustProfile (no PEM). Both files were edited as text: spec.json mixes escaped and unescaped slashes, so re-encoding it rewrites every line, and openapi_fix.php rewrites the schemas file with var_export and deletes its comments (the Domains trap). api/v1/lib/openapi_check.php passes. The usage page is REST API: Domains.

---

5. Tests

Test	Proves	Needs
tests/domains-expected-records.php	the comparison and its three outcomes (part 1); the save rules (part 2)	nothing for part 1; FREEITSM_TEST_DATABASE for part 2
tests/domains-internal-endpoints.php part 2b	the HTTP check against a real local TLS server	nothing (it starts its own servers)
tests/api-domain-monitoring.php	the two gates, company limits, partial PATCH, 422s - over real HTTP	FREEITSM_TEST_DATABASE and FREEITSM_BASE_URL

tests/domains-expected-records.php

php tests/domains-expected-records.php
FREEITSM_TEST_DATABASE=<a throwaway database> php tests/domains-expected-records.php

Part 1 (12 checks, no database, no DNS). domainRecordValues() directly (trim, case, dot, duplicates, order; IPv6 spelling; an invalid value dropped), then domainRunInternalChecks() with a stub resolver - a map of host|type to records, or false for "could not ask". The control comes first: with no expected records the findings are exactly slice 1's, healthy and complete. Then a matching set in another order, a CNAME with no values, IPv6 written another way and an SOA by its primary name server all pass; different values are a mismatch that names what was found; an absent CNAME is missing; a resolver failure is int_expected_not_checked, neither a pass nor a problem, and the run is not completed.

Part 2 (11 checks). Through DomainsService on a throwaway internal domain and a public one: a valid record is saved normalised (App. host, a type, duplicate value) - the control; empty values store NULL; both writes are in domain_audit; then six refusals - a name outside the domain, an invalid value (the message names it), an unknown type (TXT), two CNAMEs, the same name and type twice, a public domain; and a delete. Runs only when FREEITSM_TEST_DATABASE names the database config.php points at.

tests/domains-internal-endpoints.php, part 2b

php tests/domains-internal-endpoints.php

The throwaway TLS servers from part 2 now answer a request line by path: /ok is 200, /moved is 301 with Location: https://example.org/, anything else 404. A bare certificate check sends nothing and the server just closes. Eight checks:

• /ok is healthy, and the trace is exactly the TLS handshake then https://host:port/ok@127.0.0.1 - asked once, at the resolved address, after the certificate;
• /missing is http_status with status: 404, expected: '200';
• /moved with 301, 302 expected is healthy, and the trace has one https:// entry and nothing mentioning example.org - the redirect was not followed;
• /moved with the default 200 is a problem;
• with the server's bundle the certificate is untrusted and the trace has no https:// entry - the page is never asked;
• domainHttpExpect() tidies, defaults to 200, and ignores non-codes.

Part 3 (with FREEITSM_TEST_DATABASE) adds five save checks when httpCheckReady(): a path without /, a path with a space and a code of 999 are refused; a valid check is stored with its codes tidied; codes without a path store nothing. Parts 1-2 are 38 checks; part 3 adds 22.

tests/api-domain-monitoring.php

FREEITSM_TEST_DATABASE=<db> FREEITSM_BASE_URL=http://localhost/<that install> php tests/api-domain-monitoring.php

🔴 Both variables, for the same install. It inserts API keys into the database it is pointed at and calls the install at the URL. Without FREEITSM_BASE_URL it refuses to run, rather than call another install with keys that exist only here.

17 checks through real HTTP with curl, so the router, the permission table and the response envelope are all in the path. It makes its own internal domain and keys (zz full with read + internal, zz read with read only, acting as the first active analyst, which must hold Internal monitoring or the test skips):

• Endpoints: read lists (empty); read alone cannot add (403); internal adds one with codes 204 200 stored as 200, 204; a host outside the domain is 422; PATCH {"warn_days": 30} changes the window and keeps http_path.
• Expected records: added with a list, returned sorted; an invalid value is 422; PATCH {"expected_values": []} clears the values and keeps the host; read lists them.
• Trust profiles: read cannot list them (403); internal can.
• The analyst and the companies: a key with both permissions whose analyst lacks Internal monitoring gets 403 with the capability named (skipped when every active analyst holds it); on a multi-company install, a company-limited key cannot create a trust profile (403; skipped on a single-company install).
• Removing: expected record and endpoint deleted; deleting the endpoint again is 404; at least five history rows with source = 'api'.

Its finally removes the domain's expected records and endpoints by hand, the domain, the keys and their rate-limit rows.

What is not covered: no test runs expected records through DomainsService::runChecks() end to end (part 1 calls domainRunInternalChecks() directly, part 2 only saves), and no test runs the HTTP check with the server's bundle rather than a trust profile.

---

6. Traps in one place

Trap	Why it matters
domainRunInternalChecks()'s $resolve is for tests	production must get system_only; any other resolver can send an internal name to Google
values compared through domainRecordValues() on both sides	skip it on either side and order, case, a trailing dot or IPv6 spelling becomes a "change"
invalid values are dropped by the comparison, refused by the save	so a typo can never make a check pass, and the user is told which value was wrong
expected-record results live in findings, not on the row	a new or edited record shows Not checked, or the old result, until a check runs
a new child table goes in deleteDomain()'s list	no foreign key cascades; domain_expected_records was once missing and the tests hid it
the HTTP check is a raw TLS stream, not domainHttpGet()	the shared client follows up to three redirects and resolves names itself
a 3xx is the result	following it would mean judging a new target mid-check
http_path must be printable ASCII with no space - checked on save AND before sending	it goes straight into the request line; this is what stops header injection
both handshakes resolve the CA bundle the same way	otherwise the certificate passes and the page check fails on an install with no PHP CA store
the HTTP check runs inside the CA-file try	the trust profile's temporary file must outlive both handshakes
apiDomainRequireInternal() checks the analyst, not the key	a key never reaches further than its analyst; losing the role closes every key
trust profiles have no company	only an unlimited key may change them on a multi-company install

Related pages

• Domains - the module, for users
• Domains - Developer Guide - § 18 scope and health, § 18.1 endpoints and trust profiles
• Internal monitoring in Domains - design - the review and decisions
• REST API: Domains - the routes for API users
• Developer Tests - System & infrastructure - how the tests are run