SOA Serial Number for Zones

[peteeckel]

I mentioned the issue with the SOA serial already in #30, and now I'd like to come back to this. The question is, how do we want to handle the zone serial number in SOA records.

There are several ways of managing the serial numbers, all of them equally bad :-)

One way is to leave the management of the serial number for a zone to the user. That's the most flexible and probably the worst way of handling them, as changing the serial is something that tends to be forgotten and in the case of PTR zones with auto-generated records is quite intransparent, too.

The second way is to leave the generation of the serial number to whatever uses the information in NetBox DNS to generate zone files or entries in a database of e.g. PowerDNS, or to PowerDNS itself. That's definitely an approach we should support, and actually we already do so - the SOA is exposed in the API, and so a third party tool can even update it before or after the export. The disadvantage is that that third party tool then actually needs to do that, and it might be unnecessarily difficult to determine when to increase the serial number and when not to (trust me, I've done that using Ansible and it's no fun).

The third way is to create the serial number automatically within NetBox DNS. This is easier than it sounds.

Based on NetBox standard classes, all tables, and specifically netbox_dns_zone and netbox_dns_record, maintain a last_updated column. So it is an easy task to find the latest update on a zone itself and all records within it, and the highest of all of these timestamps will be the last time anything in the zone changed. If that value increases, the zone serial needs to be updated as well.

The question what could be used for an automatically generated serial number can then be answered by looking at RFC 1982, Section 7. The largest possible serial number is 4294967295 (2^32-1), the largest increment that may be used 2147483647 (2^31-1). The current epoch timestamp is "1637482759" (well, it was when I last checked), which is well within range for a serial number. The disadvantages of using the epoch timestamp are:

1. It stops working on Sun Feb 7 06:28:15 2106 GMT.
2. DNS updates will fail if a zone does not change at least once in about 68 years.

For 1. the solution (should the issue ever arise, which I doubt) would be to trigger a serial reset by setting the serial to 0 and then starting over with the epoch modulo 2**32, buying time for the next 136 years. For 2. the question may be raised what the user needs a DNS management tool for if they don't manage DNS for decades. And anyway, it fixes itself after SOA EXPIRE seconds, which might be an acceptable time frame after 68 or more years of inactivity.

So the epoch time (rounded up and converted to integer, of course) is actually a usable SOA SERIAL value.

The next issue might be a bit more relevant, depending on how NetBox DNS will be used. Using the API or automatically generated records, it's highly probable that there are several updates of the DNS zone data within one second, so there will be several states of the zone with the same serial and different zone data during the phase these updates are being executed.

When a zone data extraction/export is run during that phase it could happen that a zone with a serial of n is exported and shipped to a staging system or the name servers itself, and a fraction of a second later there is a different dataset for the same zone with the same serial n in NetBox DNS. A subsequent extraction/export at any time after that will create the correct zone files, but with a serial number that is the same as the first export, and so DNS slaves won't get it until after SOA EXPIRE seconds.

Depending on how the extraction/export is done, there are two solutions that immediately come to my mind:

1. Accept the risk. That's probably safe when the same person does the DNS changes in NetBox DNS and then manually triggers the export.
2. Create a mechanism in the extraction/export logic that increases the serial number by 1 using the NetBox API for each export if the zone is actually newer than the one retrieved by the last extraction/export.

In any case, to cover the general case, creating SOA SERIAL numbers automatically must be optional, as other mechanisms or numbering conventions might be in use.

Any thoughts on this?

[peteeckel]

Special handling is required for the case of records being deleted with no further changes done to either the zone or the record set: In such a case the maximum last_update_time of a zone's records and the zone itself will not change, or the maximum last_update_time of the records belonging to a zone might even decrease (which woud not have any even more adverse effects, as due to the SOA SERIAL update the zone will still have the old timestamp).

So in case of record deletions, since we need an increased serial number, we will need to take caution that the zone is still detected as updated. The easiest way to achieve this without actually changing anything in the zone object is to set the zone's last_updated timestamp to the current time.