Remove TimeTable in favor of Object Timestamps

[schmichael]

Background

As of version 1.5 Nomad uses a special TimeTable data structure to track approximate timestamps for Raft indexes. These timestamps are used by Nomad's state garbage collection to determine how old an object is based on its Raft index compared to the wallclock time and GC threshold for that particular object.

For example in the case of Nodes:

• The default node_gc_threshold is 24 hours.
• Node A went down at Raft index 123
• The TimeTable recorded 2009-11-10 23:00:00 UTC as the Nearest Time for Raft index 123.

Therefore at the first Node GC internal after 2009-11-11 23:00:00 UTC, Node A is eligible for garbage collection.

Since all objects record their Raft create and/or modified index, TimeTable has historically provided a generic mechanism for determining when any object is created or modified. TimeTable must make a tradeoff between space used, history kept, and precision:

	History	Precision	Entries	Size
Current	72h	5m	864	27kb
30 days / High Precision	720h	5m	8,640	270kb
30 days / Low Precision	720h	1h	720 entries	23kb

Thanks to being able to binary search (O(log N)) to find its time entry, TimeTable lookups are fairly cheap even for the 8.6k entry case.

However TimeTable is extra accounting (cpu), extra memory, extra state to snapshot. Never very much, but always overhead if there's a timestamp that can be used instead.

Proposal

Deprecate TimeTable and move all uses to object-based timestamps. The following objects and operations would need to be migrated:

• Allocation - use ModifyTime instead
• Evaluations - use ModifyTime instead
  • pruneUnblockIndexes
  • evalGC
  • expiredACLTokenGC
• Job - use SubmitTime
• Node - use StatusUpdatedAt
• Deployment - currently lacks a timestamp!
• CSIVolumes - currently lacks a timestamp!
• RootKeyMeta - use CreateTime instead

[tgross]

Related: #16359