-
Notifications
You must be signed in to change notification settings - Fork 13
/
ntp.txt
239 lines (181 loc) · 10.7 KB
/
ntp.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
File: /usr2/fs/misc/ntp.txt Version: 0.6 Date: 230308
NTP recommendations
This document contains recommendations for set-up and use of NTP for
robust FS operations. If you have reliable NTP time available, this
is appropriate, and recommended. Please see section (2) for an
appropriate NTP configuration for your FS computer. Please note that
section (7) is critical.
Please also see '/usr2/fs/misc/fstime.txt' for more explanation about
time in the FS. That document is a little out of date because it was
written at a time when the best model was 'rate'. It still provides
useful details and an explanation about time in the FS. The use
'rate' is still recommended for stations that do not have reliable NTP
time available. The current document ('ntp.txt'), provides all the
information needed to set-up your system for NTP.
1. Use the 'computer' model in 'time.ctl':
* rate (secs/day) span (hours) model (none/offset/rate/computer)
* -use 'computer' if you have reliable
* NTP available; if not, use 'rate'
* -to be safe, reboot if changing the model
0.000 1.000 computer
If you use the 'computer' model, the FS time becomes the computer
time.
2. In 'ntp.conf' (IPv4 assumed):
2a. Use a mix of local and remote servers, as many as you can, up
to 10. These are preferably stratum 1 servers, but
_independent_ stratum 2s can be used to fill-in. You should
try for as many stratum 1s as you can. NTP strongly prefers to
have at least three servers and you should try to have at least
three local ones if you can in case of Internet outages. You
can get by with less than three local servers. Including
remote serves is strongly recommended.
2b. In the 'ntp.conf' file, list servers by their aliases if you
are able to follow the advice of item (6d) below. This can
make the 'ntp.conf' file easier to read and simplify
maintenance.
If you are not able to follow the advice of item (6d), only
list servers by their IP addresses. This makes operation more
robust if there is a DNS outage. You can include a comment
line to help identify each server in the file.
2c. Use 'iburst minpoll 4' for each server to speed-up the initial
sync. A report of an 'ntpd' bug:
https://blog.ntpsec.org/2017/09/19/jitter-bug-perfume.html
may help explain some of NTP's strange start-up behavior. One
thing that has been noticed is that it prefers local servers at
start-up, but will eventually sync to a better remote server.
2d. Use 'restrict' with 'notrap nomodify nopeer noquery' with
each server for security.
The overall result is typically something like this for each
NTP server:
server myntpclock iburst minpoll 4
restrict myntpclock notrap nomodify nopeer noquery
2e. Use 'noselect' after 'iburst minpoll 4' for monitoring
(only) of local devices.
2f. Serve other devices on the local network, with something like:
# serve this private subnet
restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap
2g. Block access from all other nodes (especially remote) with:
restrict -4 default ignore
restrict -6 default ignore
3. Other devices at the station that support NTP
For other devices that use NTP, point them to the FS as the only
NTP server. This will make sure that the other devices do not
somehow get on a different time than the FS.
If you have a backup FS computer point the other devices to both FS
computers. Use exactly the same NTP set-up on both FS computers.
This will make the station more robust if the operational computer
fails. Be sure each FS computer is monitoring the other as
'noselect' (see 2e.). This will make it possible to monitor the
agreement between the two computers with 'ntpq -c pe'. If the
other devices use only the FS computers as NTP servers and there is
no discrepancy between the FS computers, all the other local
devices should agree.
You can make a script similar to 'check_ntp' in (6a.) below to
monitor and log the time of other devices (and possibly a backup FS
computer). You will probably want to filter the 'ntpq' output on
the nodes that you want displayed.
4. Use 'sy=run setcl &' in 'midob' to monitor the difference between
the FS and the formatter. This will make it possible to detect if
the formatter is different from the FS time including whether an
integer second jump occurs.
5. 'ntpd' vs 'ntpdate'
In the old days, if the time from the hardware clock was very far
off, NTP would not adjust the time. 'ntpdate' was used to get close
first. More recently the 'ntpd' daemon seems to handle this if
started with '-g' option. That is probably the standard
configuration for most systems now, but it might be worth checking
if your system is set-up this way. A big advantage of '-g' over
early versions of 'ntpdate' is that it will use the servers already
configured in 'ntp.conf'. Some later versions of 'ntpdate' may
also do that.
You check the options that 'ntpd' was started with using the command:
ps aux | grep ntpd
6. 'check_ntp' SNAP procedure
6a. Have the 'check_ntp' procedure in the 'station' procedure
library:
sy=popen 'uptime 2>&1' -n uptime &
sy=popen 'ntpq -p 2>&1|grep -v "^[- x#]" 2>&1' -n ntpq &
The 'grep' command selects only the servers marked as '*'
(peer) and '+' (candidates), which are generally the highest
quality servers. You can get information on more servers by
removing '-', '#' and 'x' from the pattern. You can eliminate
the entire 'grep' command (starting with "|", up to, but not
removing, the following "'"), but that will include servers
that can't be reached or are specified 'noselect'; that may be
useful.
'popen' is available in all FS versions starting with
9.11.9. See 'help=sy' for details on its use.
6b. Put a call to 'check_ntp' at the end of 'initi' for a visible
initial check at FS start up.
6c. Put a call to 'check_ntp' at the start of 'sched_initi' to
record the ntp state in the experiment log at the experiment
start (and restarts) for troubleshooting.
6d. If you can, list all servers in '/etc/hosts' with an alias of
at most 15 characters. It should be inserted just before the
canonical name, which is normally the first field after the IP
address. An example of the new entry format is (tabs are often
used for white space to facilitate alignment between lines):
192.168.1.1 myntpclock myntpclock.my.domain
This will substitute the alias for the canonical name in the
'ntpq -p' output, which may otherwise be truncated and/or
difficult read. It is good if the alias makes it easy to
identify the server. The alias can also be used for the server
in the '/erc/ntp.conf' file.
For devices in DNS, the first field of the DNS name may be
suitable to use as an alias. If that is difficult to
read/identify, you can substitute a descriptive name like
'cns'. For local devices not in DNS, you can also select a
descriptive name. For remote devices not in DNS, you might use
something like 'ntp_server.x', where 'x' is the last octet of
its IP address. The 'x' can be up to three digits with no
spaces. If necessary to avoid having more than one alias with
the same 'x', you might append a lowercase letter to
distinguish them.
This should have no impact on other uses of '/etc/hosts', but
don't do this if it is against your site's IT policies.
Listing the computer's own canonical name in its normal field
position (which may be required by site policies) will not
cause a problem with the 'ntpq -p' output. It is only other
devices that need their aliases before the canonical name.
However, defining an alias in this way, as opposed to just
using the FQDN, will mean that if the device's IP address
changes you will need to update '/etc/hosts'. If the IP
address changes only rarely, it is probably worth it. If the
device is not listed in DNS, this is the only way to avoid
referring to it by its IP address.
6e. If your site IT polices require that you protect IP addresses
and FQDNs from being publicly disseminated, you will need to be
careful of what the 'check_ntp" procedure puts in the log. The
logs are often uploaded to public data servers.
To avoid putting FQDNs in the log, you should set up an alias
for each NTP server as described in (6d) above. That will also
make the ''ntpq -p' output easier to read.
(Always using an '/etc/hosts' alias for devices in FS commands
and control files and 'ntp.conf' is an easy way to avoid having
their FQDNs and IP addresses appear in the log. It also makes
it easier to update a device's IP address or FQDN. It only
needs to be updated in one place, '/etc/hosts'.)
To avoid putting any IP addresses in the log with 'check_ntp',
you can add the following two additional pipeline stages before
the closing single quite (') in the 'ntpq -p' command in
'check_ntp' (see 6a):
|sed -E "s/^(.)([0-9].{14})(.*)$/\1REDACTED \3/" 2>&1|sed -E "s/^(.{17})([0-9].{14})(.*)/\1REDACTED \3/" 2>&1
The above is one line and must be inserted literally starting
with the leading pipe character (|) and ending with the numeral
one (1), 121 characters total. This will replace any 'refid'
with 'REDACTED' if it is a network device. It will also
replace any server IP address with 'REDACTED'. The latter may
happen if it does not have an alias in '/etc/hosts'.
7. Make sure NTP is synced before operations are started.
After a power failure, restart the FS computer and wait until its
NTP is synced before starting other devices that use it as an NTP
source. This will assure that they get a good initial time.
After a FS start, particularly after a computer boot, operations
should not start until the 'check_ntp' output shows a server with a
'*' in column one of the 'ntpq' output. That is the server the NTP
daemon is synced with. The operator can run 'check_ntp' manually
as many times as necessary until the output shows a sync. It
should sync quickly if 'ntp.conf' is configured as described above.
Ideally the offsets, the next to last column of the 'ntpq' output,
should be a few milliseconds or less for several servers, hopefully
all.