ntpstats 3.6.0

The 2012 thesis, finished (#40). No breaking changes.

Added

• Clock disciplines in the bench (ntpstats.disciplines), steered in closed loop like the PTP
  servos: ntpd (ntpd 4.2.8's clock filter, state machine and hybrid PLL/FLL, from
  ntp_loopfilter.c), ntpd-rfc (the same with the RFC 5905 appendix's constants), lockclock
  (J. Levine's NIST frequency-lock loop, J. Res. NIST 2020) and levine-kalman (with Levine's scalar
  Kalman time estimate, PTTI 2011). The appendix's PLL gain of 65536 gives a phase time constant of
  about 48 days at poll 6; implementations follow ntpd's 16 (Trace replay & PTP).
• ntpstats poll and poll_advice(): the predicted error just before the next poll for each
  candidate interval (the log decimated to it, then the holdover model), and the longest interval
  that meets --target (exit code 3 if none) (Metrology).
• research/thesis-2012: the 2012 thesis assessed and its experiments re-run, now with the ntpd
  model's step and ramp responses and the poll-interval advice for the 2012 clock.

Stable API changes since 3.5.0

• Added: ntpd_discipline, lockclock, poll_advice, PollAdvice. Nothing removed or changed.

ntpstats 3.5.0

GNSS/PPS timing on Linux hosts and in the lab. No breaking changes.

Added

• gpsd (format gpsd, gpspipe -w logs): PPS and TOFF series per device, GNSS time minus the
  system clock's time stamp, with precision and the receiver's qErr (gpsd_json(5)).
  ntpstats watch gpsd samples a running gpsd (host:port), also from the web UI's Live
  workspace. ntpstats sawtooth uses the qErr of a gpsd PPS log when no UBX log is given. (#21)
• TAPR TICC (format ticc): Timestamp, Period, 3-Corner-Hat and Time Interval output; per-channel
  phase against the nominal period, chA - chB of paired events, WRAP seconds unwrapped, seconds and
  fractions kept apart so picoseconds survive (format from the TICC firmware and manual).

Stable API changes since 3.4.0

• None. Code written against the stable API of an earlier release keeps working.

ntpstats 3.4.0

GNSS timing receivers and time-stamping sources. No breaking changes.

Added

• u-blox UBX receiver logs (format ubx, detected automatically, also mixed with NMEA and in the
  web UI): UBX-NAV-CLOCK becomes a receiver-clock series (bias as reported, with drift, time and
  frequency accuracy), UBX-NAV-TIMEUTC time-stamps it, and UBX-TIM-TP gives the time-pulse
  quantization error qErr. Layouts from the u-blox interface description. (#21)
• ntpstats sawtooth and ntpstats.ubx.apply_qerr: remove the PPS quantization sawtooth from a
  time-interval-counter measurement with the receiver's qErr. The sign with which qErr applies
  (not stated by u-blox) is chosen from the data and reported, or given with --sign; TDEV before
  and after is printed.
• chrony time-stamping sources: measurements.log series carry interleaved, tx_timestamp and
  rx_timestamp (daemon, kernel or hardware) and a summary in meta["timestamping"], also shown on
  the web UI's network page.
• Gallery notebook Removing the GNSS PPS sawtooth with u-blox qErr, with a synthetic receiver log
  and counter log (examples/data/ubx-timing.ubx, pps-tic.csv). (#38)

Changed

• Times below a femtosecond are printed as 0 (they are numerical noise).

Stable API changes since 3.3.0

• None. Code written against the stable API of an earlier release keeps working.

ntpstats 3.3.0

A quality release: the web UI, the command line and the documentation were checked end to end
(every page with every example dataset, on desktop, at phone width and in the dark theme; every
command on every example file), and what that found is fixed. No breaking changes.

Added

• Gallery notebook PTP through networks without PTP support: ptp4l-style and SPTP-style clients,
  delay filters and a Huygens-style estimator on the same PTP exchanges with ground truth, how much
  transparent-clock coverage is enough, a narrower servo, and RFC 10030 corrections on a capture.
• Command palette: choose the input format for the next files (type "format"), and simulate the PTP
  presets.
• Docs site: a description for every main page and a clearer site description, so that searches
  for NTP/PTP stability, time-error or capture analysis can find it; the tools landscape lists
  ntpxyz, chrony_ntp_logconv, PTP-DAL and timeTools.

Changed

• Command line: a missing file, unreadable input or impossible request prints one line
  (ntpstats: error: …) and exits with status 2 instead of a Python traceback;
  NTPSTATS_DEBUG=1 shows the traceback.
• Time error: when the sampling is too slow for the filter bandwidth, dTE_H is reported as not
  available (it was numerical noise), and a limit on it is reported as not checked.
• Web UI: the reference picker lists datasets that overlap the active one first and marks the
  others; the network page explains when a dataset has no round-trip delay instead of failing a
  request.

Fixed

• Web UI: log-scale charts hung (and raised a script error) on deviations that are zero to
  numerical precision, e.g. a noise-free ramp; such values are now gaps, with a note.
• Web UI at phone width: the header, page tabs, comparison table, events table, reference
  picker, chart legends and the audit rules no longer push the page sideways; the dTE_H card no
  longer shows a raw number.
• Web UI: charts of a page left mid-render no longer stay marked as loading.
• Loading a directory skips broken links and other non-regular files.
• Stability slopes no longer warn on a zero deviation.

Stable API changes since 3.2.0

• None. Code written against the stable API of an earlier release keeps working.

ntpstats 3.2.0

New daemons and protocols as sources (#21). No breaking changes.

Added

• ntpd-rs as a live source: ntpstats watch ntpd-rs samples ntp-ctl -f prometheus status,
  or the metrics exporter's URL (--command-override http://127.0.0.1:9975/metrics). Each sample
  is the inverse-variance mean of the per-source offsets ntpd-rs 1.x exports, with the number of
  sources, the best source's uncertainty and delay, and the system root delay, dispersion and
  stratum. The 2.0 pre-releases no longer export per-source offsets; watch says so. Also in the
  web UI's Live workspace.
• CSPTP in captures: client-server PTP (sdoId 0x300, as in ntpd-rs 2.0 and statime) is read
  from pcap/pcapng files as one series per client/server pair. Every exchange gives all four
  timestamps; transparent-clock corrections are removed in both directions; the PTP-timescale
  offset is inferred; the CSPTP status TLV gives the grandmaster and steps removed. These messages
  are no longer mistaken for master/slave flows.
• ntpstats.sources.parse_prometheus_text, a reader for the Prometheus/OpenMetrics text format.
• Tests built from the example lines of the chrony 4.9 documentation (measurements, tracking,
  statistics): columns and signs match the parsers.

Changed

• Docs: the prom page notes that ntpd-rs offset metrics are those of the 1.x series.

Stable API changes since 3.1.0

• None. Code written against the stable API of an earlier release keeps working.

ntpstats 3.1.0

PTP exchanges in the research bench, and NTP over PTP in captures. No breaking changes.

Added

• PTP exchanges in the bench (completes #29):
  a scenario with protocol = "ptp" simulates the E2E delay mechanism, with a Sync every poll
  seconds and a Delay_Req every delay_interval seconds, each with its own one-way delay and
  timestamp noise. transparent is the fraction of queueing delay that transparent clocks correct.
  The series carries the one-way ms/sm measurements and the slave's offset, so every estimator
  runs on it. New presets ptp-lan (switches without PTP support) and ptp-tc (transparent
  clocks), also on the Simulate page of the web UI.
• Two PTP client models as estimators: ptp4l (moving-median path delay, the linuxptp PI servo or
  linreg, in closed loop) and sptp (SPTP-style: complete exchanges, path-delay outlier discard,
  PI servo). ntpstats simulate --preset ptp-lan --benchmark scores them with the others.
• NTP over PTP (RFC 10030, chrony 4.9): NTP messages in the NTP TLV of PTP event messages are
  read from pcap/pcapng files as NTP exchanges. The transparent-clock corrections (PTP correction
  field of the response, Network Correction extension field for the request) are applied as the
  RFC specifies, and refused when it forbids them. Uncorrected values and both corrections are
  kept as columns. Example: examples/data/ntp-over-ptp.pcap.

Fixed

• ntpstats trace on captures shorter than about one detrend window (15 minutes by default):
  the clock's offset is now removed with at least four windows when there are enough exchanges,
  so its drift no longer appears as delay variation.
• Web UI overview: the Events card listed "none detected" under a non-zero count.
• ntpstats simulate --benchmark on PTP presets does not score the first 300 s, while the servos
  lock.

Stable API changes since 3.0.0

• Changed: PathModel: new method floor; Scenario: new optional fields protocol,
  delay_interval, transparent.
• Removed: nothing. Code written against the stable API of an earlier release keeps working.

ntpstats 3.0.0

The stable API is final. No breaking changes: code written for 2.15 or later runs unchanged.

Changed

• ntpstats.api is final for the whole 3.x series. Stable names are only added or deprecated
  within 3.x, and a deprecated name keeps working, with a warning, until the next major
  release. The command line and the files ntpstats writes follow the same rule
  (Stable API docs page, CONTRIBUTING).
• Package metadata: "Production/Stable", supported Python versions (3.9 to 3.13), typed, and
  links to the documentation and changelog. Security fixes go to the latest 3.x minor release
  (SECURITY.md).

Stable API changes since 2.15.0

Generated with python docs/api_changes.py v2.15.0.

• Added: ChainResult (class, from ntpstats.ptpsim); ChainScenario (class, from ntpstats.ptpsim); DelayTrace (class, from ntpstats.trace); LinRegServo (class, from ntpstats.ptpsim); Link (class, from ntpstats.ptpsim); PIServo (class, from ntpstats.ptpsim); TracePath (class, from ntpstats.trace); inet_oscillator (function, from ntpstats.simio); load_trace (function, from ntpstats.trace); read_omnetpp_vec (function, from ntpstats.simio); simulate_chain (function, from ntpstats.ptpsim); trace_from_series (function, from ntpstats.trace); write_omnetpp_vec (function, from ntpstats.simio).
• Changed: Scenario: new init trace=; new fields trace; ServerSpec: new init trace=; new fields trace.
• Removed: nothing. Code written against the stable API of an earlier release keeps working.

Added

• docs/api_changes.py: lists the stable-API changes between a tag and the working tree, from
  the frozen surface, for release notes.
• A draft software paper for the Journal of Open Source Software (paper/), and a workflow that
  builds its PDF (#38).

Documentation

• New docs home page organised by task; the README introduction, the tools landscape, the
  roadmap ("After 3.0") and the wiki describe the toolkit as it is now.

ntpstats 2.17.0

Research bench v2, part 2: completes #29.

Added

• Reference algorithms:

  • hull: Huygens-style. The max-margin line through the offset bounds θm ± δ/2 of a sliding
    window, causal.
  • kalman-combine: multi-server, ntpd-rs-style. Per-source Kalman filters, interval
    intersection, inverse-variance mean.

  With symmetric floors hull is the most accurate single-server estimator on the presets.
  kalman-combine rejects falsetickers and stepping servers.

• A reproducible benchmark on a replayed trace: examples/scenarios/replay-chrony.toml scores
  every estimator on the delays of a real log under a simulated clock
  (docs/examples/trace-benchmark.html). CI runs it.

• Stable API: delay traces, PTP chains and the simulator formats join ntpstats.api
  (load_trace, TracePath, ChainScenario, simulate_chain, read_omnetpp_vec,
  inet_oscillator and the rest).

• Web UI overhaul, still plain JavaScript with uPlot, no build step, offline.

  • Five workspaces: Analyze, Compare (side by side, against a reference, N-cornered hat),
    Comply (time error with TE/TEL and MTIE/TDEV charts; UTC audit with a verdict and an evidence
    report), Lab (simulator, estimator bench, PTP boundary-clock chains) and Live.
  • A command palette (Ctrl K), keyboard shortcuts, a link for every page (#/comply/audit), and
    dataset sparklines, filter and rename.
  • Expandable charts, one-way delays on the Network page, and stale answers are never drawn.
  • New API endpoints: audit, trace, compare, hat, estimators, bench, chain, and the
    audit.html export.
  • A Playwright test drives every page in CI.

• Simulator interop (#29): OMNeT++ .vec result files are read like logs (INET clock
  timeChanged vectors become time error against simulation time; other vectors on request) and
  written (convert --to omnetpp-vec). ns-3 time value text is written with convert --to ns3.
  ntpstats noise --inet writes INET RandomDriftOscillator settings that reproduce the fitted
  random-walk FM, and lists what INET's oscillator cannot represent.

ntpstats 2.16.0

Research bench v2, part 1 (#29). The new modules are provisional (not yet in ntpstats.api) until
2.17 adds the simulator exchange formats.

Added

• Delay traces (ntpstats.trace, ntpstats trace): per-direction one-way delays extracted
  from any two-way exchanges.
  • Inputs: NTP and PTP captures, chrony measurements.log, ntpd peerstats/rawstats, and
    simulator output.
  • The clock offset between the two ends is removed (--detrend floor|linear|none).
  • The asymmetry is an explicit assumption (--asymmetry), since two-way timestamps cannot
    measure it.
  • ntpstats trace reports floors, PDV percentiles, loss and the correlation of the two
    directions; --csv exports the trace.
• Trace replay in the bench: ntpstats bench trace:FILE, or a [trace] table in a scenario
  file.
  • Modes: replay plays the trace in order; bootstrap draws random blocks, keeping the
    short-term correlation and both directions together.
  • scale multiplies the queueing.
  • Checked by a round trip: delays extracted from a simulated network and replayed give the
    same estimator scores as the model they came from.
• PTP servos and boundary-clock chains (ntpstats.ptpsim, ntpstats chain): a grandmaster
  and N boundary clocks, simulated with ground truth.
  • Each node has its own oscillator, the E2E delay mechanism with a moving-median filter, and
    linuxptp's PI servo (with its default gains) or an adaptive linear-regression servo.
  • Links have asymmetry, timestamp noise, and modelled or replayed PDV.
  • Time-error metrics are given per node and per hop: max|TE|, |cTE|, dTE_L MTIE, dTE_H.
  • Each hop is checked against the commonly quoted G.8273.2 T-BC class limits (A/B/C) or your
    own limits, and the end of the chain against a budget (default 1.1 µs); exit code 3 on
    failure.
  • --kp/--ki tune the servo. The warm-up follows the servo's lock time, and the command
    warns when a run is too short to lock.
• New notebook: a PTP chain against a time-error budget. With linuxptp's default gains, dynamic
  time error grows much faster than the number of hops (gain peaking): about 50 ns at 10 hops,
  about 600 ns at 20. A narrower loop keeps 20 hops near 25 ns. The bench notebook now also
  replays a real log.

Fixed

• Noise identification no longer divides by zero on perfectly alternating or noise-free data (lag-1
  autocorrelation of −1, the white-PM limit).
• Commands piped into head and similar tools stop quietly instead of printing a
  BrokenPipeError traceback.

ntpstats 2.15.0

Added

• Stable API (#32): from ntpstats import api as nt gives the public surface in one namespace
  (loading, stability, analysis, metrology, time error and assurance, estimators, simulator,
  bench, reports, plugin types).

  • Covered by a deprecation policy: one full minor release of NtpstatsDeprecationWarning
    before a stable name or parameter changes. The warning is a FutureWarning, so it is shown
    in scripts and notebooks.
  • Helpers for contributors in ntpstats.deprecation: deprecated, renamed_parameter,
    moved.
  • The surface is frozen in tests/data/api_surface.json; CI fails on incompatible changes.
  • Documented on the new Stable API page; the policy is in CONTRIBUTING.

• Large files (#32, continues #13): load_large and iter_chunks read line-oriented logs in
  blocks, gzip included, and give the same series as load.

  • In a 300 000-line test, peak memory was about 2.4 times lower and reading twice as fast.
  • load switches to block reading by itself for text logs above 256 MB.
  • Supported formats: ntpd/NTPsec stats, chrony logs, linuxptp, CSV and the 2012 log.

• Notebooks and reproduction gallery (#32, #38): four notebooks, executed in CI on every change:

  • a chrony log to stability with intervals and a noise model;
  • benchmarking your own estimator;
  • compliance evidence (PTP time error, a UTC bound, a mask, an HTML report);
  • a reproduction of the NIST SP 1065 test suites.

  A docs page explains how to contribute a gallery entry.

With these, #32 is complete. Its last step, a 3.0 changelog that lists every change to the stable
API, is part of the 3.0 release checklist (CONTRIBUTING). The research bench v2 (#29) is planned
for 2.16 and 2.17.