Repository navigation
ML Models
PFR Sentinel includes two machine learning models that analyse pier camera frames as they are captured: a Roof Classifier that detects whether the observatory roof is open or closed, and a Sky Classifier that assesses sky conditions while the roof is open. Both run locally on your PC; no cloud service or internet connection is involved.
ML analysis is a Beta feature and is off by default. Turn it on with Enable ML Analysis in the ML Models (Beta) card on the Image Processing tab. It is part of the released application, not a developer-only feature.
Decides whether the roof is open or closed from the pier camera image.
| Property | Value |
|---|---|
| Input | Frame reduced to 128x128 greyscale, plus 4 context values |
| Output | Open or Closed, with a confidence percentage |
| Accuracy | 99.8% on a held-out test set of 1,120 frames, and 327 of 327 on frames added after the June model (September 27, 2026 model) |
Alongside the image, the model uses:
| Context value | Description |
|---|---|
| Corner-to-centre ratio | Brightness of the frame corners compared with the centre. A closed roof tends to be evenly dark; an open sky has a gradient. |
| Median brightness | Overall frame brightness, scaled correctly for 8-bit and 16-bit frames. |
| Night flag | Whether it is astronomical night at your location (sun more than 18° below the horizon), from the latitude and longitude in Weather Setup. With no location set, the PC clock between 22:00 and 05:00. |
| Hour | Hour of day from the PC clock. |
New in the next release — not available in version 3.7.7 or earlier.
The night flag follows real twilight at the observatory. Earlier versions used the PC clock (20:00 to 06:00), which disagrees with the sky by up to several hours depending on season and latitude, and this flag is what tips a borderline frame. Set your location under Weather Setup for it to apply. The September 27, 2026 models were trained with the corrected flag; the models shipped before that were trained on the clock-based one, for which the corrected flag is still closer to what they learned than the clock was.
Assesses the sky. It only runs when the roof classifier reports Open, because with the roof closed the camera can't see the sky.
| Property | Value |
|---|---|
| Input | Frame reduced to 384x384 greyscale, plus 6 context values |
| Output | Sky condition, whether stars are visible, star density, whether the moon is visible |
| Accuracy (sky condition) | 93.2% overall on a held-out test set of 340 frames: Clear 95.4% (269 of 282), Partly Cloudy 39 of 47, Overcast 9 of 11; 86.0% on the 93 test frames added after the June model (September 27, 2026 model) |
New in the next release — not available in version 3.7.7 or earlier.
The sky classifier is now built on an image-recognition network pre-trained on general photographs and fine-tuned on pier camera frames, instead of a network trained from nothing. On frames neither model had seen, it is right 86% of the time against 70% for the June model, and Partly Cloudy went from 54% to 71%. The same network trained from nothing scores exactly like the June model, so the gain comes from the pre-training. It costs about 4 ms more per frame on a CPU.
| Output | Values |
|---|---|
| Sky condition | Clear, Partly Cloudy, Overcast |
| Stars visible | Yes / No |
| Star density | 0.0 (none) to 1.0 (dense star field) |
| Moon visible | Yes / No |
The sky classifier uses the same four context values as the roof classifier, plus:
| Context value | Description |
|---|---|
| Moon illumination | Current moon phase as a percentage |
| Moon is up | Whether the moon is above the horizon at your location |
"Moon is up" needs latitude and longitude in the Weather API card on the Settings tab (see Weather Setup). Without coordinates the model is always told the moon is down.
Non-square frames are centre-cropped to a square before they are reduced, so the image isn't squashed.
- Trained on one observatory. Both models were trained on frames from a single pier camera (a ZWO ASI676MC with square frames). Other cameras, lenses, fields of view and enclosures may give noticeably different results. Watch the predictions for a few nights before relying on them.
- Overexposure hurts roof detection. In testing, the roof classifier's remaining errors were overexposed frames, typically bright daytime frames with the roof open. Bright moonlight on its own was not a significant cause of errors.
- Moonlit clear nights can read as Partly Cloudy. Moon glow and twilight wash out the sky in a way that looks like thin cloud. Much of the training data carries the same ambiguity, so treat Partly Cloudy on a bright-moon night with caution.
- Partly Cloudy is the weakest class. The September 27, 2026 model was right on 39 of 47 test frames (83%), up from about 65% for the June model, but thin cloud against a clear sky is a judgement call even for a person, and the labels carry that ambiguity. Overcast has only a handful of test frames, so its score is not a reliable estimate.
- Time features use the PC clock. Make sure the observatory PC's time and time zone are correct.
- The models see the raw frame. Predictions are made before stretching, brightness and overlays, so image processing settings don't affect them. Very underexposed or overexposed raw frames do.
With ML enabled, these tokens can be used in text overlays (see Overlay Tokens):
| Token | Example Value |
|---|---|
{ROOF_STATUS} |
Open (95%) |
{SKY_CONDITION} |
Clear (87%) |
{STARS_VISIBLE} |
Yes |
{STAR_DENSITY} |
High (0.85) |
While the roof is closed, the three sky tokens show N/A. Star density is labelled High above 0.6, Medium above 0.3, and Low otherwise.
The status strip in Live Monitoring has Roof and Sky tiles. Roof shows Open or Closed, and Sky shows the sky condition, each colour-coded. While the roof is closed, the Sky and Seeing tiles show "Roof closed". Both tiles show "Not configured" when ML is off.
New in the next release — not available in version 3.7.7 or earlier.
A frame that is almost entirely sensor noise gives the models nothing to judge. This is typical of a closed roof at night with the exposure at its maximum. On such a frame the roof model can report a low-confidence Open, and the sky model a confident Clear, when neither is true.
PFR Sentinel measures each frame for this. When a frame is only noise, the Roof tile turns amber and reads Open · unreliable or Closed · unreliable, the Sky tile reads Too much static, and hovering either tile explains why. The log records when it starts and stops.
This is a warning only. The roof reading, the overlay tokens, roof change notifications and the ASCOM safety file all still use whatever the models reported.
New in the next release — not available in version 3.7.7 or earlier.
With Skip Sky Features When Roof Closed on (the default), star detection and the All-Sky Overlay, including its background calibration, pause while the roof reads Closed. The roof has to read Closed on two frames in a row before they pause, matching the safety file and the roof alert, so one misread frame — typically when the exposure changes — does not blank the overlay. If your camera has no roof, for example an open-air all-sky camera, turn this off so a mistaken Closed reading can't switch those features off.
The roof reading is now corroborated by the frame itself. A frame flagged as sensor noise only, a frame exposed for less than the Minimum exposure floor, or a roof that reads Open while no stars are detected on three frames in a row also pauses those features — a misread Open on a lit roof, or an overcast sky, no longer gets constellation labels. The floors are on the All-Sky page; see When the overlay is drawn. A Closed reading is never overridden by detected stars.
The Meteor Detection roof check is separate and does not follow this setting or the star check. With ML analysis on, meteor detection pauses whenever the roof is not reported Open, which includes Closed and uncertain (N/A) readings, and also when only the sky model loaded. See Meteor Detection for details. The roof change notifications, the timelapse roof mode and the ASCOM safety file likewise follow the roof classifier alone.
Discord and Hermes can post a message when the roof opens or closes. A change is only reported after two consecutive frames agree, so a single misread frame doesn't trigger a notification.
The Timelapse tab has a Roof Open (Beta) recording window that records while the roof classifier reports the roof as open. ML analysis must be enabled for it to work.
The Image Library stores the roof and sky results with each frame and uses them to colour its night-by-night condition band.
PFR Sentinel can write the roof status to a text file watched by NINA's GenericFile safety monitor, so NINA can react when the roof closes. The file is written in ZWO Camera mode only.
| Setting | Default | Description |
|---|---|---|
| ASCOM Safety File | Off | Write roof status to the file. |
| File Path | %LOCALAPPDATA%\PFRSentinel\RoofStatusFile.txt |
The file NINA monitors. Type a path or use Browse. |
File format:
Roof Status: OPEN
Confidence: 95%
Sky Condition: Clear (87%)
Updated: 2026-09-17 22:45:30
The Confidence line is left out when there is no roof reading, and the Sky Condition line when there is no sky result.
NINA GenericFile monitor configuration:
- Preamble:
Roof Status: - Safe trigger:
OPEN - Unsafe trigger:
CLOSED
The file is designed to fail safe:
-
Only a confident Open is safe.
OPENis written only when the roof reads Open with at least 70% confidence. A Closed reading, a low-confidence reading, or a frame where ML is switched off, the models aren't loaded or the prediction failed all count as unsafe. Losing the ML result therefore moves the file toCLOSEDrather than leaving it stuck on a staleOPEN. -
Two frames to change state. The file only switches between
OPENandCLOSEDafter two consecutive frames agree, so one noisy frame can't flip it. -
Starts unsafe. After PFR Sentinel starts, the first write is always
CLOSED, which also clears anyOPENleft over from a previous run. It then needs two consecutive confident Open frames to switch toOPEN; if the very first frame is a confident Open, it counts as the first of the two. - Kept fresh. While the status is steady, the file is rewritten every 5 minutes so NINA's maximum-file-age check doesn't treat a healthy session as stale. Set NINA's maximum age comfortably above both 5 minutes and your capture interval.
- Write failures are reported. If the file can't be written (bad path, disk problem), an error is logged and an "ASCOM Safety Write Failed" error notification is raised, which reaches Discord or Hermes if you have error notifications turned on there.
Because the file only changes when frames are processed, it can only be as current as your capture interval.
The models ship with the application and run with ONNX Runtime. They load when you turn on Enable ML Analysis, or at startup if it is already on. The status line in the ML card ("Status: ...") shows the result:
| Status | Meaning |
|---|---|
| ✓ Both models loaded | Roof and sky models ready |
| ✓ Roof model only | Sky model failed to load; roof results still available |
| ✓ Sky model only | Roof model failed to load. The sky model only runs when the roof reads Open, so no sky results are produced either |
| ✗ No models available | Neither model loaded; the reason is shown in brackets |
| ✗ Error: ... | Something went wrong while checking the models; the start of the error is shown |
| Disabled | ML analysis is off |
In ZWO Camera mode, ML results drive everything on this page. In Directory Watch mode the models only provide the overlay tokens: the Live Monitoring tiles, roof change notifications, the ASCOM safety file, the timelapse and meteor detection roof checks, and data contribution are ZWO Camera mode features.
Development builds can additionally save per-frame calibration data (FITS and JSON files with the model predictions) for retraining. That is a developer tool and is not part of the released application.
As of the September 27, 2026 models, the roof classifier was trained on 5,598 labelled pier camera frames, 1,000 of them new since June, with 1,632 sky labels rechecked by hand before the retrain. The sky classifier was trained on the 2,262 roof-open frames among them, which are dominated by clear nights. The June 2026 models used about 4,000 frames, reviewed to remove moon-glow and closed-roof mislabels.
Ground truth came from:
- The safety monitor state reported by NINA's Advanced API plugin
- OpenWeatherMap cloud cover and conditions
- Moon phase and position
- Manual labelling of sky condition, star visibility and moon visibility
An opt-in programme lets you contribute anonymous training data so future models work better across different cameras and observatories.
- Turn on Enable Data Contribution in the Community Data Contribution card on the Image Processing tab.
- While capturing in ZWO Camera mode, PFR Sentinel saves at most one sample every 30 minutes.
- Samples are stored locally in
%LOCALAPPDATA%\PFRSentinel\ml_contribution\. - When you're ready, click Export for Upload to create a ZIP, then Open Upload Form and submit it through the Google Form.
- After a successful upload, click Clear to delete the local samples.
Collection stops automatically at 500 samples (about 130 MB). This works whether or not Enable ML Analysis is on.
- FITS image: 256x256 luminance-only, 32-bit float (about 260 KB). The caption in the app says about 80 KB, which is out of date.
- Calibration JSON: metadata and statistics (about 5 KB)
| Field | Description |
|---|---|
| Camera name | Model identifier, for example ZWO ASI676MC |
| Exposure, gain, bit depth, Bayer pattern | Capture parameters |
| Brightness statistics | Median, mean, percentiles and dynamic range |
| Spatial analysis | Corner-to-centre brightness ratio |
| Time context | Time of day, and that day's sun and moon rise and set times |
| Moon context | Phase, illumination and whether it is up |
| Roof state | The safety monitor state from NINA's Advanced API plugin, if NINA is running with the plugin available |
| Weather conditions | From OpenWeatherMap, if configured |
Before a sample is saved, these are stripped out:
- Coordinates and location names
- Weather city and location
- File paths
- All-sky camera addresses and file names
The sun and moon rise and set times are kept. Because they depend on where you are, they give a rough indication of your location.