Skip to content

v1.3.0 - Overwrite mode for destinations holding known-bad data

Choose a tag to compare

@mayerwin mayerwin released this 01 Sep 07:32
· 12 commits to main since this release

New feature

⚠️ Overwrite mode, for destinations holding known-bad data

Requested in #14. Until now the import never touched data the destination already had: it filled in front of the oldest entry, and with gap-fill it filled empty stretches. That is the right default, but it cannot help when the destination's existing data is itself wrong. A new sensor that recorded zeros while it was being commissioned alongside the old one looks fully covered, because a stored 0 is a real value, not a hole. The same applies after an import made with the wrong unit: the destination now has data, so a corrected re-run has nothing to fill.

Under Options there is now an ⚠️ Overwrite existing destination data checkbox, off by default. With it on, wherever the source has data, the source wins.

What it does

  • Destination state rows inside the source's time span are deleted and replaced by the source's states, whatever the gap-fill setting says.
  • Destination statistics values are overwritten, hourly and 5-minute, for every column the source provides. Columns the source does not provide keep their existing values.
  • Data outside the source's time span is never touched, and the most recent slots Home Assistant may still be compiling are still skipped.
  • For energy sensors the splice offset is computed against the destination rows that survive the overwrite. Without that, a corrected series would be pulled down to line up with the very values it is replacing.

Guard rails

Deleted state rows cannot be recovered without a database backup, so the option is deliberately hard to trigger by accident:

  • Ticking the box unfolds a full warning explaining exactly what will be removed.
  • Starting an import with it on asks for a second confirmation, listing the destination entities that will lose data.
  • Preview works as usual and reports the number of rows that would be replaced along with the exact time window, so you can check the range before committing.
  • The results panel shows the same counts and window after the run, and the deletion is logged as a warning.
  • The deletes happen in the same transaction as the inserts. If anything fails, the deletes roll back too.

Also available on the service call as overwrite: true.

Notes

  • Verified against a SQLite database mirroring Home Assistant's states schema with foreign keys enforced: rows outside the span and other entities stay untouched, references from surviving rows into the deleted range are cleared the same way Home Assistant's own purge does, and a simulated mid-run failure restores every row.
  • Re-running an overwrite is stable: the second run replaces its own rows with identical values.
  • Back up your database before using this. It is the one option in the integration that deletes existing data.

Thanks to @Ryeera for the suggestion and the clear description of the case.