Skip to content

v2.2.0

Choose a tag to compare

@lambda2 lambda2 released this 04 Sep 18:38
· 152 commits to master since this release
b24f4eb

Hardening of the data provenance system, driven by curating real species rather than by reading the code.

Source arbitration

  • A tie no longer overwrites. Only a strictly stronger source may replace a value that is already filled. When two sources of equal authority disagree, the existing value stays and the newcomer is recorded as a conflicting fact, instead of the outcome depending on which pipeline ran last.
  • Values that predate provenance recording are no longer defenceless. They carry no fact, so nothing used to defend them and any source could overwrite them. They are now treated as claimed at a configurable rank, so only a genuinely stronger source may replace them — while weaker ones can still fill a gap.
  • Regional floras (Flora Iberica, Flora of North America, Flora of China, Flora Europaea) are ranked above the global aggregators for the flora they cover; dendrological and horticultural compilations rank below them.

Data contract

  • Closed vocabularies. A free-text column can now declare the values it accepts, and the ingester refuses anything else — recording the attempt as a rejected fact rather than discarding it silently. nitrogen_fixation takes None / Low / Medium / High, the USDA PLANTS rating scale the column originates from.
  • Cultivation guidance now applies to crops, not to anything with an edible part. Recording that a foraged thistle has edible roots used to make it look less complete, because it unlocked five planting fields nobody would ever fill for a noxious weed.

Provenance API

  • GET /api/v1/species/{id}/facts returned raw bitmask integers for month and part fields — bloom_months: "28". It now returns "mar|apr|may": readable, and stable if the underlying flag order ever changes. Rejected and outranked claims are just as readable as winning ones.

Data quality checks

  • The plausibility check now names what was violated — the specific range or the specific vocabulary — instead of stating only that something was.
  • Every ingester converter is now covered by tests, which surfaced a guard that raised an opaque NoMethodError instead of its intended error.

Notes for API clients: everything is additive. No existing payload changed shape, except that facts values are now readable strings where they were bitmask integers.