v0.5.0

panoptes-data 0.5.0

This release moves the package onto the documents panoptes-pipeline writes
and the parquet index built over them, and gives it a query surface that can
express benchmark selection. It is mostly breaking.

Metadata is read from the processed tree -- an observation.json per sequence,
a metadata.json per frame -- and from observations.parquet /
frames.parquet, rather than from a Firestore-derived observations.csv and a
cloud function. Both were downstream of a pipeline that stopped producing them.
There is no fallback: it is not that the cloud sources are deprecated, it is
that nothing makes them.

Set PANOPTES_PROCESSED_ROOT to the document tree, PANOPTES_INDEX_ROOT if
the index sits elsewhere, and PANOPTES_ARCHIVE_ROOT to a local copy of the
raw frames. The three are different trees and conflating them is the easiest
mistake to make here.

• Every column a caller touches is renamed to the contract's flattened
  document names: uid is image_uid, num_images is num_frames,
  coordinates.mount_ra is mount_ra, time is sequence_time. Joined with
  _ and never ., because a dotted name is a view over a nested map rather
  than storage.

• search_observations takes keyword arguments only. It has eighteen
  parameters, and inserting one used to rebind every positional argument after
  it.

• min_num_images is min_num_frames, and the status argument is gone;
  num_usable is the better question.

• download_images and the download CLI command raise. Archived frames
  404 anonymously in every era of the bucket, and there is no public
  replacement. Frames are read from a local archive instead.

• SurveySettings.img_metadata_url and observations_url are removed.

• The two example notebooks are removed; their worked examples are in the
  README.

• All-sky search. A position is optional, so a unit and a date range, or a
  frame count and a duration, are each a complete query.

• Benchmark cuts: min_num_usable, min_duration_minutes, field_name,
  camera_id, and a trailing query string. num_usable counts frames the
  pipeline processed cleanly, as distinct from frames that exist.

• duration in place of end_date: "90 days", "6 months before",
  "10 days before and after", or a timedelta.

• iso, airmass, moonfrac, moonsep attached to every sequence, since
  observations.parquet has no column for a per-frame reading.

• find_simultaneous, pairing sequences of one field recorded at the same
  time by different cameras or units, on interval overlap rather than a
  calendar night. Plus a pairs CLI command.

• duration_minutes and a total_exptime that is populated for the long
  sequences the old summary left null.

• get_metadata raises on a partial read rather than wrapping the whole loop
  in except Exception: pass, which made half the archive and all of it the
  same return value.

• The search and get-metadata CLI commands no longer bind one short flag to
  two options: -s set the end date and -r set the radius.

• A search cone spanning 0h RA finds sequences again, and is widened per
  sequence by that sequence's own pointing drift.

• A frame document that is missing or unreadable raises rather than yielding a
  quietly shorter observation.

The full list is in CHANGELOG.md.

Fetch works on every sequence again

Fixed

• ObservationInfo no longer requires a URL column in the image metadata, so
  it works on every sequence again. It had read public_url, which records
  written from 2025 onward do not carry, and raised an AttributeError on all
  of them. Image URLs are now derived from the uid, which is present in every
  era and is the archive path with underscores for separators.
• A missing required metadata column now raises a ValueError naming the
  sequence and listing the columns that did arrive, instead of an
  AttributeError from pandas.
• The download and get-metadata CLI commands exit non-zero when they fail,
  instead of catching every exception and printing it in red.

Added

• ObservationInfo.public_urls, the *_url columns of the metadata, for
  display and lookup. Which ones exist depends on when the record was written;
  nothing in the package requires any of them.

---

On PyPI: https://pypi.org/project/panoptes-data/0.3.0/

Hotfix for v0.2.2 - testing the release building process

Merge pull request #11 from panoptes/modernize-repo-hotfix-2

Modernize repo hotfix 2 - Trying to test the release build.

Hotfix for v0.2.1

What's Changed

• • Add testing dependencies. by @wtgee in #10

Full Changelog: v0.2.1...v0.2.2

Update repo, add tests

What's Changed

• Modernize repo by @wtgee in #9

Full Changelog: v0.2.0...v0.2.1

Fix the search

Fixes the search columns and allows for some better searching options. Disables the rich exception output as its overly verbose.

Fix image downloading

Fixes the image list and pattern matching for new url scheme that contains full image id.

v0.1.8

Fix the default image storage bucket and allow it to be flexible in the future.

v0.1.7

Updating the image_list to work from latest processing and just use public_url.

v0.1.6

Fix the image_list to point to the correct bucket and also allow for changing the bucket.