Add tests for more complex sorting scenarios.

[delucchi-cmu]

Change Description

This doesn't make any functional changes to the catalog import process. This just provides demonstration (proof?) of discussions around tweaking sort options during catalog import to achieve particular ordering of resulting parquet files.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (9ed8952) 100.00% compared to head (d102816) 100.00%.

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #167   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           27        27           
  Lines         1158      1158           
=========================================
  Hits          1158      1158           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[hombit]

Thank you!

It would be helpful to check that sources are still sorted by _hipscat_index.

[hombit]

The other (or additional) way to test it is doing something like the following:

1. Concatenate pandas data frames,
2. sort them by (healpy.ang2pix(ra, dec), source_id) in the first case and by (healpy.ang2pix(ra, dec), object_id, time) in the second case,
3. and compare hipcat files with these sorted dataframes (skipping hipscat-specific columns like _hipscat_index)

[delucchi-cmu]

The other (or additional) way to test it is doing something like the following:

1. Concatenate pandas data frames,
2. sort them by (healpy.ang2pix(ra, dec), source_id) in the first case and by (healpy.ang2pix(ra, dec), object_id, time) in the second case,
3. and compare hipcat files with these sorted dataframes (skipping hipscat-specific columns like _hipscat_index)

Added dataframe comparisons alongside more boring series comparisons.