Kartothek Integration #137
Conversation
…oned parquet file and provides same API as original Dataset class
lsst_dashboard/dataset.py
Outdated
| if self.tracts is None: | ||
| self.tracts = list(set(x for v in self.metadata['visits'].values() for x in v.keys())) | ||
|
|
||
| def parse_metadata_from_bulter(self): |
There was a problem hiding this comment.
Typo here (bulter -> butler)
lsst_dashboard/dataset.py
Outdated
| except: | ||
| raise | ||
| print(f'{self.path} is not available in Butler attempting to read parquet files instead') | ||
| self.parse_metadata_from_bulter() |
| except: | ||
| raise |
There was a problem hiding this comment.
A good thing to do here would be to catch & print the exception information before proceeding to test data, since often this code may hide some stack problem even if actually on the LSST system
| self.tracts = list(set(x for v in self.metadata['visits'].values() for x in v.keys())) | ||
|
|
||
| def fetch_coadd_table(self, coadd_version='unforced'): | ||
| table = 'qaDashboardCoaddTable' |
There was a problem hiding this comment.
I don't think we will be using any dataset with this name anymore.
| # hack label in ... | ||
| coadd_df['label'] = 'star' | ||
|
|
||
| self.coadd[table] = coadd_df |
There was a problem hiding this comment.
Let's change the key here (and wherever else necessary) to be dataset rather than table, because the above table definition will not exist in the new format.
| ddf = self.get_visits_by_metric_filter(filt, metric) | ||
| except: | ||
| print('WARNING: problem loading visits for {} metric and {} filter'.format(metric, filt)) | ||
| ddf = None |
There was a problem hiding this comment.
Maybe print a traceback here, too? Or tighten the except to catch precisely what happens when the metric doesn't exist in the visit data?
There was a problem hiding this comment.
@timothydmorton yes, I think at least using traceback.print_exc() would help the cause. The PR is getting large so I'm going to hit these items in a follow up
|
Can we also on this PR update the test data so that the CI works? (maybe make a 0.1% (or smaller) dataset or something if the download speed is an issue?) |
…up overview tract selection to skyplot, etc.
|
@timothydmorton the 1% is still too big, I'll try for a 0.1 or 0.01% dataset. |
…hboard into kartothek-integration
No description provided.