Skip to content

Commit

Permalink
Reply to reviewer comments.
Browse files Browse the repository at this point in the history
Add TOTFlux explicitly.

Hook up TOTFlux output.

Fix linting.
  • Loading branch information
morriscb committed Mar 19, 2019
1 parent 9d0c77d commit 89c5465
Show file tree
Hide file tree
Showing 3 changed files with 181 additions and 31 deletions.
133 changes: 120 additions & 13 deletions data/ppdb-ap-pipe-schema-extra.yaml
Expand Up @@ -17,7 +17,114 @@ columns:
type: INT
nullable: false
description: Total number of DiaSources associated with this DiaObject.

- name: uTOTFluxMean
type: FLOAT
nullable: true
description: Weighted mean forced photometry flux for u filter.
ucd: phot.count
unit: nJy
- name: uTOTFluxMeanErr
type: FLOAT
nullable: true
description: Standard error of uTOTFluxMean.
ucd: stat.error
unit: nJy
- name: uTOTFluxSigma
type: FLOAT
nullable: true
description: Standard deviation of the distribution of uTOTFlux.
ucd: stat.stdev
unit: nJy
- name: gTOTFluxMean
type: FLOAT
nullable: true
description: Weighted mean forced photometry flux for g filter.
ucd: phot.count
unit: nJy
- name: gTOTFluxMeanErr
type: FLOAT
nullable: true
description: Standard error of gTOTFluxMean.
ucd: stat.error
unit: nJy
- name: gTOTFluxSigma
type: FLOAT
nullable: true
description: Standard deviation of the distribution of gTOTFlux.
ucd: stat.stdev
unit: nJy
- name: rTOTFluxMean
type: FLOAT
nullable: true
description: Weighted mean forced photometry flux for r filter.
ucd: phot.count
unit: nJy
- name: rTOTFluxMeanErr
type: FLOAT
nullable: true
description: Standard error of rTOTFluxMean.
ucd: stat.error
unit: nJy
- name: rTOTFluxSigma
type: FLOAT
nullable: true
description: Standard deviation of the distribution of rTOTFlux.
ucd: stat.stdev
unit: nJy
- name: iTOTFluxMean
type: FLOAT
nullable: true
description: Weighted mean forced photometry flux for i filter.
ucd: phot.count
unit: nJy
- name: iTOTFluxMeanErr
type: FLOAT
nullable: true
description: Standard error of iTOTFluxMean.
ucd: stat.error
unit: nJy
- name: iTOTFluxSigma
type: FLOAT
nullable: true
description: Standard deviation of the distribution of iTOTFlux.
ucd: stat.stdev
unit: nJy
- name: zTOTFluxMean
type: FLOAT
nullable: true
description: Weighted mean forced photometry flux for z filter.
ucd: phot.count
unit: nJy
- name: zTOTFluxMeanErr
type: FLOAT
nullable: true
description: Standard error of zTOTFluxMean.
ucd: stat.error
unit: nJy
- name: zTOTFluxSigma
type: FLOAT
nullable: true
description: Standard deviation of the distribution of zTOTFlux.
ucd: stat.stdev
unit: nJy
- name: yTOTFluxMean
type: FLOAT
nullable: true
description: Weighted mean forced photometry flux for y filter.
ucd: phot.count
unit: nJy
- name: yTOTFluxMeanErr
type: FLOAT
nullable: true
description: Standard error of yTOTFluxMean.
ucd: stat.error
unit: nJy
- name: yTOTFluxSigma
type: FLOAT
nullable: true
description: Standard deviation of the distribution of yTOTFlux.
ucd: stat.stdev
unit: nJy
- name: uPSFluxMedian
type: FLOAT
nullable: true
Expand Down Expand Up @@ -70,7 +177,7 @@ columns:
type: FLOAT
nullable: true
description: Linear best fit slope of the u band fluxes.
unit: nJy/MJD
unit: nJy/day
- name: uPSFluxLinearIntercept
type: FLOAT
nullable: true
Expand All @@ -79,7 +186,7 @@ columns:
type: FLOAT
nullable: true
description: Maximum slope between u band flux obsevations max(delta_flux/delta_time)
unit: nJy/MJD
unit: nJy/day
- name: uPSFluxErrMean
type: FLOAT
nullable: true
Expand Down Expand Up @@ -138,7 +245,7 @@ columns:
type: FLOAT
nullable: true
description: Linear best fit slope of the g band fluxes.
unit: nJy/MJD
unit: nJy/day
- name: gPSFluxLinearIntercept
type: FLOAT
nullable: true
Expand All @@ -148,7 +255,7 @@ columns:
type: FLOAT
nullable: true
description: Maximum slope between g band flux obsevations max(delta_flux/delta_time)
unit: nJy/MJD
unit: nJy/day
- name: gPSFluxErrMean
type: FLOAT
nullable: true
Expand Down Expand Up @@ -207,7 +314,7 @@ columns:
type: FLOAT
nullable: true
description: Linear best fit slope of the r band fluxes.
unit: nJy/MJD
unit: nJy/day
- name: rPSFluxLinearIntercept
type: FLOAT
nullable: true
Expand All @@ -217,7 +324,7 @@ columns:
type: FLOAT
nullable: true
description: Maximum slope between r band flux obsevations max(delta_flux/delta_time)
unit: nJy/MJD
unit: nJy/day
- name: rPSFluxErrMean
type: FLOAT
nullable: true
Expand Down Expand Up @@ -276,7 +383,7 @@ columns:
type: FLOAT
nullable: true
description: Linear best fit slope of the i band fluxes.
unit: nJy/MJD
unit: nJy/day
- name: iPSFluxLinearIntercept
type: FLOAT
nullable: true
Expand All @@ -286,7 +393,7 @@ columns:
type: FLOAT
nullable: true
description: Maximum slope between i band flux obsevations max(delta_flux/delta_time)
unit: nJy/MJD
unit: nJy/day
- name: iPSFluxErrMean
type: FLOAT
nullable: true
Expand Down Expand Up @@ -345,7 +452,7 @@ columns:
type: FLOAT
nullable: true
description: Linear best fit slope of the z band fluxes.
unit: nJy/MJD
unit: nJy/day
- name: zPSFluxLinearIntercept
type: FLOAT
nullable: true
Expand All @@ -355,7 +462,7 @@ columns:
type: FLOAT
nullable: true
description: Maximum slope between z band flux obsevations max(delta_flux/delta_time)
unit: nJy/MJD
unit: nJy/day
- name: zPSFluxErrMean
type: FLOAT
nullable: true
Expand Down Expand Up @@ -414,7 +521,7 @@ columns:
type: FLOAT
nullable: true
description: Linear best fit slope of the y band fluxes.
unit: nJy/MJD
unit: nJy/day
- name: yPSFluxLinearIntercept
type: FLOAT
nullable: true
Expand All @@ -424,7 +531,7 @@ columns:
type: FLOAT
nullable: true
description: Maximum slope between y band flux obsevations max(delta_flux/delta_time)
unit: nJy/MJD
unit: nJy/day
- name: yPSFluxErrMean
type: FLOAT
nullable: true
Expand Down
39 changes: 36 additions & 3 deletions python/lsst/ap/association/afwUtils.py
Expand Up @@ -104,7 +104,6 @@ def make_dia_object_schema(filter_names=None):
doc='Standard error of uPSFluxMean.')
schema.addField('uPSFluxSigma', type='F',
doc='Standard deviation of the distribution of uPSFlux.')

schema.addField('uPSFluxChi2', type='F',
doc='Chi^2 statistic for the scatter of uPSFlux around uPSFluxMean.')
schema.addField('uPSFluxNdata', type='I',
Expand All @@ -121,7 +120,6 @@ def make_dia_object_schema(filter_names=None):
doc='Standard error of gPSFluxMean.')
schema.addField('gPSFluxSigma', type='F',
doc='Standard deviation of the distribution of gPSFlux.')

schema.addField('gPSFluxChi2', type='F',
doc='Chi^2 statistic for the scatter of gPSFlux around gPSFluxMean.')
schema.addField('gPSFluxNdata', type='I',
Expand Down Expand Up @@ -263,7 +261,42 @@ def make_dia_object_schema(filter_names=None):
doc='Last time when non-forced DIASource was seen for this object')
schema.addField('nDiaSources', type='I',
doc='Total number of DiaSources associated with this DiaObject.')

schema.addField('uTOTFluxMean', type='F',
doc='Weighted mean forced photometry flux for u filter.')
schema.addField('uTOTFluxMeanErr', type='F',
doc='Standard error of uTOTFluxMean.')
schema.addField('uTOTFluxSigma', type='F',
doc='Standard deviation of the distribution of uTOTFlux.')
schema.addField('gTOTFluxMean', type='F',
doc='Weighted mean forced photometry flux for g filter.')
schema.addField('gTOTFluxMeanErr', type='F',
doc='Standard error of uTOTFluxMean.')
schema.addField('gTOTFluxSigma', type='F',
doc='Standard deviation of the distribution of gTOTFlux.')
schema.addField('rTOTFluxMean', type='F',
doc='Weighted mean forced photometry flux for r filter.')
schema.addField('rTOTFluxMeanErr', type='F',
doc='Standard error of uTOTFluxMean.')
schema.addField('rTOTFluxSigma', type='F',
doc='Standard deviation of the distribution of rTOTFlux.')
schema.addField('iTOTFluxMean', type='F',
doc='Weighted mean forced photometry flux for i filter.')
schema.addField('iTOTFluxMeanErr', type='F',
doc='Standard error of uTOTFluxMean.')
schema.addField('iTOTFluxSigma', type='F',
doc='Standard deviation of the distribution of iTOTFlux.')
schema.addField('zTOTFluxMean', type='F',
doc='Weighted mean forced photometry flux for z filter.')
schema.addField('zTOTFluxMeanErr', type='F',
doc='Standard error of uTOTFluxMean.')
schema.addField('zTOTFluxSigma', type='F',
doc='Standard deviation of the distribution of zTOTFlux.')
schema.addField('yTOTFluxMean', type='F',
doc='Weighted mean forced photometry flux for y filter.')
schema.addField('yTOTFluxMeanErr', type='F',
doc='Standard error of uTOTFluxMean.')
schema.addField('yTOTFluxSigma', type='F',
doc='Standard deviation of the distribution of yTOTFlux.')
schema.addField('uPSFluxMedian', type='F',
doc='Median u band flux.')
schema.addField('uPSFluxMAD', type='F',
Expand Down
40 changes: 25 additions & 15 deletions python/lsst/ap/association/association.py
Expand Up @@ -129,7 +129,7 @@ def _set_flux_stats(dia_object_record, dia_sources, filter_name, filter_id):
dia_object_record['%sPSFluxLinearSlope' % filter_name] = m
dia_object_record['%sPSFluxLinearIntercept' % filter_name] = b

dia_object_record['%sPSFluxStetsonJ' % filter_name] = _stentson_J(
dia_object_record['%sPSFluxStetsonJ' % filter_name] = _stetson_J(
fluxes, fluxErrors)

dia_object_record['%sPSFluxErrMean' % filter_name] = \
Expand All @@ -143,14 +143,14 @@ def _set_flux_stats(dia_object_record, dia_sources, filter_name, filter_id):
totFluxErrors = totFluxErrors[noNanMask]

if len(totFluxes) == 1:
dia_object_record['%sFPFluxMean' % filter_name] = totFluxes
dia_object_record['%sTOTFluxMean' % filter_name] = totFluxes
elif len(totFluxes) > 1:
fluxMean = np.average(totFluxes, weights=1 / totFluxErrors ** 2)
dia_object_record['%sFPFluxMean' % filter_name] = fluxMean
dia_object_record['%sFPFluxMeanErr' % filter_name] = np.sqrt(
dia_object_record['%sTOTFluxMean' % filter_name] = fluxMean
dia_object_record['%sTOTFluxMeanErr' % filter_name] = np.sqrt(
1 / np.sum(1 / totFluxErrors ** 2))
dia_object_record['%sFPFluxSigma' % filter_name] = np.std(totFluxes,
ddof=1)
dia_object_record['%sTOTFluxSigma' % filter_name] = np.std(totFluxes,
ddof=1)


def _fit_linear_flux_model(fluxes, errors, times):
Expand All @@ -177,8 +177,8 @@ def model(x):
return ans


def _stentson_J(fluxes, errors):
"""Compute the single band StentsonJ statistic.
def _stetson_J(fluxes, errors):
"""Compute the single band stetsonJ statistic.
Parameters
----------
Expand All @@ -189,23 +189,28 @@ def _stentson_J(fluxes, errors):
Returns
-------
stentsonJ : `float`
StentsonJ statistic for the input fluxes and errors.
stetsonJ : `float`
stetsonJ statistic for the input fluxes and errors.
References
----------
.. [1] Stetson, P. B., "On the Automatic Determination of Light-Curve
Parameters for Cepheid Variables", PASP, 108, 851S, 1996
"""
n_points = len(fluxes)
flux_mean = _stentson_mean(fluxes, errors)
flux_mean = _stetson_mean(fluxes, errors)
delta_val = (
np.sqrt(n_points / (n_points - 1)) * (fluxes - flux_mean) / errors)
p_k = delta_val ** 2 - 1

return np.mean(np.sign(p_k) * np.sqrt(np.fabs(p_k)))


def _stentson_mean(values, errors, alpha=2., beta=2., n_iter=20, tol=1e-6):
"""Compute the Stentson mean of the fluxes which down-weights outliers.
def _stetson_mean(values, errors, alpha=2., beta=2., n_iter=20, tol=1e-6):
"""Compute the stetson mean of the fluxes which down-weights outliers.
Weighted biased on an error weighted difference scaled by a constant
(1/``a``) and raised to the power beta. Higher beta's more harshly penalize
(1/``a``) and raised to the power beta. Higher betas more harshly penalize
outliers and ``a`` sets the number of sigma where a weighted difference of
1 occurs.
Expand All @@ -229,7 +234,12 @@ def _stentson_mean(values, errors, alpha=2., beta=2., n_iter=20, tol=1e-6):
Returns
-------
wmean : `float`
Weighted Stentson mean result.
Weighted stetson mean result.
References
----------
.. [1] Stetson, P. B., "On the Automatic Determination of Light-Curve
Parameters for Cepheid Variables", PASP, 108, 851S, 1996
"""
n_points = len(values)
n_factor = np.sqrt(n_points / (n_points - 1))
Expand Down

0 comments on commit 89c5465

Please sign in to comment.