Bad redshift pileup at z=0.89218

[deisenstein]

In blanc 80608/deep, I see a set of about 32 spectra very close to z=0.89218. All with acceptable ZWARN and DELTACHI2>100. Span all spectrographs.

Fibers = [ 112 268 357 486 487 488 491 492 759 769 1037 1052 1139 1463
1547 1604 2565 2694 2716 2770 2787 2798 2803 2958 3427 3435 3843 3861
3904 4596 4749 4928]

[sbailey]

@deisenstein thanks. I see the peak you are talking about, but note that the "Fibers" you list are actually the indices of the stacked zbest files, which are not sorted by FIBER. For tiles they could be sorted by fiber, but in general in the future with coadding and redshift fitting across tiles that becomes ill-defined so I don't want people to get used to them being sorted in any particular order.

That being said, combining zbest info with fibermap info is currently a huge pain; I'm working on a helper functions to simplify that. In the meantime I'll list the TARGETIDs and FIBERs below that correspond to those indices:

targetids = [
 39633317682741492,  39633324641093584,  39633328093006968,
 616094080396427481, 616094080396427657, 616094080396427917,
 616094080396428107, 616094080396428259, 39633328080421132,
 39633328080422072, 39633328067841607, 39633328072033899,
 39633331502975523, 616094083814785918, 39633338293553508,
 39633338306134533, 39633345021218688, 39633351644021584,
 39633351644024173, 39633354915578132, 39633354915580057,
 39633354915581618, 39633354915582229, 616094103947444601,
 616094097333027737, 616094100654916002, 39633345042188801,
 39633345042190931, 39633348364078896, 39633321184986163,
 39633328097200122, 616094073480020173]
fibers = [
 259, 260,  261,  262, 263, 279, 291, 325, 752, 775, 1232, 1263, 1325, 1350,
 1740, 1850, 2742, 2743, 2747, 2753, 2754, 2756, 2759, 2800, 3238, 3239, 3764,
 3768, 3787, 4783, 4825, 4875]

Those fibers have some suspicious groupings to investigate.

[sbailey]

All of these have a strong absorption feature at 9477 that is being fit with a negative (!) [OIII] line (a known problem of overly flexible PCA), e.g.

Also, 10 of the spectra are OBJTYPE=SKY spectra; the rest are either ELG_SV_GFIB or ELG_SV_GTOT.

Archetypes, non-negative templates, or priors against negative emission lines would help prevent this on the redrock side, but I'd also like to identify and fix what is happening in the underlying spectra.

[deisenstein]

Sorry about the fiber indexing bug. I agree that negative OII flux could be flagged/voted against in redrock, but we should try in 2-d first.

…

On Sat, Jan 16, 2021 at 11:14 PM Stephen Bailey ***@***.***> wrote: All of these have a strong absorption feature at 9477 that is being fit with a negative (!) [OIII] line (a known problem of overly flexible PCA), e.g. [image: image] <https://user-images.githubusercontent.com/218471/104830726-4ee46000-5836-11eb-9a07-a6fbd4f4e749.png> Also, 10 of the spectra are OBJTYPE=SKY spectra; the rest are either ELG_SV_GFIB or ELG_SV_GTOT. Archetypes, non-negative templates, or priors against negative emission lines would help prevent this on the redrock side, but I'd also like to identify and fix what is happening in the underlying spectra. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1101 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAPQRGX5PZM43PRVEV7Q6EDS2JPY3ANCNFSM4WA5GYHQ> .

[sbailey]

Update: the underlying problem has been considerably improved by @julienguy PR #1125 improving the sky subtraction model:

However, the divot at 9477 Angstrom is the part that is getting the negative [OIII] line fit leading to the pile up at z=0.89218, and that is the feature least improved by the updated sky subtraction model.

[dmargala]

@sbailey, could the remaining spikes near 9440A, 9480A, and 9520A be related to excess ringing near the edges of the patches used during 2d extraction?

Patch boundaries for zframe wavelength range and default patch size (nwavestep=50):

zframe.wave[::50]
array([7520., 7560., 7600., 7640., 7680., 7720., 7760., 7800., 7840.,
       7880., 7920., 7960., 8000., 8040., 8080., 8120., 8160., 8200.,
       8240., 8280., 8320., 8360., 8400., 8440., 8480., 8520., 8560.,
       8600., 8640., 8680., 8720., 8760., 8800., 8840., 8880., 8920.,
       8960., 9000., 9040., 9080., 9120., 9160., 9200., 9240., 9280.,
       9320., 9360., 9400., 9440., 9480., 9520., 9560., 9600., 9640.,
       9680., 9720., 9760., 9800.])

[sbailey]

@dmargala good call. Indeed, the divot at 9477 A was due to an unfortunate interaction between an extraction patch boundary and a bright sky line. Adjusting the patch boundary parameters following DESI-6081 (my data telecon presentation from Jan 19) fixes the problem, via desihub/specter#82. Cross posting a plot from there:

[deisenstein]

Nice! Well spotted.

…

On Sun, Feb 14, 2021 at 6:43 PM Stephen Bailey ***@***.***> wrote: @dmargala <https://github.com/dmargala> good call. Indeed, the divot at 9477 A was due to an unfortunate interaction between an extraction patch boundary and a bright sky line. Adjusting the patch boundary parameters following DESI-6081 (my data telecon presentation from Jan 19) fixes the problem, via desihub/specter#82 <desihub/specter#82>. Cross posting a plot from there: <https://user-images.githubusercontent.com/218471/107892360-6b2cf880-6ed9-11eb-8c86-482cf5bfc6f0.png> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1101 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAPQRGWXETJLQYTXQBY53QLS7BNZRANCNFSM4WA5GYHQ> .

[sbailey]

Underlying problem in the spectra fixed by desispec #1125 and desihub/specter#82. Closing.