Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sdr_viirs_l1b reader fails in 0.13, recent master, Works with version 0.12.0 #692

Closed
TAlonglong opened this issue Apr 3, 2019 · 6 comments · Fixed by #693
Closed

sdr_viirs_l1b reader fails in 0.13, recent master, Works with version 0.12.0 #692

TAlonglong opened this issue Apr 3, 2019 · 6 comments · Fixed by #693
Assignees
@mraspaud
Labels
bug component:readers

Comments

@TAlonglong
Copy link
Collaborator

The sdr viirs l1b reader fails in recent master.

Test script:

#!/usr/bin/env python3                                                                                                                                                  

from satpy import Scene, find_files_and_readers
from satpy.utils import debug_on
debug_on()

files = find_files_and_readers(reader='viirs_sdr', base_dir='/data/CSPP/noaa20-viirs-sdr/20190403-050138/')
global_scene = Scene(filenames=files)
global_scene.load(['overview'])

Output from satpy-0.13.0

[DEBUG: 2019-04-03 06:58:17 : satpy.readers] Reading ['/software/pytroll/lib-satpy-grid/lib/python3.5/site-packages/satpy-0.13.0+83.ga6479be-py3.5.egg/satpy/etc/readers/viirs_sdr.yaml']
[DEBUG: 2019-04-03 06:58:17 : satpy.scene] Setting 'PPP_CONFIG_DIR' to '/software/pytroll/lib-satpy-grid/lib/python3.5/site-packages/satpy-0.13.0+83.ga6479be-py3.5.egg/satpy/etc'
[DEBUG: 2019-04-03 06:58:17 : satpy.readers] Reading ['/software/pytroll/lib-satpy-grid/lib/python3.5/site-packages/satpy-0.13.0+83.ga6479be-py3.5.egg/satpy/etc/readers/viirs_sdr.yaml']
[DEBUG: 2019-04-03 06:58:18 : satpy.readers.yaml_reader] Assigning to viirs_sdr: ['/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI03_j01_d20190403_t0505553_e0507198_b07107_c20190403051259425344_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM11_j01_d20190403_t0510108_e0511353_b07107_c20190403051251794740_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GDNBO_j01_d20190403_t0503056_e0504283_b07107_c20190403051244716616_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI04_j01_d20190403_t0505553_e0507198_b07107_c20190403051259492004_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM15_j01_d20190403_t0501398_e0503043_b07107_c20190403051301556141_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM11_j01_d20190403_t0508468_e0510095_b07107_c20190403051300551821_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM11_j01_d20190403_t0503056_e0504283_b07107_c20190403051307595895_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM13_j01_d20190403_t0503056_e0504283_b07107_c20190403051307642640_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM07_j01_d20190403_t0510108_e0511353_b07107_c20190403051251677575_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM07_j01_d20190403_t0505553_e0507198_b07107_c20190403051300398651_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM03_j01_d20190403_t0503056_e0504283_b07107_c20190403051307370911_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM11_j01_d20190403_t0507210_e0508456_b07107_c20190403051305723932_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM05_j01_d20190403_t0505553_e0507198_b07107_c20190403051259741226_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM09_j01_d20190403_t0505553_e0507198_b07107_c20190403051259862143_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GDNBO_j01_d20190403_t0507210_e0508456_b07107_c20190403051247011494_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM15_j01_d20190403_t0503056_e0504283_b07107_c20190403051307704845_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM11_j01_d20190403_t0505553_e0507198_b07107_c20190403051259917866_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI02_j01_d20190403_t0505553_e0507198_b07107_c20190403051259360858_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM13_j01_d20190403_t0510108_e0511353_b07107_c20190403051251844644_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI05_j01_d20190403_t0504295_e0505540_b07107_c20190403051304029618_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM06_j01_d20190403_t0503056_e0504283_b07107_c20190403051307464104_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI02_j01_d20190403_t0504295_e0505540_b07107_c20190403051303787382_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVDNB_j01_d20190403_t0510108_e0511353_b07107_c20190403051251108544_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM14_j01_d20190403_t0507210_e0508456_b07107_c20190403051305812249_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM16_j01_d20190403_t0504295_e0505540_b07107_c20190403051304551878_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI04_j01_d20190403_t0503056_e0504283_b07107_c20190403051307156347_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM10_j01_d20190403_t0507210_e0508456_b07107_c20190403051305697554_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM06_j01_d20190403_t0508468_e0510095_b07107_c20190403051300412808_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM05_j01_d20190403_t0503056_e0504283_b07107_c20190403051307432660_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI03_j01_d20190403_t0501398_e0503043_b07107_c20190403051300855808_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM03_j01_d20190403_t0510108_e0511353_b07107_c20190403051251585485_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM11_j01_d20190403_t0501398_e0503043_b07107_c20190403051301432082_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI03_j01_d20190403_t0510108_e0511353_b07107_c20190403051251360743_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM06_j01_d20190403_t0505553_e0507198_b07107_c20190403051259774339_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM15_j01_d20190403_t0504295_e0505540_b07107_c20190403051304523955_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM12_j01_d20190403_t0510108_e0511353_b07107_c20190403051251821430_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM10_j01_d20190403_t0501398_e0503043_b07107_c20190403051301402718_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM12_j01_d20190403_t0503056_e0504283_b07107_c20190403051307620894_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GMTCO_j01_d20190403_t0504295_e0505540_b07107_c20190403051242352100_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI03_j01_d20190403_t0503056_e0504283_b07107_c20190403051307073635_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM07_j01_d20190403_t0503056_e0504283_b07107_c20190403051307983724_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM12_j01_d20190403_t0505553_e0507198_b07107_c20190403051259945499_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM06_j01_d20190403_t0510108_e0511353_b07107_c20190403051251656625_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVDNB_j01_d20190403_t0508468_e0510095_b07107_c20190403051259897777_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM05_j01_d20190403_t0504295_e0505540_b07107_c20190403051304231460_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM03_j01_d20190403_t0501398_e0503043_b07107_c20190403051301176731_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GITCO_j01_d20190403_t0510108_e0511353_b07107_c20190403051236377179_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM07_j01_d20190403_t0508468_e0510095_b07107_c20190403051300429213_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI02_j01_d20190403_t0501398_e0503043_b07107_c20190403051300766463_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI05_j01_d20190403_t0501398_e0503043_b07107_c20190403051301031117_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVDNB_j01_d20190403_t0504295_e0505540_b07107_c20190403051303493844_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM12_j01_d20190403_t0501398_e0503043_b07107_c20190403051301461151_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM09_j01_d20190403_t0507210_e0508456_b07107_c20190403051305683314_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI05_j01_d20190403_t0510108_e0511353_b07107_c20190403051251455196_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI04_j01_d20190403_t0501398_e0503043_b07107_c20190403051300943917_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM06_j01_d20190403_t0507210_e0508456_b07107_c20190403051305608749_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM11_j01_d20190403_t0504295_e0505540_b07107_c20190403051304407002_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM07_j01_d20190403_t0507210_e0508456_b07107_c20190403051305622890_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI05_j01_d20190403_t0503056_e0504283_b07107_c20190403051307237495_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI04_j01_d20190403_t0508468_e0510095_b07107_c20190403051300154162_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GDNBO_j01_d20190403_t0505553_e0507198_b07107_c20190403051239742066_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM04_j01_d20190403_t0504295_e0505540_b07107_c20190403051304197857_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM01_j01_d20190403_t0504295_e0505540_b07107_c20190403051304110439_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM06_j01_d20190403_t0501398_e0503043_b07107_c20190403051301280768_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVDNB_j01_d20190403_t0505553_e0507198_b07107_c20190403051259055834_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM02_j01_d20190403_t0504295_e0505540_b07107_c20190403051304137593_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM10_j01_d20190403_t0505553_e0507198_b07107_c20190403051259889682_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM08_j01_d20190403_t0503056_e0504283_b07107_c20190403051307520196_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVDNB_j01_d20190403_t0507210_e0508456_b07107_c20190403051305171262_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM04_j01_d20190403_t0510108_e0511353_b07107_c20190403051251609871_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM04_j01_d20190403_t0508468_e0510095_b07107_c20190403051300382194_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM16_j01_d20190403_t0510108_e0511353_b07107_c20190403051251945105_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM15_j01_d20190403_t0508468_e0510095_b07107_c20190403051300685046_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM07_j01_d20190403_t0501398_e0503043_b07107_c20190403051301858353_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM10_j01_d20190403_t0508468_e0510095_b07107_c20190403051300508755_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM16_j01_d20190403_t0507210_e0508456_b07107_c20190403051305866022_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM14_j01_d20190403_t0504295_e0505540_b07107_c20190403051304496111_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM08_j01_d20190403_t0501398_e0503043_b07107_c20190403051301344861_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM14_j01_d20190403_t0510108_e0511353_b07107_c20190403051251889673_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM13_j01_d20190403_t0505553_e0507198_b07107_c20190403051259968643_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM09_j01_d20190403_t0503056_e0504283_b07107_c20190403051307545201_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM04_j01_d20190403_t0505553_e0507198_b07107_c20190403051259707977_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI01_j01_d20190403_t0504295_e0505540_b07107_c20190403051303704327_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GITCO_j01_d20190403_t0505553_e0507198_b07107_c20190403051239927122_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GMTCO_j01_d20190403_t0507210_e0508456_b07107_c20190403051248377440_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI01_j01_d20190403_t0508468_e0510095_b07107_c20190403051300107659_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM16_j01_d20190403_t0503056_e0504283_b07107_c20190403051307731100_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI01_j01_d20190403_t0503056_e0504283_b07107_c20190403051306907067_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI05_j01_d20190403_t0508468_e0510095_b07107_c20190403051300225702_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM01_j01_d20190403_t0507210_e0508456_b07107_c20190403051305535545_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI04_j01_d20190403_t0504295_e0505540_b07107_c20190403051303950336_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM02_j01_d20190403_t0503056_e0504283_b07107_c20190403051307345078_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM09_j01_d20190403_t0501398_e0503043_b07107_c20190403051301373985_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM13_j01_d20190403_t0504295_e0505540_b07107_c20190403051304456636_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM04_j01_d20190403_t0503056_e0504283_b07107_c20190403051307401933_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM03_j01_d20190403_t0507210_e0508456_b07107_c20190403051305562834_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVDNB_j01_d20190403_t0501398_e0503043_b07107_c20190403051300448113_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM14_j01_d20190403_t0503056_e0504283_b07107_c20190403051307678891_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI02_j01_d20190403_t0507210_e0508456_b07107_c20190403051305372970_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM05_j01_d20190403_t0510108_e0511353_b07107_c20190403051251636668_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM08_j01_d20190403_t0510108_e0511353_b07107_c20190403051251725014_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI05_j01_d20190403_t0505553_e0507198_b07107_c20190403051259556293_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GITCO_j01_d20190403_t0504295_e0505540_b07107_c20190403051241490581_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM02_j01_d20190403_t0510108_e0511353_b07107_c20190403051251562222_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM14_j01_d20190403_t0508468_e0510095_b07107_c20190403051300655294_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GDNBO_j01_d20190403_t0510108_e0511353_b07107_c20190403051236200342_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM02_j01_d20190403_t0505553_e0507198_b07107_c20190403051259647788_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM09_j01_d20190403_t0510108_e0511353_b07107_c20190403051251752383_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI01_j01_d20190403_t0510108_e0511353_b07107_c20190403051251328506_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM06_j01_d20190403_t0504295_e0505540_b07107_c20190403051304266685_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM05_j01_d20190403_t0507210_e0508456_b07107_c20190403051305595243_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM12_j01_d20190403_t0508468_e0510095_b07107_c20190403051300592248_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM02_j01_d20190403_t0501398_e0503043_b07107_c20190403051301147551_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI04_j01_d20190403_t0510108_e0511353_b07107_c20190403051251374835_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI04_j01_d20190403_t0507210_e0508456_b07107_c20190403051305403578_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GMTCO_j01_d20190403_t0501398_e0503043_b07107_c20190403051240472387_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM03_j01_d20190403_t0504295_e0505540_b07107_c20190403051304164256_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM02_j01_d20190403_t0508468_e0510095_b07107_c20190403051300345704_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM03_j01_d20190403_t0505553_e0507198_b07107_c20190403051259675005_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM14_j01_d20190403_t0505553_e0507198_b07107_c20190403051300005807_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM15_j01_d20190403_t0510108_e0511353_b07107_c20190403051251917175_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI02_j01_d20190403_t0510108_e0511353_b07107_c20190403051251346945_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI02_j01_d20190403_t0508468_e0510095_b07107_c20190403051300122269_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GDNBO_j01_d20190403_t0508468_e0510095_b07107_c20190403051243826382_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM12_j01_d20190403_t0504295_e0505540_b07107_c20190403051304433044_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM16_j01_d20190403_t0508468_e0510095_b07107_c20190403051300713898_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GITCO_j01_d20190403_t0508468_e0510095_b07107_c20190403051244010745_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GDNBO_j01_d20190403_t0504295_e0505540_b07107_c20190403051241303817_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM03_j01_d20190403_t0508468_e0510095_b07107_c20190403051300365538_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVDNB_j01_d20190403_t0503056_e0504283_b07107_c20190403051306692860_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM01_j01_d20190403_t0501398_e0503043_b07107_c20190403051301118486_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM13_j01_d20190403_t0507210_e0508456_b07107_c20190403051305774305_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM07_j01_d20190403_t0504295_e0505540_b07107_c20190403051304814003_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM13_j01_d20190403_t0508468_e0510095_b07107_c20190403051300616341_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM04_j01_d20190403_t0501398_e0503043_b07107_c20190403051301211943_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM05_j01_d20190403_t0508468_e0510095_b07107_c20190403051300397837_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM08_j01_d20190403_t0507210_e0508456_b07107_c20190403051305656929_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM15_j01_d20190403_t0507210_e0508456_b07107_c20190403051305838778_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GITCO_j01_d20190403_t0501398_e0503043_b07107_c20190403051239585732_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM12_j01_d20190403_t0507210_e0508456_b07107_c20190403051305751006_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI01_j01_d20190403_t0501398_e0503043_b07107_c20190403051300677875_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI02_j01_d20190403_t0503056_e0504283_b07107_c20190403051306990796_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM08_j01_d20190403_t0505553_e0507198_b07107_c20190403051259834866_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM01_j01_d20190403_t0508468_e0510095_b07107_c20190403051300320835_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM16_j01_d20190403_t0505553_e0507198_b07107_c20190403051300064457_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GITCO_j01_d20190403_t0507210_e0508456_b07107_c20190403051247260191_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM13_j01_d20190403_t0501398_e0503043_b07107_c20190403051301485727_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM10_j01_d20190403_t0504295_e0505540_b07107_c20190403051304380853_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GMTCO_j01_d20190403_t0508468_e0510095_b07107_c20190403051244947996_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM10_j01_d20190403_t0510108_e0511353_b07107_c20190403051251767691_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM08_j01_d20190403_t0504295_e0505540_b07107_c20190403051304327915_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM01_j01_d20190403_t0505553_e0507198_b07107_c20190403051259620487_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM09_j01_d20190403_t0504295_e0505540_b07107_c20190403051304354764_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM09_j01_d20190403_t0508468_e0510095_b07107_c20190403051300492387_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM01_j01_d20190403_t0510108_e0511353_b07107_c20190403051251538042_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM01_j01_d20190403_t0503056_e0504283_b07107_c20190403051307319597_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI01_j01_d20190403_t0507210_e0508456_b07107_c20190403051305358367_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM14_j01_d20190403_t0501398_e0503043_b07107_c20190403051301526879_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GMTCO_j01_d20190403_t0505553_e0507198_b07107_c20190403051240796141_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM10_j01_d20190403_t0503056_e0504283_b07107_c20190403051307570259_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GITCO_j01_d20190403_t0503056_e0504283_b07107_c20190403051244978504_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM02_j01_d20190403_t0507210_e0508456_b07107_c20190403051305548999_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI05_j01_d20190403_t0507210_e0508456_b07107_c20190403051305469953_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM05_j01_d20190403_t0501398_e0503043_b07107_c20190403051301246298_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM04_j01_d20190403_t0507210_e0508456_b07107_c20190403051305581523_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GMTCO_j01_d20190403_t0503056_e0504283_b07107_c20190403051246120863_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GMTCO_j01_d20190403_t0510108_e0511353_b07107_c20190403051237331161_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/GDNBO_j01_d20190403_t0501398_e0503043_b07107_c20190403051239403562_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM08_j01_d20190403_t0508468_e0510095_b07107_c20190403051300464625_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM15_j01_d20190403_t0505553_e0507198_b07107_c20190403051300034801_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI01_j01_d20190403_t0505553_e0507198_b07107_c20190403051259296242_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI03_j01_d20190403_t0504295_e0505540_b07107_c20190403051303867972_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVM16_j01_d20190403_t0501398_e0503043_b07107_c20190403051301585913_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI03_j01_d20190403_t0507210_e0508456_b07107_c20190403051305388022_cspp_dev.h5', '/data/CSPP/noaa20-viirs-sdr/20190403-050138/SVI03_j01_d20190403_t0508468_e0510095_b07107_c20190403051300138739_cspp_dev.h5']
[DEBUG: 2019-04-03 06:58:21 : satpy.composites] Looking for composites config file viirs.yaml
[DEBUG: 2019-04-03 06:58:21 : satpy.composites] Looking for composites config file visir.yaml
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.yaml_reader] No coordinates found for DatasetID(name='m_longitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] No scaling factors found for DatasetID(name='m_longitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Unknown units for file key 'DatasetID(name='m_longitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] No scaling factors found for DatasetID(name='m_longitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Unknown units for file key 'DatasetID(name='m_longitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] No scaling factors found for DatasetID(name='m_longitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Unknown units for file key 'DatasetID(name='m_longitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] No scaling factors found for DatasetID(name='m_longitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Unknown units for file key 'DatasetID(name='m_longitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] No scaling factors found for DatasetID(name='m_longitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Unknown units for file key 'DatasetID(name='m_longitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] No scaling factors found for DatasetID(name='m_longitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Unknown units for file key 'DatasetID(name='m_longitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] No scaling factors found for DatasetID(name='m_longitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Unknown units for file key 'DatasetID(name='m_longitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.yaml_reader] No coordinates found for DatasetID(name='m_latitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] No scaling factors found for DatasetID(name='m_latitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Unknown units for file key 'DatasetID(name='m_latitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] No scaling factors found for DatasetID(name='m_latitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Unknown units for file key 'DatasetID(name='m_latitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] No scaling factors found for DatasetID(name='m_latitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Unknown units for file key 'DatasetID(name='m_latitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] No scaling factors found for DatasetID(name='m_latitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Unknown units for file key 'DatasetID(name='m_latitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] No scaling factors found for DatasetID(name='m_latitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Unknown units for file key 'DatasetID(name='m_latitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] No scaling factors found for DatasetID(name='m_latitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Unknown units for file key 'DatasetID(name='m_latitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] No scaling factors found for DatasetID(name='m_latitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Unknown units for file key 'DatasetID(name='m_latitude', wavelength=None, resolution=742, polarization=None, calibration=None, level=None, modifiers=())'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Adjusting scaling factors to convert '1' to '%'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Adjusting scaling factors to convert '1' to '%'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Adjusting scaling factors to convert '1' to '%'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Adjusting scaling factors to convert '1' to '%'
[DEBUG: 2019-04-03 06:58:21 : satpy.readers.viirs_sdr] Adjusting scaling factors to convert '1' to '%'
[DEBUG: 2019-04-03 06:58:22 : satpy.readers.viirs_sdr] Adjusting scaling factors to convert '1' to '%'
[DEBUG: 2019-04-03 06:58:22 : satpy.readers.viirs_sdr] Adjusting scaling factors to convert '1' to '%'
[DEBUG: 2019-04-03 06:58:22 : satpy.readers.viirs_sdr] Adjusting scaling factors to convert '1' to '%'
[DEBUG: 2019-04-03 06:58:22 : satpy.readers.viirs_sdr] Adjusting scaling factors to convert '1' to '%'
[DEBUG: 2019-04-03 06:58:22 : satpy.readers.viirs_sdr] Adjusting scaling factors to convert '1' to '%'
[DEBUG: 2019-04-03 06:58:22 : satpy.readers.viirs_sdr] Adjusting scaling factors to convert '1' to '%'
[DEBUG: 2019-04-03 06:58:22 : satpy.readers.viirs_sdr] Adjusting scaling factors to convert '1' to '%'
Traceback (most recent call last):
  File "test-viirs.py", line 9, in <module>
    global_scene.load(['overview'])
  File "/software/pytroll/lib-satpy-grid/lib/python3.5/site-packages/satpy-0.13.0+83.ga6479be-py3.5.egg/satpy/scene.py", line 896, in load
    self.read(**kwargs)
  File "/software/pytroll/lib-satpy-grid/lib/python3.5/site-packages/satpy-0.13.0+83.ga6479be-py3.5.egg/satpy/scene.py", line 804, in read
    return self._read_datasets(nodes, **kwargs)
  File "/software/pytroll/lib-satpy-grid/lib/python3.5/site-packages/satpy-0.13.0+83.ga6479be-py3.5.egg/satpy/scene.py", line 671, in _read_datasets
    new_datasets = reader_instance.load(ds_ids, **kwargs)
  File "/software/pytroll/lib-satpy-grid/lib/python3.5/site-packages/satpy-0.13.0+83.ga6479be-py3.5.egg/satpy/readers/yaml_reader.py", line 818, in load
    ds = self._load_dataset_with_area(dsid, coords)
  File "/software/pytroll/lib-satpy-grid/lib/python3.5/site-packages/satpy-0.13.0+83.ga6479be-py3.5.egg/satpy/readers/yaml_reader.py", line 755, in _load_dataset_with_area
    ds = self._load_dataset_data(file_handlers, dsid, **slice_kwargs)
  File "/software/pytroll/lib-satpy-grid/lib/python3.5/site-packages/satpy-0.13.0+83.ga6479be-py3.5.egg/satpy/readers/yaml_reader.py", line 609, in _load_dataset_data
    proj = self._load_dataset(dsid, ds_info, file_handlers)
  File "/software/pytroll/lib-satpy-grid/lib/python3.5/site-packages/satpy-0.13.0+83.ga6479be-py3.5.egg/satpy/readers/yaml_reader.py", line 581, in _load_dataset
    projectable = fh.get_dataset(dsid, ds_info)
  File "/software/pytroll/lib-satpy-grid/lib/python3.5/site-packages/satpy-0.13.0+83.ga6479be-py3.5.egg/satpy/readers/viirs_sdr.py", line 354, in get_dataset
    data = self.scale_swath_data(data, factors)
  File "/software/pytroll/lib-satpy-grid/lib/python3.5/site-packages/satpy-0.13.0+83.ga6479be-py3.5.egg/satpy/readers/viirs_sdr.py", line 251, in scale_swath_data
    factors = xr.DataArray(da.repeat(factors, gran_size, axis=0),
  File "/software/pytroll/lib/python3.5/site-packages/dask/array/creation.py", line 737, in repeat
    return concatenate(out, axis=axis)
  File "/software/pytroll/lib/python3.5/site-packages/dask/array/core.py", line 2709, in concatenate
    ndim = len(seq[0].shape)
IndexError: list index out of range

With 0.12 it works as expected:

Environment Info:

  • OS: Linux
  • SatPy Version: 0.13 ( my fork but resent master merged in)
@TAlonglong TAlonglong changed the title sdr_viirs_l1b reader fails in recent master, Works with version 12.0 sdr_viirs_l1b reader fails in 0.13, recent master, Works with version 12.0 Apr 3, 2019
@TAlonglong TAlonglong changed the title sdr_viirs_l1b reader fails in 0.13, recent master, Works with version 12.0 sdr_viirs_l1b reader fails in 0.13, recent master, Works with version 0.12.0 Apr 3, 2019
@mraspaud
Copy link
Member

mraspaud commented Apr 3, 2019

Thanks for reporting this. I cannot reproduce this with our local data. What software and which version is used to generate these products ?

@TAlonglong
Copy link
Collaborator Author

pip list
Package Version

configobj 5.0.6
dask 1.1.5
h5py 2.9.0
numpy 1.16.2
pandas 0.24.2
Pillow 6.0.0
pip 19.0.3
pkg-resources 0.0.0
pykdtree 1.3.1
pyproj 2.1.2
pyresample 1.11.2
python-dateutil 2.8.0
pytz 2018.9
PyYAML 5.1
satpy 0.13.0
setuptools 40.8.0
six 1.12.0
toolz 0.9.0
trollimage 1.7.0
trollsift 0.3.2
wheel 0.33.1
xarray 0.11.3

@mraspaud
Copy link
Member

mraspaud commented Apr 3, 2019

How about CSPP ?

@TAlonglong
Copy link
Collaborator Author

Ah, CSPP v 3.1.2

@TAlonglong
Copy link
Collaborator Author

This seems to work when all data is in daylight. Of part of the data is in night this error seems to occur.

@mraspaud mraspaud added the bug label Apr 3, 2019
@mraspaud mraspaud self-assigned this Apr 3, 2019
@mraspaud
Copy link
Member

mraspaud commented Apr 3, 2019

Confirmed

@mraspaud mraspaud mentioned this issue Apr 3, 2019
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mraspaud mraspaud

Labels
bug component:readers
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix VIIRS SDR reading of visible channels at nighttime mraspaud/satpy
2 participants
@mraspaud @TAlonglong