Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SOOP TMV NRT issues #434

Closed
4 tasks done
ggalibert opened this issue Jul 11, 2019 · 8 comments
Closed
4 tasks done

SOOP TMV NRT issues #434

ggalibert opened this issue Jul 11, 2019 · 8 comments
Assignees
@lbesnard
Labels
data

Comments

@ggalibert
Copy link

ggalibert commented Jul 11, 2019 •
edited by lbesnard
Loading

  • metadata record should include chlorophyl parameter.
  • link to THREDDS resource in metadata record is not correct. Should be http://thredds.aodn.org.au/thredds/catalog/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/catalog.html with l at the end.
  • some netcdf 1 sec files still have CPHL and TURB in count while metadata says engineering units: see IMOS_SOOP-TMV_TSUB_20180805T220408Z_VLST_FV00_transect-M2S_END-20180806T212840Z.nc . Data should all be in engineering units.
  • all netcdf 1 sec should include the calibration_formula, calibration_blank and calibration_scale variable attributes for TURB and CPHL with:
calibration_formula = "value_engineering_units = (counts - calibration_blank) x calibration_scale" ;
CPHL:calibration_blank = 40. ;
CPHL:calibration_scale = 0.0123 ;
TURB:calibration_blank = 50. ;
TURB:calibration_scale = 0.006 ;

See http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/transect/2014/01/IMOS_SOOP-TMV_KSTUZ_20140101T082410Z_VLST_FV02_transect-M2D_END-20140101T183810Z.nc.html as an example.
@ggalibert ggalibert added the data label Jul 11, 2019
@lbesnard
Copy link 

* some netcdf 1 sec files still have CPHL and TURB in count while metadata says engineering units: see IMOS_SOOP-TMV_TSUB_20180805T220408Z_VLST_FV00_transect-M2S_END-20180806T212840Z.nc . Data should all be in engineering units.

I don't understand why yet, but this is because those files were not reprocessed. Their date is "2019-03-07T00:10:03Z"
I need to investigate, but most likely due to the fact that maybe the log files were lost (?) and not re-pushed.

all netcdf 1 sec should include the calibration_formula, calibration_blank and calibration_scale variable attributes for TURB and CPHL with:

This is the case for ALL new realtime files re-processed in mid-June. But linked to the reason written above

@lbesnard
Copy link

affected files:

 select url from soop_tmv.indexed_file where last_indexed < '2019-06-01' and url like 'IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/%' order by url;                                                                                
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                             url                                                                             │
├─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2017/IMOS_SOOP-TMV_TSUB_20170201T184741Z_VLST_FV00_mooring-DEV_END-20170201T184741Z.nc   │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2017/IMOS_SOOP-TMV_TSUB_20170210T182231Z_VLST_FV00_mooring-DEV_END-20170210T182231Z.nc   │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2018/IMOS_SOOP-TMV_TSUB_20180806T212841Z_VLST_FV00_mooring-SYD_END-20180812T223550Z.nc   │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190226T184251Z_VLST_FV00_mooring-DEV_END-20190227T085550Z.nc   │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190227T184803Z_VLST_FV00_mooring-MEL_END-20190228T090420Z.nc   │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190228T184441Z_VLST_FV00_mooring-DEV_END-20190301T085120Z.nc   │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190301T182711Z_VLST_FV00_mooring-MEL_END-20190301T222600Z.nc   │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190302T074641Z_VLST_FV00_mooring-DEV_END-20190302T105320Z.nc   │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190302T202041Z_VLST_FV00_mooring-MEL_END-20190303T000000Z.nc   │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190303T091831Z_VLST_FV00_mooring-DEV_END-20190303T115850Z.nc   │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190303T211703Z_VLST_FV00_mooring-MEL_END-20190304T083340Z.nc   │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190304T184511Z_VLST_FV00_mooring-DEV_END-20190305T085820Z.nc   │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170201T090721Z_VLST_FV00_transect-M2D_END-20170201T090721Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170202T083511Z_VLST_FV00_transect-D2M_END-20170202T083511Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170203T082021Z_VLST_FV00_transect-M2D_END-20170203T082021Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170203T214321Z_VLST_FV00_transect-D2M_END-20170203T214321Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170204T094231Z_VLST_FV00_transect-M2D_END-20170204T094231Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170205T214411Z_VLST_FV00_transect-D2M_END-20170205T214411Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170207T085231Z_VLST_FV00_transect-D2M_END-20170207T085231Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170209T083911Z_VLST_FV00_transect-D2M_END-20170209T083911Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170210T214511Z_VLST_FV00_transect-D2M_END-20170210T214511Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170211T214511Z_VLST_FV00_transect-D2M_END-20170211T214511Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170212T215241Z_VLST_FV00_transect-D2M_END-20170212T215241Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2018/IMOS_SOOP-TMV_TSUB_20180805T220351Z_VLST_FV00_transect-M2D_END-20180805T220351Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2018/IMOS_SOOP-TMV_TSUB_20180805T220408Z_VLST_FV00_transect-M2S_END-20180806T212840Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2018/IMOS_SOOP-TMV_TSUB_20180812T223551Z_VLST_FV00_transect-S2M_END-20180813T235020Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2018/IMOS_SOOP-TMV_TSUB_20181031T182731Z_VLST_FV00_transect-M2S_END-20181031T191900Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2018/IMOS_SOOP-TMV_TSUB_20181117T194731Z_VLST_FV00_transect-M2S_END-20181117T203200Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190209T195211Z_VLST_FV00_transect-M2S_END-20190209T202130Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190226T082911Z_VLST_FV00_transect-M2D_END-20190226T184250Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190227T085551Z_VLST_FV00_transect-D2M_END-20190227T184800Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190228T090421Z_VLST_FV00_transect-M2D_END-20190228T184440Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190301T085121Z_VLST_FV00_transect-D2M_END-20190301T182710Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190301T222602Z_VLST_FV00_transect-M2D_END-20190302T074640Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190302T105321Z_VLST_FV00_transect-D2M_END-20190302T202040Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190303T000003Z_VLST_FV00_transect-M2D_END-20190303T091830Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190303T115851Z_VLST_FV00_transect-D2M_END-20190303T211700Z.nc │
│ IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190304T083341Z_VLST_FV00_transect-M2D_END-20190304T184510Z.nc │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
(38 rows)

need to find the associated log files now

@lbesnard
Copy link

Some files with a time dimension equal to one must be deleted. For the rest, I need to find the log files

ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2017/IMOS_SOOP-TMV_TSUB_20170201T184741Z_VLST_FV00_mooring-DEV_END-20170201T184741Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2017/IMOS_SOOP-TMV_TSUB_20170210T182231Z_VLST_FV00_mooring-DEV_END-20170210T182231Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2018/IMOS_SOOP-TMV_TSUB_20180806T212841Z_VLST_FV00_mooring-SYD_END-20180812T223550Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190226T184251Z_VLST_FV00_mooring-DEV_END-20190227T085550Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190227T184803Z_VLST_FV00_mooring-MEL_END-20190228T090420Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190228T184441Z_VLST_FV00_mooring-DEV_END-20190301T085120Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190301T182711Z_VLST_FV00_mooring-MEL_END-20190301T222600Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190302T074641Z_VLST_FV00_mooring-DEV_END-20190302T105320Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190302T202041Z_VLST_FV00_mooring-MEL_END-20190303T000000Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190303T091831Z_VLST_FV00_mooring-DEV_END-20190303T115850Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190303T211703Z_VLST_FV00_mooring-MEL_END-20190304T083340Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/mooring/1sec/2019/IMOS_SOOP-TMV_TSUB_20190304T184511Z_VLST_FV00_mooring-DEV_END-20190305T085820Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170201T090721Z_VLST_FV00_transect-M2D_END-20170201T090721Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170202T083511Z_VLST_FV00_transect-D2M_END-20170202T083511Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170203T082021Z_VLST_FV00_transect-M2D_END-20170203T082021Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170203T214321Z_VLST_FV00_transect-D2M_END-20170203T214321Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170204T094231Z_VLST_FV00_transect-M2D_END-20170204T094231Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170205T214411Z_VLST_FV00_transect-D2M_END-20170205T214411Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170207T085231Z_VLST_FV00_transect-D2M_END-20170207T085231Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170209T083911Z_VLST_FV00_transect-D2M_END-20170209T083911Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170210T214511Z_VLST_FV00_transect-D2M_END-20170210T214511Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170211T214511Z_VLST_FV00_transect-D2M_END-20170211T214511Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2017/IMOS_SOOP-TMV_TSUB_20170212T215241Z_VLST_FV00_transect-D2M_END-20170212T215241Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2018/IMOS_SOOP-TMV_TSUB_20180805T220351Z_VLST_FV00_transect-M2D_END-20180805T220351Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2018/IMOS_SOOP-TMV_TSUB_20180805T220408Z_VLST_FV00_transect-M2S_END-20180806T212840Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2018/IMOS_SOOP-TMV_TSUB_20180812T223551Z_VLST_FV00_transect-S2M_END-20180813T235020Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2018/IMOS_SOOP-TMV_TSUB_20181031T182731Z_VLST_FV00_transect-M2S_END-20181031T191900Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2018/IMOS_SOOP-TMV_TSUB_20181117T194731Z_VLST_FV00_transect-M2S_END-20181117T203200Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190209T195211Z_VLST_FV00_transect-M2S_END-20190209T202130Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190226T082911Z_VLST_FV00_transect-M2D_END-20190226T184250Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190227T085551Z_VLST_FV00_transect-D2M_END-20190227T184800Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190228T090421Z_VLST_FV00_transect-M2D_END-20190228T184440Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190301T085121Z_VLST_FV00_transect-D2M_END-20190301T182710Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190301T222602Z_VLST_FV00_transect-M2D_END-20190302T074640Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190302T105321Z_VLST_FV00_transect-D2M_END-20190302T202040Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190303T000003Z_VLST_FV00_transect-M2D_END-20190303T091830Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190303T115851Z_VLST_FV00_transect-D2M_END-20190303T211700Z.nc | grep "TIME = "
ncdump -h http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190304T083341Z_VLST_FV00_transect-M2D_END-20190304T184510Z.nc | grep "TIME = "

@ggalibert
Copy link
Author

all netcdf 1 sec should include the calibration_formula, calibration_blank and calibration_scale variable attributes for TURB and CPHL with:

This is the case for ALL new realtime files re-processed in mid-June. But linked to the reason written above

I haven't been able to confirm this. Please see http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/realtime/transect/1sec/2019/IMOS_SOOP-TMV_TSUB_20190620T093451Z_VLST_FV00_transect-M2D_END-20190620T194830Z.nc.html, the variable attributes mentionned above are missing for a file processed 2019-06-21T09:39:52Z

@lbesnard
Copy link

sorry my bad
fix here:
aodn/python-aodndata#147

@ggalibert ggalibert mentioned this issue Jul 16, 2019
Merged
@lbesnard
Copy link

lbesnard commented Jul 24, 2019 •
edited
Loading

The smallest count of CPHL across the current dataset is 19 (−0.4428÷0.0123)+55 = 19, so unless we use a calibration_blank < 19, we will always have negative values. but at the same time, maybe this count value is bad data anyway.

I'll change the calibration_blank to 40. Will still be an improvement

@atkinsn
Copy link

atkinsn commented Jul 25, 2019

Metadata record has been updated to include chl parameter, and THREDS link has been fixed.

@lbesnard
Copy link

all files were reprocessed over the weekend. + cleaning of files over land
as an example
http://thredds.aodn.org.au/thredds/dodsC/IMOS/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/transect/2014/04/IMOS_SOOP-TMV_KSTUZ_20140401T083740Z_VLST_FV02_transect-M2D_END-20140401T184530Z.nc.html

I still have some files to delete (see comments above)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lbesnard lbesnard

Labels
data
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@lbesnard @ggalibert @atkinsn