Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Incident report: GFMS Flood_byStor_2022103103.bin incomplete data #38

Closed
webgisdeveloper opened this issue Oct 31, 2022 · 8 comments
Closed

Incident report: GFMS Flood_byStor_2022103103.bin incomplete data #38

webgisdeveloper opened this issue Oct 31, 2022 · 8 comments
Assignees
@webgisdeveloper
Labels
fixed fixed issues incident something wrong from the data source

Comments

@webgisdeveloper
Copy link
Contributor 

Traceback (most recent call last):
  File "rasterio/_io.pyx", line 701, in rasterio._io.DatasetReaderBase._read
  File "rasterio/shim_rasterioex.pxi", line 162, in rasterio._shim.io_multi_band
  File "rasterio/_err.pyx", line 192, in rasterio._err.exc_wrap_int
rasterio._err.CPLE_AppDefinedError: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.vrt, band 1: IReadBlock failed at X offset 9, Y offset 12: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.bin, band 1: IReadBlock failed at X offset 0, Y offset 794: Failed to read scanline 794.
@webgisdeveloper webgisdeveloper added the incident something wrong from the data source label Oct 31, 2022
@webgisdeveloper webgisdeveloper self-assigned this Oct 31, 2022
@webgisdeveloper
Copy link
Contributor Author 

Traceback (most recent call last):
  File "rasterio/_io.pyx", line 701, in rasterio._io.DatasetReaderBase._read
  File "rasterio/shim_rasterioex.pxi", line 162, in rasterio._shim.io_multi_band
  File "rasterio/_err.pyx", line 192, in rasterio._err.exc_wrap_int
rasterio._err.CPLE_AppDefinedError: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.vrt, band 1: IReadBlock failed at X offset 9, Y offset 12: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.bin, band 1: IReadBlock failed at X offset 0, Y offset 791: Failed to read scanline 791.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/geogateway/MoMProduction/HWRF_MoM.py", line 708, in <module>
    main()   
  File "/home/geogateway/MoMProduction/HWRF_MoM.py", line 705, in main
    batchrun_HWRF_MoM()
  File "/home/geogateway/MoMProduction/HWRF_MoM.py", line 693, in batchrun_HWRF_MoM
    update_HWRF_MoM(testdate)
  File "/home/geogateway/MoMProduction/HWRF_MoM.py", line 62, in update_HWRF_MoM
    GFMS_fixdate(adate[:8])
  File "/home/geogateway/MoMProduction/GFMS_tool.py", line 469, in GFMS_fixdate
    GFMS_processing(processing_dates)
  File "/home/geogateway/MoMProduction/GFMS_tool.py", line 392, in GFMS_processing
    GFMS_data_extractor(bin_file)
  File "/home/geogateway/MoMProduction/GFMS_tool.py", line 341, in GFMS_data_extractor
    GFMS_extract_by_watershed(vrt_file)
  File "/home/geogateway/MoMProduction/GFMS_tool.py", line 308, in GFMS_extract_by_watershed
    data_points = GFMS_extract_by_mask(vrt_file, test_json)
  File "/home/geogateway/MoMProduction/GFMS_tool.py", line 236, in GFMS_extract_by_mask
    out_image, out_transform = mask(src, [mask_json['features'][0]['geometry']], crop=True)
  File "/home/geogateway/miniconda3/envs/mom/lib/python3.9/site-packages/rasterio/mask.py", line 189, in mask
    out_image = dataset.read(
  File "rasterio/_io.pyx", line 367, in rasterio._io.DatasetReaderBase.read
  File "rasterio/_io.pyx", line 704, in rasterio._io.DatasetReaderBase._read
rasterio.errors.RasterioIOError: Read or write failed. /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.vrt, band 1: IReadBlock failed at X offset 9, Y offset 12: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.bin, band 1: IReadBlock failed at X offset 0, Y offset 791: Failed to read scanline 791.

@webgisdeveloper
Copy link
Contributor Author

Data for debug:
2022103103.zip

Flood_byStor_2022103103.csv Flood_byStor_2022103103.vrt
Flood_byStor_2022103103.bin Flood_byStor_2022103103.hdr

webgisdeveloper added a commit that referenced this issue Oct 31, 2022
@webgisdeveloper
#38 fix gfms scanlin issue
5604ecf
webgisdeveloper added a commit that referenced this issue Oct 31, 2022
@webgisdeveloper
#38 clean up
5b8b62b
@webgisdeveloper
Copy link
Contributor Author

webgisdeveloper commented Oct 31, 2022 •
edited
Loading

error when generate gfms tiff image

Input file size is 4916, 1600
0...10...20...30...40...50...60...70...80..ERROR 3: Failed to read scanline 791.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.bin, band 1: IReadBlock failed at X offset 0, Y offset 791: Failed to read scanline 791.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.vrt, band 1: IReadBlock failed at X offset 0, Y offset 12: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103103.bin, band 1: IReadBlock failed at X offset 0, Y offset 791: Failed to read scanline 791.
Input file size is 4916, 1600

@webgisdeveloper
Copy link
Contributor Author

09 is also failed

0...10...20...30...40.ERROR 3: Failed to read scanline 418.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103109.bin, band 1: IReadBlock failed at X offset 0, Y offset 418: Failed to read scanline 418.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103109.vrt, band 1: IReadBlock failed at X offset 0, Y offset 6: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103109.bin, band 1: IReadBlock failed at X offset 0, Y offset 418: Failed to read scanline 418.

@webgisdeveloper
Copy link
Contributor Author

12 is also failed

0...10...20...30...40.ERROR 3: Failed to read scanline 419.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103112.bin, band 1: IReadBlock failed at X offset 0, Y offset 419: Failed to read scanline 419.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103112.vrt, band 1: IReadBlock failed at X offset 0, Y offset 6: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103112.bin, band 1: IReadBlock failed at X offset 0, Y offset 419: Failed to read scanline 419.

@webgisdeveloper
Copy link
Contributor Author

15 is also failed

0...10...20...30...40.ERROR 3: Failed to read scanline 408.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103115.bin, band 1: IReadBlock failed at X offset 0, Y offset 408: Failed to read scanline 408.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103115.vrt, band 1: IReadBlock failed at X offset 0, Y offset 6: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103115.bin, band 1: IReadBlock failed at X offset 0, Y offset 408: Failed to read scanline 408.

@webgisdeveloper
Copy link
Contributor Author

18

0...10...20...30...40.ERROR 3: Failed to read scanline 409.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103118.bin, band 1: IReadBlock failed at X offset 0, Y offset 409: Failed to read scanline 409.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103118.vrt, band 1: IReadBlock failed at X offset 0, Y offset 6: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103118.bin, band 1: IReadBlock failed at X offset 0, Y offset 409: Failed to read scanline 409.

@webgisdeveloper
Copy link
Contributor Author

21

Input file size is 4916, 1600
0...10...20...30...40.ERROR 3: Failed to read scanline 407.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103121.bin, band 1: IReadBlock failed at X offset 0, Y offset 407: Failed to read scanline 407.
ERROR 1: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103121.vrt, band 1: IReadBlock failed at X offset 0, Y offset 6: /home/geogateway/MoMraw/Processing/gfms/Flood_byStor_2022103121.bin, band 1: IReadBlock failed at X offset 0, Y offset 407: Failed to read scanline 407.

@webgisdeveloper webgisdeveloper added the fixed fixed issues label Nov 1, 2022
@webgisdeveloper webgisdeveloper mentioned this issue Nov 1, 2022
Open
This was referenced Nov 9, 2022
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@webgisdeveloper webgisdeveloper

Labels
fixed fixed issues incident something wrong from the data source
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@webgisdeveloper