Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tmp files not always removed in case of failure #83

Closed
mbattude-cs opened this issue Nov 3, 2022 · 0 comments · Fixed by #84
Closed

tmp files not always removed in case of failure #83

mbattude-cs opened this issue Nov 3, 2022 · 0 comments · Fixed by #84

Comments

@mbattude-cs
Copy link
Contributor

First run:

[2022-11-03 13:01:31,933] [world-cereal] [DEBUG ] [2022-11-03 13:01:31] CRITICAL:ewoc_l8.cli:Error during L8 ARD generation: No L8 ARD on 60KXG for ['LC08_L2SR_074072_20200827_20200923_02_T1', 'LC08_L2SR_
074071_20200827_20200923_02_T2'] !
[2022-11-03 13:01:32,164] [world-cereal] [ERROR ] Errors found in the output from command: ewoc_generate_l8_ard -v --prod-id c728b264-5c97-4f4c-81fe-1500d4c4dfbd_12036_20220926141537 --only-tir prd_ids 60
KXG LC08_L2SR_074072_20200827_20200923_02_T1 LC08_L2SR_074071_20200827_20200923_02_T2

Second run:

[2022-11-03 13:02:03,450] [world-cereal] [DEBUG ] Try number 2 (max 2). Processing product(s): LC08_L2SR_074072_20200827_20200923_02_T1 LC08_L2SR_074071_20200827_20200923_02_T2
[2022-11-03 13:02:03,450] [world-cereal] [INFO  ] Starting the processor with cmd : ewoc_generate_l8_ard -v --prod-id c728b264-5c97-4f4c-81fe-1500d4c4dfbd_12036_20220926141537 --only-tir prd_ids 60KXG LC0
8_L2SR_074072_20200827_20200923_02_T1 LC08_L2SR_074071_20200827_20200923_02_T2
[2022-11-03 13:02:03,545] [world-cereal] [INFO  ] The timeout for process end is 1800
[2022-11-03 13:02:03,546] [world-cereal] [INFO  ] Starting the watchdog for process messages timeout. Maximum time allowed for not printing any message is 0
[2022-11-03 13:02:03,546] [world-cereal] [INFO  ] Watchdog: the timeout is 0. Watching the SIGTERM signal only
[2022-11-03 13:02:05,130] [world-cereal] [INFO  ] [2022-11-03 13:02:05] INFO:ewoc_l8.l8_process:Following bands will be processed: ['B10', 'QA_PIXEL_TIR']
[2022-11-03 13:02:10,580] [world-cereal] [INFO  ] [2022-11-03 13:02:10] INFO:ewoc_l8.utils:SRS of 60KXG is EPSG:32760
[2022-11-03 13:02:10,581] [world-cereal] [INFO  ] [2022-11-03 13:02:10] INFO:ewoc_l8.utils:Bounding box of 60KXG is (600000, 8090200, 709800, 8200000)
[2022-11-03 13:02:10,582] [world-cereal] [INFO  ] [2022-11-03 13:02:10] INFO:ewoc_l8.l8_process:Processing B10 band
[2022-11-03 13:02:10,809] [world-cereal] [INFO  ] [2022-11-03 13:02:10] INFO:ewoc_dag.bucket.ewoc:EWoC bucket ewoc-ard is hosted on aws and functional
[2022-11-03 13:02:11,179] [world-cereal] [ERROR ] Errors found in the output from command: ewoc_generate_l8_ard -v --prod-id c728b264-5c97-4f4c-81fe-1500d4c4dfbd_12036_20220926141537 --only-tir prd_ids 60
KXG LC08_L2SR_074072_20200827_20200923_02_T1 LC08_L2SR_074071_20200827_20200923_02_T2
[2022-11-03 13:02:11,180] [world-cereal] [ERROR ] {{{
[2022-11-03 13:02:11,180] [world-cereal] [ERROR ] /opt/ewoc_l8_venv/lib/python3.8/site-packages/eotile/eotiles/get_bb_from_tile_id.py:87: FutureWarning: The frame.append method is deprecated and will be r
emoved from pandas in a future version. Use pandas.concat instead.
  output = output.append(tile)
Traceback (most recent call last):
  File "/opt/ewoc_l8_venv/bin/ewoc_generate_l8_ard", line 8, in <module>
    sys.exit(run())
  File "/opt/ewoc_l8_venv/lib/python3.8/site-packages/ewoc_l8/cli.py", line 273, in run
    main(sys.argv[1:])
  File "/opt/ewoc_l8_venv/lib/python3.8/site-packages/ewoc_l8/cli.py", line 237, in main
    run_id(args.l8c2l2_prd_ids,
  File "/opt/ewoc_l8_venv/lib/python3.8/site-packages/ewoc_l8/cli.py", line 106, in run_id
    process_group(
  File "/opt/ewoc_l8_venv/lib/python3.8/site-packages/ewoc_l8/l8_process.py", line 259, in process_group
    up_count, up_size, upload_path, bucket_name = process_group_band(
  File "/opt/ewoc_l8_venv/lib/python3.8/site-packages/ewoc_l8/l8_process.py", line 95, in process_group_band
    tmp_folder.mkdir(parents=True, exist_ok=False)
  File "/usr/lib/python3.8/pathlib.py", line 1288, in mkdir
    self._accessor.mkdir(self, mode)
FileExistsError: [Errno 17] File exists: '/tmp/tmp/2020-08-27/B10'

[2022-11-03 13:02:11,180] [world-cereal] [ERROR ] }}}
@mbattude-cs mbattude-cs linked a pull request Nov 3, 2022 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant