Error using movefile() #432
Comments
|
1. please always send exact command you are running and the full error
message.
2. This image does not exist in new or failed...
…On Tue, Apr 9, 2024 at 5:35 PM YarinMS ***@***.***> wrote:
Pipeline runs on last02e fails when trying to move images to the failed
directory.
image.png (view on web)
<https://github.com/EranOfek/AstroPack/assets/118757319/4435efa4-9032-479a-bc51-de0c3fcb0e8a>
In debug mode the arguments to movefile() seem correct:
Source =
'/last02e/data1/archive/LAST.01.02.01/new/LAST.01.02.01_20230823.014746.906_clear_356+34_001_001_001_sci_raw_Image_1.fits'
Destination{Ifile} =
'/last02e/data1/archive/LAST.01.02.01/failed/LAST.01.02.01_20230823.014746.906_clear_356+34_001_001_001_sci_raw_Image_1.fits'
- AstroPack V2.1 is the current version on last02e.
—
Reply to this email directly, view it on GitHub
<#432>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABJUQ4JO23CHVXVTCLPXRFTY4P4C3AVCNFSM6AAAAABF6WVBY6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGIZTGNJZGY3DAOI>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
I used
|
Debug mode suggest that the pipeline moves the visit to the raw folder before compiling this line. |
|
Please always send the full error message!
As I said before I cant find these inages.
…On Tue, 9 Apr 2024 at 18:40 YarinMS ***@***.***> wrote:
- The pipeline fails in line 2546 : io.files.moveFiles(RawImageList,
FN_Sci_Groups(Igroup).genFull('FullPath',FailedPath));
Debug mode suggest that the pipeline moves the visit to the raw folder
before compiling this line.
—
Reply to this email directly, view it on GitHub
<#432 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABJUQ4NSJNUUW57HBQWPSWDY4QDVPAVCNFSM6AAAAABF6WVBY6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBVGUYDSOJUHA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
Let me pack it in one comment, by reproducing the error. % Initilize pipeline object on last02e and load calib nearJD of 22/08/23.
Note that these images could be found in the 'new' directory at the start of the run: 
Debuging the pipeline suggest that these files are moved to the 'raw' directory in line 2455 : Visually confiriming (top20) The pipeline chooses to move the images to the failed directory (from 'new' and not from 'raw'...) only later on, at line 2546, the full error is: Where As I've mentioned. You can find these images in the raw directory.
|
|
Your screenshot of the error message is truncated.
From the message, it looks like it failed in identifyBadImages.
Where these images reduced successfully in the past?
Run in debug mode and put a break point in imProc.stat.identifyBadImages
execute:
```
Image = Obj(Iim).(Args.DataProp)(Args.CCDSEC(3):Args.CCDSEC(4),
Args.CCDSEC(1):Args.CCDSEC(2));
```
1. If it fails identify the cause.
2. Are the images copied from new/ to raw/? This not suppose to happen - if
it is check which line is responsible.
…On Tue, Apr 9, 2024 at 8:37 PM YarinMS ***@***.***> wrote:
Let me pack it in one comment, by reproducing the error.
% Initilize pipeline object on last02e and load calib nearJD of 22/08/23.
D.main('StopWhenDone',true,'StartJD',[21 08 2023],'EndJD',[24 08
2023],'SaveEpochProduct',{'Cat','Mask','Image',[]})
image.png (view on web)
<https://github.com/EranOfek/AstroPack/assets/118757319/db5e6f51-d262-4ada-b28e-d16bb1dbb338>
Note that these images could be found in the 'new' directory at the start
of the run:
image.png (view on web)
<https://github.com/EranOfek/AstroPack/assets/118757319/6dabfcd0-3a0c-4ce8-9189-41902628e9f9>
Debuging the pipeline we see that these files are moved to the 'raw'
directory in line 2455 :
image.png (view on web)
<https://github.com/EranOfek/AstroPack/assets/118757319/03b3d187-e12f-4b51-a84d-64d38d7b6d66>
Visually confiriming (top20)
/last02e/data1/archive/LAST.01.02.01/2023/08/22/raw$:
image.png (view on web)
<https://github.com/EranOfek/AstroPack/assets/118757319/6d54d23c-4d79-4e2b-a717-c1586bf93018>
The pipeline chooses to move the images to the failed directory (from
'new' and not from 'raw'...) only later on, at line 2546, the full error is:
image.png (view on web)
<https://github.com/EranOfek/AstroPack/assets/118757319/0702adee-a07e-4913-b6c1-8a3e7aaf46d5>
Where io.files.moveFiles tries to move the images from new -> failed :
image.png (view on web)
<https://github.com/EranOfek/AstroPack/assets/118757319/c5c48075-5a3b-478f-bd55-25df5262ff97>
As I've mentioned. You can find these images in the raw directory.
- Not sure if movefile() should change the source to the raw directory
instead of the new or there is something deeper I'm not aware of.
—
Reply to this email directly, view it on GitHub
<#432 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABJUQ4NT2IVL5KF5DABH4SLY4QRPBAVCNFSM6AAAAABF6WVBY6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBVG42TQNBSHA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
The images reduced successfully in the last Iteration
|
|
The problem is likely due to the following: If this is the problem, the the simplest solution will be to create a new new/ dir - e.g., call it newYS |
|
The solution I suggested is not good enough - because the automatic process will copy the proc images to marvin over the existing v0 version... |
Pipeline runs on last02e fails when trying to move images to the failed directory.
In debug mode the arguments to movefile() seem correct:
Source = '/last02e/data1/archive/LAST.01.02.01/new/LAST.01.02.01_20230823.014746.906_clear_356+34_001_001_001_sci_raw_Image_1.fits'Destination{Ifile} = '/last02e/data1/archive/LAST.01.02.01/failed/LAST.01.02.01_20230823.014746.906_clear_356+34_001_001_001_sci_raw_Image_1.fits'The text was updated successfully, but these errors were encountered: