"Extension ('WAVELENGTH', 1) not found." #430

kcrasmus · 2022-07-27T23:27:01Z

Instrument

NIRSpec (Stages 1-3)

What happened?

I am running

s3_spec, s3_meta = s3.reduce(eventlabel, ecf_path=ecf_path,
s2_meta=s2_meta)

at which point it fails to find the "Wavelength" extension. S2 is executing normally, and I did go in and fix the issue the code suggested I fix with the bgerr for issue 193.

I've included the full output for clarity.

Error traceback output

S3_error.txt

What operating system are you using?

Mac OSX

What version of Python are you running?

3.9

What Python packages do you have installed?

(eureka) rasmussen@Kaitlins-MacBook-Pro demos % conda list

packages in environment at /Users/rasmussen/opt/anaconda3/envs/eureka:

Code of Conduct

I agree to follow this project's Code of Conduct

taylorbell57 · 2022-07-27T23:28:48Z

Can you confirm that you did not set skip_extract1d to True in Stage 2?

kcrasmus · 2022-07-27T23:35:07Z

hmm... I did have it set to true, but now I'm getting this error: Traceback (most recent call last): File "/Users/rasmussen/Research/WORKING_Eureka/demos/run_JN_eureka.py", line 33, in <module> s2_meta = s2.calibrateJWST(eventlabel, ecf_path=ecf_path) File "/Users/rasmussen/opt/anaconda3/envs/eureka/lib/python3.9/site-packages/eureka/S2_calibrations/s2_calibrate.py", line 159, in calibrateJWST pipeline.run_eurekaS2(filename, meta, log) File "/Users/rasmussen/opt/anaconda3/envs/eureka/lib/python3.9/site-packages/eureka/S2_calibrations/s2_calibrate.py", line 252, in run_eurekaS2 self(filename) File "/Users/rasmussen/opt/anaconda3/envs/eureka/lib/python3.9/site-packages/stpipe/step.py", line 483, in run self.prefetch(*args) File "/Users/rasmussen/opt/anaconda3/envs/eureka/lib/python3.9/site-packages/stpipe/step.py", line 600, in prefetch self._precache_references(args[0]) File "/Users/rasmussen/opt/anaconda3/envs/eureka/lib/python3.9/site-packages/stpipe/pipeline.py", line 312, in _precache_references self._precache_references_opened(model) File "/Users/rasmussen/opt/anaconda3/envs/eureka/lib/python3.9/site-packages/stpipe/pipeline.py", line 335, in _precache_references_opened self._precache_references_impl(model_or_container) File "/Users/rasmussen/opt/anaconda3/envs/eureka/lib/python3.9/site-packages/stpipe/pipeline.py", line 359, in _precache_references_impl crds_refs = crds_client.get_multiple_reference_paths(model.get_crds_parameters(), fetch_types, model.crds_observatory) File "/Users/rasmussen/opt/anaconda3/envs/eureka/lib/python3.9/site-packages/stpipe/crds_client.py", line 55, in get_multiple_reference_paths refpaths = _get_refpaths(parameters, tuple(reference_file_types), observatory) File "/Users/rasmussen/opt/anaconda3/envs/eureka/lib/python3.9/site-packages/stpipe/crds_client.py", line 68, in _get_refpaths bestrefs = crds.getreferences( File "/Users/rasmussen/opt/anaconda3/envs/eureka/lib/python3.9/site-packages/crds/core/heavy_client.py", line 127, in getreferences best_refs_paths = api.cache_references( File "/Users/rasmussen/opt/anaconda3/envs/eureka/lib/python3.9/site-packages/crds/client/api.py", line 848, in cache_references wanted = _get_cache_filelist_and_report_errors(bestrefs) File "/Users/rasmussen/opt/anaconda3/envs/eureka/lib/python3.9/site-packages/crds/client/api.py", line 891, in _get_cache_filelist_and_report_errors raise last_error crds.core.exceptions.CrdsLookupError: Error determining best reference for 'apcorr' = parameter='META.EXPOSURE.TYPE' value='NRS_WATA' is not in ['NRS_AUTOWAVE', 'NRS_BRIGHTOBJ', 'NRS_FIXEDSLIT', 'NRS_IFU', 'NRS_LAMP', 'NRS_MSASPEC'] Thanks, kaitlin

…

On Wed, Jul 27, 2022 at 4:29 PM Taylor James Bell ***@***.***> wrote: Can you confirm that you did not set skip_extract1d to True in Stage 2? — Reply to this email directly, view it on GitHub <#430 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A2E4CUFZLYTGAIAIRPWGHMDVWHA3VANCNFSM543KHSPQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

taylorbell57 · 2022-07-27T23:40:21Z

Alright, yeah frustratingly we do need to run the extract1d step for reasons I don't fully understand. Not sure about this new S2 error, but typically things like that are driven by a bad jwst package version. Anymore insight @AarynnCarter?

kcrasmus · 2022-07-27T23:51:52Z

Hello, It might be my jwst install—how can I test to make sure it's up and running? I get a zsh: no matches found: .[jwst] message when I try and do the pip install. I have an M1 machine so I did have to go the conda route .yml route installing. Thanks, kaitlin

…

On Wed, Jul 27, 2022 at 4:40 PM Taylor James Bell ***@***.***> wrote: Alright, yeah frustratingly we do need to run the extract1d step for reasons I don't fully understand. Not sure about this new S2 error, but typically things like that are driven by a bad jwst package version. Anymore insight @AarynnCarter <https://github.com/AarynnCarter>? — Reply to this email directly, view it on GitHub <#430 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A2E4CUBIGP5KCVDE7ZVSDGDVWHCHBANCNFSM543KHSPQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

taylorbell57 · 2022-07-27T23:57:43Z

For zsh you need to make sure you put the quotation marks that are in the documentation (pip install ".[jwst]"). But I can see in your list of installed packages that you do have jwst==1.6.2 installed. I think that's a bit newer than what's in the environment.yml file and setup.cfg file, but typically I've had similar issues when using too old a version, not too new. Maybe try doing pip install --upgrade jwst?

AarynnCarter · 2022-07-28T00:36:32Z

@taylorbell57 Do we need to run the extract1d step? I don't think I ever did for the ERS PRISM data because it takes too long...

taylorbell57 · 2022-07-28T02:35:35Z

We definitely shouldn't need to run extract1d, but for some reason some datasets end up with no wavelength array of you don't.....

kcrasmus · 2022-07-28T03:31:05Z

Hey, I figured out the issue. I had two files which apparently weren't data, which is why they had weird exposure types. Folks with this issue should check and make sure their downloaded data is contains the extension "NRS_FIXEDSLIT'" Cheers, Kaitlin

…

On Wed, Jul 27, 2022 at 7:35 PM Taylor James Bell ***@***.***> wrote: We definitely shouldn't need to run extract1d, but for some reason some datasets end up with no wavelength array of you don't..... — Reply to this email directly, view it on GitHub <#430 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A2E4CUBCWCTNTWQDHESS2SLVWHWYDANCNFSM543KHSPQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

kcrasmus · 2022-10-11T09:04:24Z

Gotcha. Yeah, upgrading jwst didn't change the "Error determining best reference for 'apcorr' = parameter='META.EXPOSURE.TYPE' value='NRS_WATA' is not in ['NRS_AUTOWAVE', 'NRS_BRIGHTOBJ', 'NRS_FIXEDSLIT', 'NRS_IFU', 'NRS_LAMP', 'NRS_MSASPEC']" error I'm getting. thanks, Kaitlin

…

On Wed, Jul 27, 2022 at 4:57 PM Taylor James Bell ***@***.***> wrote: For zsh you need to make sure you put the quotation marks that are in the documentation (pip install ".[jwst]"). But I can see in your list of installed packages that you do have jwst==1.6.2 installed. I think that's a bit newer than what's in the environment.yml file and setup.cfg file, but typically I've had similar issues when using too old a version, not too new. Maybe try doing pip install --upgrade jwst? — Reply to this email directly, view it on GitHub <#430 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A2E4CUFA3ISWHQNATMCHT2DVWHEIFANCNFSM543KHSPQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

AarynnCarter · 2022-10-11T18:47:05Z

Hey Kaitlin,

Just coming back to this, are you still having issues or just providing an update? Now that I look closer, NRS_WATA is signalling that this is a target acquisition observation (Wide Aperture Target Acquisition), so upgrading the jwst package won't fix anything. You just need to make sure you only have science observations in the files you're using (which it looks like you already figured out...)

@taylorbell57 If anything we should have catch somewhere to ignore files that aren't science / treat them differently.

kcrasmus · 2022-10-11T18:49:02Z

Hello, Thanks for following up, we had Eureka installed on our local computing cluster and it runs just fine over there. no more M1 issues. Cheers, Kaitlin

…

On Tue, Oct 11, 2022 at 11:47 AM Aarynn Carter ***@***.***> wrote: Hey Kaitlin, Just coming back to this, are you still having issues or just providing an update? Now that I look closer, NRS_WATA is signalling that this is a target acquisition observation (Wide Aperture Target Acquisition), so upgrading the jwst package won't fix anything. You just need to make sure you only have science observations in the files you're using (which it looks like you already figured out...) — Reply to this email directly, view it on GitHub [github.com] <https://urldefense.com/v3/__https://github.com/kevin218/Eureka/issues/430*issuecomment-1275127074__;Iw!!K-Hz7m0Vt54!jtVKj-GRJ11ebIZdgfrgS5_KFQYqnrVRTIv80RTyBKHZNVehyMZgqJ_AP5OS_OKLwiEmDlk9iXuhxcvW9EMTZWQ$>, or unsubscribe [github.com] <https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/A2E4CUB7GZE3DDEWF4KA2C3WCWY3JANCNFSM543KHSPQ__;!!K-Hz7m0Vt54!jtVKj-GRJ11ebIZdgfrgS5_KFQYqnrVRTIv80RTyBKHZNVehyMZgqJ_AP5OS_OKLwiEmDlk9iXuhxcvWF0otW9s$> . You are receiving this because you authored the thread.Message ID: ***@***.***>

github-actions bot added the NIRSpec label Jul 27, 2022

github-actions bot assigned AarynnCarter Jul 27, 2022

taylorbell57 closed this as completed Jul 28, 2022

taylorbell57 mentioned this issue Oct 11, 2022

[Enhancement]: Ignore non-science files #473

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"Extension ('WAVELENGTH', 1) not found." #430

"Extension ('WAVELENGTH', 1) not found." #430

kcrasmus commented Jul 27, 2022

taylorbell57 commented Jul 27, 2022

kcrasmus commented Jul 27, 2022 via email

taylorbell57 commented Jul 27, 2022

kcrasmus commented Jul 27, 2022 via email

taylorbell57 commented Jul 27, 2022

AarynnCarter commented Jul 28, 2022

taylorbell57 commented Jul 28, 2022

kcrasmus commented Jul 28, 2022 via email

kcrasmus commented Oct 11, 2022 via email

AarynnCarter commented Oct 11, 2022 •

edited

kcrasmus commented Oct 11, 2022 via email

"Extension ('WAVELENGTH', 1) not found." #430

"Extension ('WAVELENGTH', 1) not found." #430

Comments

kcrasmus commented Jul 27, 2022

Instrument

What happened?

Error traceback output

What operating system are you using?

What version of Python are you running?

What Python packages do you have installed?

packages in environment at /Users/rasmussen/opt/anaconda3/envs/eureka:

Name Version Build Channel

Code of Conduct

taylorbell57 commented Jul 27, 2022

kcrasmus commented Jul 27, 2022 via email

taylorbell57 commented Jul 27, 2022

kcrasmus commented Jul 27, 2022 via email

taylorbell57 commented Jul 27, 2022

AarynnCarter commented Jul 28, 2022

taylorbell57 commented Jul 28, 2022

kcrasmus commented Jul 28, 2022 via email

kcrasmus commented Oct 11, 2022 via email

AarynnCarter commented Oct 11, 2022 • edited

kcrasmus commented Oct 11, 2022 via email

AarynnCarter commented Oct 11, 2022 •

edited