Add entry in filename_parser for segmented guider files

[bhilbert4]

Resolves #961
This PR adds code that allows the filename_parser to understand Guider segment files. These files are used in cases where the amount of Guider data is above some threshold, in which case it is split into multiple files.

[pep8speaks]

Hello @bhilbert4, Thank you for updating !

• In the file jwql/utils/utils.py:

Line 41:34: E126 continuation line over-indented for hanging indent
Line 456:21: E117 over-indented
Line 459:5: E303 too many blank lines (2)
Line 559:86: W504 line break after binary operator
Line 560:79: W504 line break after binary operator
Line 566:80: W504 line break after binary operator
Line 567:79: W504 line break after binary operator

Comment last updated at 2022-06-24 18:59:06 UTC

[bhilbert4]

Local test shows the parser working correctly. Just need to test on a server using the search field on the home page to see if any other changes are needed in support of the filenames.

[bhilbert4]

Looks like 1118 has lots of challenging files:
Provided file jw01118008001_01_msa.fits does not follow JWST naming conventions

[bhilbert4]

I also added code to be able to parse filenames such as jw01118008001_01_msa.fits, which originally failed above. This is an MSA metadata file.

[bhilbert4]

@mfixstsci and/or @BradleySappington I think this is ready for review. I tested it on the dev server by attempting to retrieve program 1118 using the "Find a JWST Proposal or File" field on the main page. Previously this program was causing the website to crash because of the filename parser error. Now it loads successfully....with a message saying that it cannot return data from proposals that use more than 1 instrument.

[mfixstsci]

@mfixstsci and/or @BradleySappington I think this is ready for review. I tested it on the dev server by attempting to retrieve program 1118 using the "Find a JWST Proposal or File" field on the main page. Previously this program was causing the website to crash because of the filename parser error. Now it loads successfully....with a message saying that it cannot return data from proposals that use more than 1 instrument.

Just tried it myself and it works as expected. Do we know of any multi-instrument programs we can test?

[bhilbert4]

See #901. It looks at the moment like all programs are considered multi-instrument because they all use FGS for guide stars. That's a separate issue to this, and one we should try to complete before the next release. In the meantime, it might take some searching to find a single-instrument program that also has a segmented FGS file in it. And it won't be possible to find a program that has only nirspec and an MSA metadata file, since FGS will also be used in all of those.

[bhilbert4]

I also added tests for these two new possibilities in the filename_parser

[BradleySappington]

Looks good @bhilbert4
Could potentially store all constants as a dictionary to make import easier, but that would also create more lines as you'd have to extract the values before putting into the regex to keep it from getting too muddled. A-OK to keep as is :)

[bhilbert4]

Looks good @bhilbert4 Could potentially store all constants as a dictionary to make import easier, but that would also create more lines as you'd have to extract the values before putting into the regex to keep it from getting too muddled. A-OK to keep as is :)

I agree. I started to implement this as a dictionary, but that made readability in filename_parser() even worse.

[BradleySappington]

Yeah, you’d have to assign the “variables” from the dictionary to make it legible, it would essentially just move the massive import line to a massive dictionary dump line. Keep it as is I say. From: Bryan Hilbert ***@***.***> Reply-To: spacetelescope/jwql ***@***.***> Date: Friday, June 24, 2022 at 2:58 PM To: spacetelescope/jwql ***@***.***> Cc: Bradley Sappington ***@***.***>, Mention ***@***.***> Subject: Re: [spacetelescope/jwql] Add entry in filename_parser for segmented guider files (PR #962) Looks good @bhilbert4<https://urldefense.com/v3/__https:/github.com/bhilbert4__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!x8aBl0UMYrgWE5MNLwipxwU7HDdGYdklROIv0iSqTqMG7WoQC07iUUbhVOWfWoZ-kuDT_bdgvhcUR3n8KOpkykJPONRT$> Could potentially store all constants as a dictionary to make import easier, but that would also create more lines as you'd have to extract the values before putting into the regex to keep it from getting too muddled. A-OK to keep as is :) I agree. I started to implement this as a dictionary, but that made readability in filename_parser() even worse. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/spacetelescope/jwql/pull/962*issuecomment-1165849448__;Iw!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!x8aBl0UMYrgWE5MNLwipxwU7HDdGYdklROIv0iSqTqMG7WoQC07iUUbhVOWfWoZ-kuDT_bdgvhcUR3n8KOpkyse4t0-u$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AYEBMN5YNYV22DWBNH73SLLVQYAOJANCNFSM5XWBXS7A__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!x8aBl0UMYrgWE5MNLwipxwU7HDdGYdklROIv0iSqTqMG7WoQC07iUUbhVOWfWoZ-kuDT_bdgvhcUR3n8KOpkykHTnaj8$>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[mfixstsci]

Looks good @bhilbert4 Could potentially store all constants as a dictionary to make import easier, but that would also create more lines as you'd have to extract the values before putting into the regex to keep it from getting too muddled. A-OK to keep as is :)

Agreed, I think it could be done this way and not cost much, but I think that is something we could improve on if we need to import those constants anywhere else :)

[bhilbert4]

Woo hoo! Merging! Thanks for the review!