Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Escape regex special characters in FITS keyword values used for IFC filtering #770

Merged
merged 10 commits into from May 20, 2021
4 changes: 4 additions & 0 deletions CHANGES.rst
Expand Up @@ -10,6 +10,10 @@ Other Changes and Additions
Bug Fixes
^^^^^^^^^

- When filtering an ``ImageFileCollection`` by keyword value, and not
explicitly using a regex search pattern (``regex_match=True``), escape all
special characters in the keyword value for a successful search. [#770]

2.1.1 (2021-03-15)
------------------

Expand Down
2 changes: 2 additions & 0 deletions ccdproc/image_collection.py
Expand Up @@ -710,6 +710,8 @@ def _find_keywords_by_values(self, **kwd):
pattern = re.compile(value,
flags=re.IGNORECASE)
else:
# Escape all special characters that might be present
value = re.escape(value)
# This pattern matches the prior behavior.
pattern = re.compile('^' + value + '$',
flags=re.IGNORECASE)
Expand Down
25 changes: 25 additions & 0 deletions ccdproc/tests/test_image_collection.py
Expand Up @@ -1100,3 +1100,28 @@ def test_filtered_collection_with_no_files(self, triage_setup):
ifc = ImageFileCollection(triage_setup.test_dir)

ifc_no_files = ifc.filter(object='really fake object')

def test_filter_on_regex_escape_characters(self, triage_setup):
# Test for implementation of bugfix at
#
# https://github.com/astropy/ccdproc/issues/770
#
# which escapes regex special characters in keyword values used for
# filtering a collection, when option `regex_match=False`.

# For a few different special characters, make test files with FILTER
# keyword containing these

special_kwds = ['CO+', 'GG420 (1)', 'V|R|I', 'O[III]', 'NaD^2']
for i,kw in enumerate(special_kwds,1):
hdu = fits.PrimaryHDU()
hdu.data = np.zeros((5, 5))
hdu.header['REGEX_FL'] = kw
hdu.writeto(os.path.join(triage_setup.test_dir,
'regex_special_{:d}.fits'.format(i)))

ic = ImageFileCollection(triage_setup.test_dir)
for kw in special_kwds:
new_ic = ic.filter(regex_fl=kw)
assert len(new_ic.files) == 1
assert kw in new_ic.summary['regex_fl']