DM-22173: Implement parsing of time strings in user expression #245
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
timj
approved these changes
Mar 31, 2020
doc/lsst.daf.butler/queries.rst
Outdated
| `astropy.time.Time`_ class and parser converts time string into an instance | ||
| of that class. For string-based time formats such as ISO the conversion | ||
| of a time string to an object is done by the ``Time`` constructor. The syntax | ||
| of the string could be anyhting that is suported by ``astropy``, for details |
doc/lsst.daf.butler/queries.rst
Outdated
| Time literals | ||
| ^^^^^^^^^^^^^ | ||
|
|
||
| Timestaps in a query language are specified using syntax ``T'time-string'``. |
| if fmt is not None: | ||
| fmt = fmt.lower() | ||
| if fmt not in astropy.time.Time.FORMATS: | ||
| raise ValueError(f"Time string \"{time_str}\" specifies unknown time format \"{fmt}\"") |
There was a problem hiding this comment.
We raise early rather than waiting for astropy because we don't like the astropy exception?
There was a problem hiding this comment.
I do it to make more specific message about format, guessing that format is wrong from astropy exception would be difficult.
There was a problem hiding this comment.
Interesting. Bad format:
>>> t.Time(123456.43, format="blah")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.7/site-packages/astropy/time/core.py", line 429, in __init__
precision, in_subfmt, out_subfmt)
File "/Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.7/site-packages/astropy/time/core.py", line 484, in _init_from_vals
precision, in_subfmt, out_subfmt)
File "/Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.7/site-packages/astropy/time/core.py", line 526, in _get_time_fmt
sorted(self.FORMATS)))
ValueError: Format 'blah' is not one of the allowed formats ['byear', 'byear_str', 'cxcsec', 'datetime', 'datetime64', 'decimalyear', 'fits', 'gps', 'iso', 'isot', 'jd', 'jyear', 'jyear_str', 'mjd', 'plot_date', 'stardate', 'unix', 'yday', 'ymdhms']
but bad scale:
>>> t.Time(123456.43, scale="xc")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.7/site-packages/astropy/time/core.py", line 429, in __init__
precision, in_subfmt, out_subfmt)
File "/Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.7/site-packages/astropy/time/core.py", line 476, in _init_from_vals
sorted(self.SCALES)))
astropy.time.core.ScaleValueError: Scale 'xc' is not in the allowed scales ['local', 'tai', 'tcb', 'tcg', 'tdb', 'tt', 'ut1', 'utc']
So they have a special exception for bad time scale but not for bad format.
| @@ -256,6 +344,15 @@ def p_literal_str(self, p): | |||
| """ | |||
| p[0] = StringLiteral(p[1]) | |||
|
|
|||
| def p_literal_time(self, p): | |||
| """ literal : TIME_LITERAL | |||
There was a problem hiding this comment.
This docstring looks odd but I see it matches the others.
There was a problem hiding this comment.
ply uses docstrings to define grammar rules, so it does look different from LSST standard.
tests/test_exprParserLex.py
Outdated
| """Test for time literals""" | ||
| lexer = ParserLex.make_lexer() | ||
|
|
||
| # string can contain anyhting, lexer does not check it |
TallJimbo
approved these changes
Apr 1, 2020
andy-slac
force-pushed
the
tickets/DM-22173
branch
from
43234ae
to
11e885e
Compare
Time literal was added to a parser in the form `T'string'`, string can be one of the supported formats for astropy Time class plus optional format and scale. Some parsing is done in the parser code to determine string format and convert it to floating number if needed. For string-based representations (iso, yday, etc.) it relies on astropy parsing.
andy-slac
force-pushed
the
tickets/DM-22173
branch
from
11e885e
to
912d19d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Time literal was added to a parser in the form
T'string', string canbe one of the supported formats for astropy Time class plus optional
format and scale. Some parsing is done in the parser code to determine
string format and convert it to floating number if needed. For
string-based representations (iso, yday, etc.) it relies on astropy
parsing.
Also fixes an issue in QueryBuilder class which had issues with support of
dotted identifiers.
Documentation for parser is updated with description of supported time
literal syntax.