DM-21013: Add description for daf_butler expression parser #185
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
andy-slac
force-pushed
the
tickets/DM-21013
branch
4 times, most recently
from
02f7da5
to
e8a70b5
Compare
TallJimbo
approved these changes
Sep 4, 2019
doc/lsst.daf.butler/exprParser.rst
Outdated
| Butler expression language | ||
| ========================== | ||
|
|
||
| Butler registry supports user-supplied expression for constraining input, |
doc/lsst.daf.butler/exprParser.rst
Outdated
| QuantumGraph. This page describes the structure and syntax of the expression | ||
| language. | ||
|
|
||
| Language grammar is defined in ``exprParser.parserYacc`` module which is |
There was a problem hiding this comment.
"The language grammar ... module, which is"
doc/lsst.daf.butler/exprParser.rst
Outdated
| responsible for transforming string with user expression into a syntax tree | ||
| with nodes represented by various classes defined in ``exprParser.exprTree`` | ||
| module. Modules in ``exprParser`` package are considered butler/registry | ||
| implementation detail and are not exposed at the butler package level. |
doc/lsst.daf.butler/exprParser.rst
Outdated
| possible errors. The language for expression was at that point decided to be | ||
| a small subset of the expressions allowed in SQL WHERE clause. Later we | ||
| started generating SQL code from a parsed syntax tree and extended the | ||
| language with few additional constructs that are not in SQL. |
There was a problem hiding this comment.
I'm not sure the history is relevant for a user guide; I think I'd just say that the language is similar to a subset of SQL with some extensions.
| -------------------- | ||
|
|
||
| The expression is passed as a string to a registry methods that build a | ||
| QuantumGraph typically from a command-line application such as ``pipetask``. |
There was a problem hiding this comment.
It probably doesn't make sense to change this yet, as I imagine this ticket will land before DM-17023, but when DM-17023 does land, we should remember to update this paragraph (and the very first one) to reflect the fact that the expression language is used in more than just QuantumGraph generation there.
doc/lsst.daf.butler/exprParser.rst
Outdated
|
|
||
| Identifiers represent values external to a parser such as values stored in a | ||
| database. Parser itself cannot define identifiers or their values, it is | ||
| responsibility of translation layer (registry) to map identifiers into |
There was a problem hiding this comment.
"The parser itself"..."values; it is the responsibility of the translation later"
doc/lsst.daf.butler/exprParser.rst
Outdated
| Identifiers represent values external to a parser such as values stored in a | ||
| database. Parser itself cannot define identifiers or their values, it is | ||
| responsibility of translation layer (registry) to map identifiers into | ||
| something sensible. Like in most programming languages identifier starts with |
doc/lsst.daf.butler/exprParser.rst
Outdated
| where each item in the right hand side list is one of the supported literals. | ||
| Unlike regular SQL IN operator the list cannot contain expressions, only | ||
| literals. The extension to regular SQL IN is that literals can be range | ||
| literals as defined above. And it can also be a mixture of integer literals |
doc/lsst.daf.butler/exprParser.rst
Outdated
| and range literals (language allows mixing of string literals and ranges but | ||
| it may not make sense when translated to SQL). | ||
|
|
||
| For an example of range uses these two expressions are equivalent:: |
|
|
||
| Language supports set of regular comparison operators - ``=``, ``!=``, ``<``, | ||
| ``<=``, ``>``, ``>=``. This can be used on operands that evaluate to a numeric | ||
| values, for (in)equality operators operands can also be boolean expressions. |
There was a problem hiding this comment.
Maybe add a note that equality comparison is a single =, like SQL but unlike Python?
There was a problem hiding this comment.
I added this:
.. note :: The equality comparison operator is a single ``=`` like in SQL, not
double ``==`` like in Python or C++.
e8a70b5
to
1a10e32
Compare
andy-slac
force-pushed
the
tickets/DM-21013
branch
from
1a10e32
to
be6a0c6
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.