Skip to content

Commit

Permalink
Merge pull request #16 from aipescience/adql-2.1
Browse files Browse the repository at this point in the history
update to the final  ADQL 2.1
  • Loading branch information
kimakan committed May 21, 2024
2 parents 18094a0 + 1eede62 commit f320b66
Show file tree
Hide file tree
Showing 36 changed files with 568 additions and 2,488 deletions.
15 changes: 7 additions & 8 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,33 +10,32 @@ jobs:
steps:
- uses: actions/checkout@v3

- name: Set up Python 3.9
- name: Set up Python 3.11
uses: actions/setup-python@v2
with:
python-version: 3.9
python-version: 3.11

- name: Install prerequisites
run: |
sudo apt update
sudo apt install -y default-jre
python -m pip install --upgrade pip
- name: Build queryparser
run: |
wget http://www.antlr.org/download/antlr-4.11.1-complete.jar
wget http://www.antlr.org/download/antlr-4.13.1-complete.jar
make
pip install -r requirements.txt -I -e .
pip install -I -e .[test]
pip install pytest-cov
pip install coveralls
- name: Run Tests
run: |
pytest lib/ --cov=queryparser
coveralls --service=github
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}



coveralls:
name: Indicate completion to coveralls
needs: build
Expand Down
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,19 @@
## 0.7.0 (2024-05-21)

major overhaul for ADQL 2.1 recommendation 2023-12-15
- COOSYS is not required for the geometry constructors anymore, since it's deprecated
- the geometry constructors return the correct datatype (double precision[])
and correct units (degrees)
- droped the maintenance/support for the translation from ADQL to MySQL.
- bumped the version of `antlr4-python3-runtime` to 4.13.1
- fixed `BOX` constructor, although it's deprecated in ADQL 2.1
- fixed `CONTAINS` for the case `0=CONTAINS()`
- fixed `INTERSECTS` for the case `0=INTERSECTS()`
- new requirements for the `pg_sphere` extension
([link](https://github.com/kimakan/pgsphere/tree/aiprdbms16))
- removed not supported optional ADQL functions, such as `CENTROID`, `REGION`, etc.
- replaced `setup.py` by `pyproject.toml` since `python setup.py install` is deprecated

## 0.6.1 (2022-11-17)

- fixed the `ORDER BY` clause for the dot-separated column references
Expand Down
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ lib/queryparser/postgresql/%.py: src/queryparser/postgresql/%.py

lib/queryparser/testing: \
lib/queryparser/testing/__init__.py \
lib/queryparser/testing/utils.py \
lib/queryparser/testing/tests.yaml \
lib/queryparser/testing/test_adql.py \
lib/queryparser/testing/test_mysql.py \
Expand Down
68 changes: 42 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
queryparser
===========

**Tool for parsing and processing of MySQL/PostgreSQL and translation of
**Tool for parsing and processing of (MySQL\*)/PostgreSQL and translation of
ADQL SELECT-like queries**

Designed to be used in conjunction with [django-daiquri](https://github.com/django-daiquiri/daiquiri)
as a query processing backend but it can be easily used as a stand-alone tool
or integrated into another project.

**\*NOTE: Since version 0.7.0 MySQL is not supported (maintained) anymore.**


[![pytest Workflow Status](https://github.com/aipescience/queryparser/actions/workflows/pytest.yml/badge.svg)](https://github.com/aipescience/queryparser/actions/workflows/pytest.yml)
[![Coverage Status](https://coveralls.io/repos/aipescience/queryparser/badge.svg?branch=master&service=github)](https://coveralls.io/github/aipescience/queryparser?branch=master)
Expand All @@ -22,9 +24,7 @@ Installation
The easiest way to install the package is by using the pip tool:

```bash

pip install queryparser-python3

python -m pip install queryparser-python3
```

Alternatively, you can clone the repository and install it from there.
Expand All @@ -37,45 +37,62 @@ Generating the parser from the git repository

To generate the parsers you need `python3` , `java` above version
7, and `antlr4` (`antlr-4.*-complete.jar` has to be installed inside the
`/usr/local/lib/` or `/usr/local/bin/` directories).
`/usr/local/lib/`, `/usr/local/bin/` or root directory of the project).

The current version of `antlr-4.*-complete.jar` can be downloaded via

```bash
wget http://www.antlr.org/download/antlr-4.13.1-complete.jar
```

After cloning the project run

```bash
make
make
```

and a `lib` directory will be created. After that, run

```bash
python setup.py install
python -m pip install .
```

to install the generated parser in your virtual environment.


Additional requirements
-----------------------
The queryparser assumes that the PostgreSQL database has the extension
[pg_sphere](https://github.com/kimakan/pgsphere/tree/aiprdbms16) installed.
Although the `pg_sphere` is not required for the python module, the PostgreSQL
**queries will not run** without this extension installed on the database.


Parsing MySQL and PostgreSQL
----------------------------

**Since version 0.7, MySQL part of the parser is not maintained anymore.
Thus, the MySQL related functionality cannot be guaranteed!**

Parsing and processing of MySQL queries can be done by creating an instance
of the `MySQLQueryProcessor` class

```python
from queryparser.mysql import MySQLQueryProcessor
qp = MySQLQueryProcessor()
from queryparser.mysql import MySQLQueryProcessor
qp = MySQLQueryProcessor()
```

feeding it a MySQL query

```python
sql = "SELECT a FROM db.tab;"
qp.set_query(sql)
sql = "SELECT a FROM db.tab;"
qp.set_query(sql)
```

and running it with

```python
qp.process_query()
qp.process_query()
```

After the processing is completed, the processor object `qp` will include
Expand All @@ -88,8 +105,8 @@ PostgreSQL parsing is very similar to MySQL, except it requires importing
the `PostgreSQLProcessor` class:

```python
from queryparser.postgresql import PostgreSQLQueryProcessor
qp = PostgreSQLQueryProcessor()
from queryparser.postgresql import PostgreSQLQueryProcessor
qp = PostgreSQLQueryProcessor()
```

The rest of the functionality remains the same.
Expand All @@ -102,35 +119,34 @@ Translation of ADQL queries is done similarly by first creating an instance of
the `ADQLQueryTranslator` class

```python
from queryparser.adql import ADQLQueryTranslator
adql = "SELECT TOP 100 POINT('ICRS', ra, de) FROM db.tab;"
adt = ADQLQueryTranslator(adql)
from queryparser.adql import ADQLQueryTranslator
adql = "SELECT TOP 100 POINT('ICRS', ra, de) FROM db.tab;"
adt = ADQLQueryTranslator(adql)
```

and calling

```python
adt.to_mysql()
adt.to_postgresql()
```

which returns a translated string representing a valid MySQL query if
the ADQL query had no errors. The MySQL query can then be parsed with the
`MySQLQueryProcessor` in the same way as shown above.

the ADQL query had no errors. The PostgreSQL query can then be parsed with the
`PostgreSQLQueryProcessor` in the same way as shown above.

Testing
-------

First, install `pytest`
First in the root directory of the project, install optional dependencies
(`PyYAML` and `pytest`) by running

```bash
pip install pytest
python -m pip install .[test]
```

then run the test suite for a version of python you would like to test with
then run the test suite with

```bash
pytest lib/
python -m pytest lib/
```

More elaborate testing procedures can be found in the development notes.
69 changes: 0 additions & 69 deletions docs/dev-notes.md

This file was deleted.

Loading

0 comments on commit f320b66

Please sign in to comment.