Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(oracle): port to sqlglot #8020

Merged
merged 68 commits into from
Jan 22, 2024
Merged
Show file tree
Hide file tree
Changes from 66 commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
346caa2
test(oracle): fix up conftest for oracledb direct usage
gforsyth Jan 8, 2024
59cbfd1
refactor(oracle): add compiler and port backend
gforsyth Jan 8, 2024
fbc15d0
feat(oracle): add type mapper, converter, update compiler
gforsyth Jan 9, 2024
73f7b78
feat(oracle): add rewrites for log2 and log10
gforsyth Jan 9, 2024
8a55ae7
feat(oracle): cleanup
gforsyth Jan 9, 2024
875c4fe
fix(oracle): infinities, degrees, radians, errors
gforsyth Jan 9, 2024
5e4bcbf
feat(oracle): port to sqlglot (numeric)
gforsyth Jan 9, 2024
d6d2b1c
feat(oracle): sqlglot strings
gforsyth Jan 9, 2024
2d751b3
feat(oracle): memtables
gforsyth Jan 10, 2024
1da4715
fix(oracle): ensure column names are quoted
gforsyth Jan 10, 2024
86bcfc7
fix(oracle): smol stuff
gforsyth Jan 10, 2024
814ac79
fix(oracle): set up initial agg stuff
gforsyth Jan 10, 2024
e4a64cf
test(oracle): swap oracledb error for sqlalchemy error
gforsyth Jan 10, 2024
26aaf87
fix(oracle): handle date with cast
gforsyth Jan 10, 2024
91041a3
fix(oracle): handle sqlglot rewrites locally
gforsyth Jan 11, 2024
bf9c527
fix(oracle): use group names, not numbers
gforsyth Jan 11, 2024
cb09a88
feat(oracle): count distinct, median, quantile
gforsyth Jan 11, 2024
8d20054
feat(oracle): approx median
gforsyth Jan 11, 2024
b69e27e
feat(oracle): countdistinct and countstar
gforsyth Jan 12, 2024
91c5c85
feat(oracle): use oracle positional bind parameters
gforsyth Jan 16, 2024
50e991a
fix(oracle): add max size to varchar2 defs
gforsyth Jan 16, 2024
e6635ba
fix(oracle): test fixups in asof join and numerics
gforsyth Jan 16, 2024
d95aaf4
fix(oracle): remove oracle from alchemycon
gforsyth Jan 16, 2024
37d8299
fix(oracle): fix client ddl stuff
gforsyth Jan 16, 2024
5e68c18
fix(oracle): test generic
gforsyth Jan 16, 2024
2642d3a
feat(oracle): is identical to
gforsyth Jan 16, 2024
767eab3
feat(oracle): xor
gforsyth Jan 16, 2024
ba3c483
fix(oracle): dont case nulls
gforsyth Jan 16, 2024
b68c3d4
fix(oracle): handle subquery n in limit, no support for offset yet
gforsyth Jan 16, 2024
818d69e
test(oracle): remove XPASSing xfails
gforsyth Jan 17, 2024
6fbcaa6
fix(oracle): temporal xfails
gforsyth Jan 17, 2024
d163bfe
fix(oracle): window functions
gforsyth Jan 17, 2024
685b112
fix(oracle): handle interval casting
gforsyth Jan 18, 2024
90b006d
fix(oracle): parse RAW to dt.binary
gforsyth Jan 18, 2024
4840a97
fix(oracle): add hierarchical schema support to get schema
gforsyth Jan 18, 2024
9876534
test(oracle): remove XPASSing xfails
gforsyth Jan 18, 2024
cdac4f4
fix(oracle): uuid literal and upper schema
gforsyth Jan 18, 2024
d58c422
fix(oracle): rowid not defined
gforsyth Jan 18, 2024
b1b1ea1
feat(oracle): rewrite anti and semi joins
gforsyth Jan 18, 2024
6b3cdf1
fix(oracle): no op for extract day/week of year
gforsyth Jan 18, 2024
ab699d3
refactor(oracle): more test fixups
gforsyth Jan 18, 2024
e224545
chore(oracle): update snapshots
gforsyth Jan 18, 2024
b34cd79
refactor(oracle): remove cruft
gforsyth Jan 18, 2024
3675b78
feat(oracle): regex extract
gforsyth Jan 18, 2024
0f80107
feat(oracle): regex replace
gforsyth Jan 18, 2024
246dddf
ci(oracle): re-enable CI for oracle
gforsyth Jan 18, 2024
7a48f77
chore(oracle): clean up converter
gforsyth Jan 18, 2024
ca21420
feat(oracle): datetruncate
gforsyth Jan 19, 2024
9a496e7
feat(oracle): add timestamp truncation
gforsyth Jan 19, 2024
67e0483
chore(oracle): remove extraneous marks from test param
gforsyth Jan 19, 2024
5a26093
chore(oracle): update xfail reason for persist release
gforsyth Jan 19, 2024
7a7396a
fix(oracle): handle scale parameter in timestamp with time zone
gforsyth Jan 19, 2024
53ce9b5
test(oracle): add snapshots and followup notes
gforsyth Jan 19, 2024
4b835ac
fix(oracle): xfail dynamic offsets
gforsyth Jan 19, 2024
3876632
fix(oracle): xfail datediff for now
gforsyth Jan 19, 2024
84c5a00
fix(oracle): disable dynamic limits
gforsyth Jan 19, 2024
ad0016b
fix(oracle): raise if table not found in get_schema
gforsyth Jan 19, 2024
b91b58a
fix(oracle): horrible hack for boolean scalar parameter
gforsyth Jan 22, 2024
27d1af7
chore(oracle): xfail pyarrow decimal export
gforsyth Jan 22, 2024
677ad23
fix(oracle): restore xfail for con_dot_sql
gforsyth Jan 22, 2024
255928c
fix(oracle): hack for isNull on fake booleans
gforsyth Jan 22, 2024
ca39ffd
chore(oracle): xfail pyarrow decimal export
gforsyth Jan 22, 2024
c0650a7
fix(oracle): set oracledb to return decimals as decimals
gforsyth Jan 22, 2024
167ef36
fix(oracle): deal with dtype stuff from oracle defaults
gforsyth Jan 22, 2024
ad411e6
chore(exasol): fix test markers
gforsyth Jan 22, 2024
307cd5f
chore(tests): remove defunct error raise check
gforsyth Jan 22, 2024
78cab6d
fix(tests): filter `None` from tuples of `raises`
gforsyth Jan 22, 2024
70291fa
test(druid): remove xpassing xfail
gforsyth Jan 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
32 changes: 16 additions & 16 deletions .github/workflows/ibis-backends.yml
Original file line number Diff line number Diff line change
Expand Up @@ -155,20 +155,20 @@ jobs:
- druid
services:
- druid
# - name: oracle
# title: Oracle
# serial: true
# extras:
# - oracle
# services:
# - oracle
- name: exasol
title: Exasol
serial: true
extras:
- exasol
services:
- exasol
- name: oracle
title: Oracle
serial: true
extras:
- oracle
services:
- oracle
# - name: flink
# title: Flink
# serial: true
Expand Down Expand Up @@ -265,15 +265,15 @@ jobs:
- druid
services:
- druid
# - os: windows-latest
# backend:
# name: oracle
# title: Oracle
# serial: true
# extras:
# - oracle
# services:
# - oracle
- os: windows-latest
backend:
name: oracle
title: Oracle
serial: true
extras:
- oracle
services:
- oracle
# - os: windows-latest
# backend:
# name: flink
Expand Down
23 changes: 23 additions & 0 deletions ibis/backends/base/sqlglot/datatypes.py
Original file line number Diff line number Diff line change
Expand Up @@ -563,6 +563,29 @@
class OracleType(SqlglotType):
dialect = "oracle"

default_decimal_precision = 38
default_decimal_scale = 9

default_temporal_scale = 9

unknown_type_strings = FrozenDict({"raw": dt.binary})

@classmethod
def _from_sqlglot_FLOAT(cls) -> dt.Float64:
return dt.Float64(nullable=cls.default_nullable)

@classmethod
def _from_sqlglot_DECIMAL(cls, precision=None, scale=None) -> dt.Decimal:
if scale is None or int(scale.this.this) == 0:
return dt.Int64(nullable=cls.default_nullable)
else:
return super()._from_sqlglot_DECIMAL(precision, scale)

Check warning on line 582 in ibis/backends/base/sqlglot/datatypes.py

View check run for this annotation

Codecov / codecov/patch

ibis/backends/base/sqlglot/datatypes.py#L582

Added line #L582 was not covered by tests

@classmethod
def _from_ibis_String(cls, dtype: dt.String) -> sge.DataType:
nullable = " NOT NULL" if not dtype.nullable else ""
return "VARCHAR2(4000)" + nullable


class SnowflakeType(SqlglotType):
dialect = "snowflake"
Expand Down
5 changes: 1 addition & 4 deletions ibis/backends/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -549,10 +549,7 @@ def ddl_con(ddl_backend):
return ddl_backend.connection


@pytest.fixture(
params=_get_backends_to_test(keep=("mssql", "oracle", "sqlite")),
scope="session",
)
@pytest.fixture(params=_get_backends_to_test(keep=("mssql", "sqlite")), scope="session")
def alchemy_backend(request, data_dir, tmp_path_factory, worker_id):
"""Set up the SQLAlchemy-based backends."""
return _setup_backend(request, data_dir, tmp_path_factory, worker_id)
Expand Down