Update conda recipe for ibis-framework 0.4.0
Test conda packages for Ibis 0.4.0 are available on osx-64, linux-64, and win-64 by running: ``` conda install -c koverholt ibis- framework ``` Author: Kristopher Overholt <koverholt@gmail.com> Closes #537 from koverholt/update-conda-recipes and squashes the following commits: 5e89884 [Kristopher Overholt] Update conda recipe for ibis-framework 0.4.0
ENH: Add UDA support/testing and iterate on Impala UDF wrapping API
Close #535, #290 Author: Wes McKinney <wes@cloudera.com> Closes #559 from wesm/feature/impala-udaf-support and squashes the following commits: 1c4efae [Wes McKinney] Test list_udas eef227c [Wes McKinney] Get dropping UDAs and force-dropping databases with UDAs to work 5c4a4f7 [Wes McKinney] Get count UDA working and make it possible to omit all but update_fn baed46b [Wes McKinney] Remove sample UDAs relying on Impala < 2.0 stuff 2d12e31 [Wes McKinney] Working on UDA test but discovered sample UDA code is stale b455ac7 [Wes McKinney] Get rid of pytest udf mark c27fde7 [Wes McKinney] Refactor UDF classes and simplify UDF creation workflow to hide more internal machinery 6f90f7d [Wes McKinney] Refactoring pass over UDF code; remove to_operation step in workflow
Start building Ibis API for MADLib-on-Impala
Several follow-on tasks are necessary to get this completely working with UDA invocation / iteration tools. I have a branch going in github.com/cloudera/madlibport to consolidate all the MADLib functions into a single shared library to simplify deployment and configuration by users. Some API changes to UDF wrapping workflow and type validation. Part of #337. Closes #558 Author: Wes McKinney <wes@cloudera.com> Closes #564 from wesm/feature/madlib-support and squashes the following commits: d9041cb [Wes McKinney] Setup and teardown of ToArray function, test suite passing again 71ce2dc [Wes McKinney] Preliminary support for varargs UDFs 7d2f5dd [Wes McKinney] Consolidate UDF creation code. Verify all currently wrapped madlib functions can be created in impala d2bded4 [Wes McKinney] Start building MADLib-on-Impala UDF/UDA Ibis wrapper
Closes #297. Also per #473, #332, #274, #252 Author: Wes McKinney <wes@cloudera.com> Author: Uri Laserson <laserson@cloudera.com> Author: Matthieu Monsch <monsch@alum.mit.edu> Closes #568 from wesm/refactor/python3-support and squashes the following commits: 4e9c182 [Wes McKinney] More gracefully handle HS2 errors when closing session on connection that was closed elsewhere 7cc464d [Uri Laserson] Some py3 changes to cloudpickle.py 1f3c0b8 [Wes McKinney] Python 2 bugfix ceb2fe0 [Wes McKinney] Remaining py3 fixes. Lingering HS2 __del__ failure 6305c2a [Wes McKinney] More Python 3 fixes 9361c7b [Wes McKinney] All but impala e2e tests now pass dbd3deb [Matthieu Monsch] Upgrade WebHDFS client to HdfsCLI 2.0.
BUG: Fix case producing duplicated WHERE in SQL translation
Close #539 and fix formatting bugs #507 and also unreported bug with sort keys. Author: Wes McKinney <wes@cloudera.com> Closes #574 from wesm/bug/duplicate-where-predicate and squashes the following commits: 321cbf1 [Wes McKinney] Fix duplicated where bug case in SQL translation #539 and expand Column type display #507 2ec72e5 [Wes McKinney] Make sort keys expressions and fix repr bug resulting in duplicated output
ENH: Add a rename method to ImpalaTable, wrapping ALTER TABLE DDL
ENH: add a default backend option so that non-backend-dependent exprs…
… work in interactive mode. Close #484
BUG: fix test_data_admin.py table creation
Also makes test_create_table_with_location() require superuser privileges. Fixes #575.
SQLAlchemy translator backend and preliminary SQLite client
Essentially refactoring to compartmentalize SQLAlchemy-based translator backends from homegrown ones (like the Impala translator). More code cleaning and reorganization to maximize reuse. This doesn't have complete support for SQLite SQL keywords and built-in functions yet. I will make some follow up issues to organize the process and address in follow-up pull requests. Closes #584 This also paves the road to Postgres / Vertica / Redshift / MySQL support in the near future. Author: Wes McKinney <wes@cloudera.com> Closes #585 from wesm/feature/sqlite-support and squashes the following commits: 5a4f7b5 [Wes McKinney] Enable literal expressions with no FROM clause to work with SQLA 9b3a98f [Wes McKinney] Add a couple sqlite type cast tests 2ef06f7 [Wes McKinney] More impala-sqlalchemy separation code refactoring / cleaning decd66e [Wes McKinney] More code sharing with AlchemyClient 652cdcb [Wes McKinney] Add list_tables support for sqlalchemy generically 064f398 [Wes McKinney] Get expression execution working on SQLite 68c6d1b [Wes McKinney] Refactoring to make 'dialects' more pluggable 916f15b [Wes McKinney] Compare compiled/inlined sqla strings 2ffca2b [Wes McKinney] Add a hack to generate exists queries versus select queries in AlchemySelect 73b81da [Wes McKinney] Port exists case, but not yet working d6c0375 [Wes McKinney] Get correlated subquery test working and add context level walking logic bdf0585 [Wes McKinney] Check / fix uncorrelated subquery case 39b0255 [Wes McKinney] Implement and test a CTE and self-reference case ba0ae51 [Wes McKinney] Test subquery factoring in limit case 64b23d6 [Wes McKinney] Limit impl, but subqueries not yet working 3ed693e [Wes McKinney] Implement order_by for SQLA abaac0c [Wes McKinney] Get simple group_by and having statements working c60b6b4 [Wes McKinney] Basic where impl 118fc55 [Wes McKinney] Refactor some SELECT test cases to make more reusable 33d61d3 [Wes McKinney] Test left and outer joins, too ffbbec1 [Wes McKinney] Handle select * case in select set formatting 9437950 [Wes McKinney] Form join expressions, other translation scaffolding f4a3004 [Wes McKinney] Hackishly check named expr c5268b3 [Wes McKinney] And/or tests 446b796 [Wes McKinney] Subclassing more sql translation machinery for sqlalchemy. Get table column expr working 3ba15ad [Wes McKinney] More refactoring and scaffolding. Consolidate sql.context into sql.compiler ce41dcd [Wes McKinney] Add SQLiteTable subclass 64cbdd6 [Wes McKinney] Add hdfs pytest mark 1f617d8 [Wes McKinney] Get first SQLite tests working. Add --only-X flags for only running certain test groups e0f9fd4 [Wes McKinney] More sqlite scaffolding / code reorg 05f3cb2 [Wes McKinney] Chunk conversion to SQLite. Let casted exprs be named 930c78d [Wes McKinney] Working impala -> sqlite code 0f36774 [Wes McKinney] Test scaffolding 622d30b [Wes McKinney] Various refactoring to get closer to being able to target sqlalchemy 9eeb387 [Wes McKinney] Add nullable flag to data types and start writing sqlalchemy schema adapting code
TST: Added coverage computation on jenkins runs
Coverage data gets reported to codecov.io/github/cloudera/ibis Fixes #598.
ENH: Add SQLite built-in function coverage
Adds translation rules for most SQLite built-in non-esoteric functions. Closes #596. Author: Wes McKinney <wes@cloudera.com> Closes #601 from wesm/feature/sqlite-functions and squashes the following commits: 2a6e970 [Wes McKinney] Implement ibis.now using SQLite datetime builtin 99202bf [Wes McKinney] Add millisecond accessor 73200e8 [Wes McKinney] Add strftime, year, month, day, hour, minute, second 43d7174 [Wes McKinney] Casting string or integer to timestamp in SQLite is a no-op 9f1db48 [Wes McKinney] Add StringValue.replace and SQLite unit test d7b361c [Wes McKinney] Add NYI regexp test for sqlite3 2fe4a8a [Wes McKinney] Add sqla round impl a8708d7 [Wes McKinney] Add typeof value API and sqlite impl/test bd983d2 [Wes McKinney] Add nullif translation rule 47f46eb [Wes McKinney] Add ifnull translation rule 72600ec [Wes McKinney] Add failing union unit test f127abd [Wes McKinney] Implement abs, coalesce, min (least), and max (greatest) 7710a49 [Wes McKinney] Implement isnull and notnull 223258f [Wes McKinney] Implement negate using not_ a408529 [Wes McKinney] Implement StringValue.like/contains using LIKE 2be4302 [Wes McKinney] Implement StringValue.find using instr e3a9d2a [Wes McKinney] Implement a handful of string translation functions e92d9b5 [Wes McKinney] Add min/max/group_concat aggregate functions and some binary arithmetic tests
TST: Add SQLite to Jenkins setup
Fixes #613 Author: Wes McKinney <wes@cloudera.com> Closes #618 from wesm/testing/sqlite-jenkins and squashes the following commits: 3081e8f [Wes McKinney] Use tarball name and not full path baba444 [Wes McKinney] Do not require aws keys b83c3ed [Wes McKinney] TST: add sqlite database to data load and S3 tarball. Move tarball to a different S3 bucket
ENH: table distinct and count distinct support for sqlalchemy/sqlite
Closes #614 Author: Wes McKinney <wes@cloudera.com> Closes #623 from wesm/feature/sqlalchemy-distinct and squashes the following commits: bed97d4 [Wes McKinney] Test nunique on sqlite 491a22f [Wes McKinney] Add table distinct and count distinct support to SQLAlchemy translator. Preserve aggregate names when adapting to aggregation
BUG: workaround #396 by having a default cast name that is not identi…
…cal to the parent expr
ENH: Various window function usability improvements
#543, #542, #532, and #469 Author: Wes McKinney <wes@cloudera.com> Closes #628 from wesm/feature/window-usability and squashes the following commits: 3b8ae25 [Wes McKinney] ENH: propagate down windows in nested analytic expressions. #469 87aa9b4 [Wes McKinney] TST: function binding in ibis window. #532 6d5a6f5 [Wes McKinney] Window string automatic binding #542 a0899ca [Wes McKinney] TST: unit tests for Window.combine #543
ENH: Client.database gets current_database if no argument provided. W…
…orks with SQLite too. Close #634
