ENH: Change signature of pre_execute to dispatch on client type
Author: Phillip Cloud <cpcloud@gmail.com> Author: Jeff Reback <jeff@reback.net> Closes #1200 from cpcloud/pre-execute-with-client and squashes the following commits: d4a5014 [Phillip Cloud] Clean up format string in client.Database 3ffcdbe [Phillip Cloud] Remove the need for toolz in client 7ff9b9e [Phillip Cloud] Comment about pre_execute 981e885 [Phillip Cloud] Sorted in __dir__ 2277ae5 [Phillip Cloud] Fix literals issue 67d66b6 [Phillip Cloud] Remove multiple backend limitation 07bac48 [Jeff Reback] API: pre_execute now accepts (Node, Client) as a dispatching type to allow backends to operate on the physical tables
BUG: RecursionError during unpickle TableExpr
Closes #1197 Author: Phillip Cloud <cpcloud@gmail.com> Closes #1204 from cpcloud/pickle-bug and squashes the following commits: 9ae2d99 [Phillip Cloud] Fix PY2 pickle 4d41870 [Phillip Cloud] Handle total bigquery API breakage 75015d6 [Phillip Cloud] Lint 195e7da [Phillip Cloud] BUG: RecursionError during unpickle TableExpr
ENH/BUG: Add multiplication to Series and SeriesGroupBy
Closes #1216 Author: Phillip Cloud <cpcloud@gmail.com> Closes #1217 from cpcloud/mult-series and squashes the following commits: 6f3f8a6 [Phillip Cloud] Reduce the number of registrations 69ef31f [Phillip Cloud] Power should always promote to double when all operands are integers 1236846 [Phillip Cloud] Add test ids 67b2d97 [Phillip Cloud] Fix pow 5cd1e93 [Phillip Cloud] ENH/BUG: Add multiplication to Series and SeriesGroupBy
ENH: Parquet, HDF5, and CSV interfaces
closes #1175 closes #1165 on top of #1167 Author: Jeff Reback <jeff@reback.net> Closes #1194 from jreback/parquet and squashes the following commits: 6c53bdf [Jeff Reback] remove implicit casting tests e5a838a [Jeff Reback] remove whitespace dba6609 [Jeff Reback] fix up casting tests 04d9483 [Jeff Reback] subclass halffloat from halffloatvalue 33cd1da [Jeff Reback] bump 2.7 build to numpy 1.11 to allow pyarrow to install 58a6c80 [Jeff Reback] re-add scope caching 49989f8 [Jeff Reback] only need pyarrow 0.6.0 e87a756 [Jeff Reback] update recipe for newer pyarrow e00a34b [Jeff Reback] handle multiple physical tables 620a41d [Jeff Reback] fixed types 655039f [Jeff Reback] review comments 05ceb27 [Jeff Reback] review comments c32c7a9 [Jeff Reback] require pyarrow 0.7.1; use parquet_file.schmea.to_arrow_types() bf3077e [Jeff Reback] compat for strings in py2 acb83b9 [Jeff Reback] ENH: parquet backend implementation 762ef32 [Jeff Reback] ENH: initial support a file-like backend with csv & hdf5 implementations
ENH/API: Add where to nunique reduction
Author: Phillip Cloud <cpcloud@gmail.com> Closes #1220 from cpcloud/count-distinct-filter and squashes the following commits: 65c4864 [Phillip Cloud] Lowercase count distinct 4b709ff [Phillip Cloud] Fix API and impala translation e62c396 [Phillip Cloud] Fix pandas execution 08ef875 [Phillip Cloud] Fix test e73b78f [Phillip Cloud] Add impala test 52e1dd0 [Phillip Cloud] Switch args f7d006a [Phillip Cloud] ENH/API: Add where to nunique reduction
ENH/TST: Adds test for recent count distinct with where filter
Author: Phillip Cloud <cpcloud@gmail.com> Closes #1229 from cpcloud/tests-bigquery-clickhouse and squashes the following commits: b7428bb [Phillip Cloud] Add bigquery count distinct with filter test 8b9115c [Phillip Cloud] Enable bigquery struct collect test b2ac09c [Phillip Cloud] Add clickhouse count distinct with filter test
- [x] interval datatype - [x] interval rules - [x] `IntervalValue`/`IntervalScalar`/`IntervalColumn` - [x] timestamp arithmetics - [x] date arithmetics - [x] time arithmetics - [x] date shorthand - [x] IntervalPromoter - [x] unit conversions - [x] expose convertible units as properties - [x] remove timedelta - [x] to_interval - [x] timestamp.date() - [x] port currently used compilers - [x] impala - [x] clickhouse - [x] thorough testing (sort of done) - [x] interval comparison operators resolves #1233, #1228 Author: Krisztián Szűcs <szucs.krisztian@gmail.com> Closes #1243 from kszucs/interval and squashes the following commits: 34e1b43 [Krisztián Szűcs] resolved review issues 22846ff [Krisztián Szűcs] fix failed tests 6f91095 [Krisztián Szűcs] interval comparisons; complex tests; timestamp date operator fb87e94 [Krisztián Szűcs] interval can_implicit_cast and flake8[again] a0299c6 [Krisztián Szűcs] integer.to_interval conversion d1d03a6 [Krisztián Szűcs] flake8 0ac2d3d [Krisztián Szűcs] cast impala timestamp 9b549f0 [Krisztián Szűcs] sql compiler test upper unit 1ccc8c9 [Krisztián Szűcs] flake8 36013bf [Krisztián Szűcs] cast timestamp arg of impala's date_add and date_sub 83d798d [Krisztián Szűcs] uppercase plural S b5d498a [Krisztián Szűcs] uppercase impala intervals 1a16d8a [Krisztián Szűcs] attempt to fix impala tests 92d19f0 [Krisztián Szűcs] impala timestamp tests f679655 [Krisztián Szűcs] added columns to table fixture fe26028 [Krisztián Szűcs] re-enabled temporal tests 953b477 [Krisztián Szűcs] right temporal add operations 62fdf5d [Krisztián Szűcs] fix temporal add operations 3b8e484 [Krisztián Szűcs] clickhouse interval implementation 51b9b11 [Krisztián Szűcs] flake8 ba4b2da [Krisztián Szűcs] adapt impala compiler to support interval type 084183a [Krisztián Szűcs] fixed timestamp test case b2fbedd [Krisztián Szűcs] upconvert and floordivide 08a11e8 [Krisztián Szűcs] Little Interval documentation 51b75d7 [Krisztián Szűcs] replaced temporal functions 12a637b [Krisztián Szűcs] IntervalPromoter ee2712b [Krisztián Szűcs] int backed interval datatype 6425e76 [Krisztián Szűcs] interval aliases eff73f3 [Krisztián Szűcs] disable impala test case 874744f [Krisztián Szűcs] Revert "removed subsecond interval units" a9153f6 [Krisztián Szűcs] removed subsecond interval units 990494d [Krisztián Szűcs] comment two other timedelta tests acb7014 [Krisztián Szűcs] interval unit conversions as properties ae0a080 [Krisztián Szűcs] unit converter; flake8 dc8a9d0 [Krisztián Szűcs] temp disable backend test cases using timedeltas 24087bf [Krisztián Szűcs] cleanup unused method 4238c7c [Krisztián Szűcs] missed adding compat.py bf918c2 [Krisztián Szűcs] date, time timestamp arithmetics with proper rules 9013e9c [Krisztián Szűcs] fixed interval repr test case; marked unused methods dcb96d4 [Krisztián Szűcs] IntervalAdd arithmetic c72b915 [Krisztián Szűcs] fixed test cases ef95969 [Krisztián Szűcs] skeleton for interval arithmetics 3f11173 [Krisztián Szűcs] test for unsupported interval units and parsing failure ca6b64b [Krisztián Szűcs] validate interval units; second as default unit dd5b0b0 [Krisztián Szűcs] flake8: trailing whitespaces d29778f [Krisztián Szűcs] parse interval unit 50a25ef [Krisztián Szűcs] interval datatype skeleton
- docker-compose setup, replaces #1121 - refactored datamgr.py - moved test_data_admin and datamgr under testing - start-all script to spin up every backends - ~~skeleton for backend agnostic tests, related to #1237~~ - updated circleci - [x] upgrade development docs - [x] update appveyor - [x] make builds pass Author: Krisztián Szűcs <szucs.krisztian@gmail.com> Closes #1255 from kszucs/testing and squashes the following commits: c9238f5 [Krisztián Szűcs] little doc comment 7cbff6e [Krisztián Szűcs] removed testing modifications; little docs update 9c182b2 [Krisztián Szűcs] disable all backend in common client fixture 84935d7 [Krisztián Szűcs] attempt to fix appveyor builds bbe930f [Krisztián Szűcs] flake8 again 56d8f01 [Krisztián Szűcs] strange trailing whitespace issue 9021ace [Krisztián Szűcs] fix env vars in posgres test_client 68e4851 [Krisztián Szűcs] missed postgres doctest ada6912 [Krisztián Szűcs] disable sqlite vacuum 4c52f8c [Krisztián Szűcs] trying to fix sqlite analyze error ea0f15a [Krisztián Szűcs] update env variables in conftest files a5495a1 [Krisztián Szűcs] flake8 f0b365f [Krisztián Szűcs] common tests 01190e9 [Krisztián Szűcs] modify webhdfs test port 989185d [Krisztián Szűcs] fixed clickhouse column name in tests 5d67117 [Krisztián Szűcs] remove dbug cmd from circleci 3c01458 [Krisztián Szűcs] click un-underscore auto envvar prefix a43fd79 [Krisztián Szűcs] debug ibis data dir ebfb8b2 [Krisztián Szűcs] print destination directory during untar f290dfe [Krisztián Szűcs] start-all 637079b [Krisztián Szűcs] underscore ca53c89 [Krisztián Szűcs] fixed envvar parsing 4990897 [Krisztián Szűcs] move bigquery conda in requirements files 0cda557 [Krisztián Szűcs] correct download paths 71189f7 [Krisztián Szűcs] update datamgr f311516 [Krisztián Szűcs] moved testing scripts under testing 7f263e8 [Krisztián Szűcs] fix paths 0418dd9 [Krisztián Szűcs] e 4edd16e [Krisztián Szűcs] e 7ef54e1 [Krisztián Szűcs] dlake8 60082a3 [Krisztián Szűcs] ci
- quarter support - allowed conversion between year - quarter - month - named test cases resolves #1252 Author: Krisztián Szűcs <szucs.krisztian@gmail.com> Closes #1259 from kszucs/interval-quarter and squashes the following commits: 3e6d6e4 [Krisztián Szűcs] wrong side rebase; remove datamgre manually 3bfd9d6 [Krisztián Szűcs] relese notes; fixed review issues 041353d [Krisztián Szűcs] disable sqlite vacuum e262806 [Krisztián Szűcs] flake8 bf67239 [Krisztián Szűcs] quarter interval unit; allowed unit conversion betweek year-quarter-month
Clickhouse offset translation fix
Clickhouse supports `LIMIT n, offset` style clause instead of `LIMIT n OFFSET offset` Author: Krisztián Szűcs <szucs.krisztian@gmail.com> Closes #1266 from kszucs/clickhouse_offset and squashes the following commits: dae750d [Krisztián Szűcs] fix comment 4810b54 [Krisztián Szűcs] clickhouse offset translation fix
Dtype, schema inference and implicit casting
Relates to #1221 Author: Krisztián Szűcs <szucs.krisztian@gmail.com> Closes #1269 from kszucs/infer_schema and squashes the following commits: 93bd201 [Krisztián Szűcs] revert to_pandas 2a1578a [Krisztián Szűcs] schema.to_pandas test 3558d95 [Krisztián Szűcs] categorical 85beb27 [Krisztián Szűcs] to_pandas skeleton 4ccb42e [Krisztián Szűcs] missing contextmanager decorator for compat.suppress b6369c9 [Krisztián Szűcs] resolve a cuple of review issues bcc6d25 [Krisztián Szűcs] comment out test case a461e99 [Krisztián Szűcs] resolve compat issue 8b3102e [Krisztián Szűcs] flake8 ef2439e [Krisztián Szűcs] support for pandas categorical affff8b [Krisztián Szűcs] fix DatetimeTZDtype acd27ec [Krisztián Szűcs] another attempt to fix pandas types namespace 337430d [Krisztián Szűcs] fix pandas types namespace ee0a27c [Krisztián Szűcs] infer pandas timestamp e7aecc5 [Krisztián Szűcs] subtype checking 4bd5c5b [Krisztián Szűcs] little rename; trigger builds 27cae31 [Krisztián Szűcs] comment out array casting 13a09f7 [Krisztián Szűcs] numpy binary dtype 0b646f6 [Krisztián Szűcs] pass literal value to implicit casts; fix failing udf test; fb04c84 [Krisztián Szűcs] fix doctest errors 43a581e [Krisztián Szűcs] higher precedence only upcasts 6c49f56 [Krisztián Szűcs] forbid double to float implicit cast 774c317 [Krisztián Szűcs] simplify higher precedence dtype c2d82a8 [Krisztián Szűcs] infer pandas schema a54f662 [Krisztián Szűcs] tests for implicit dtype cast 191a476 [Krisztián Szűcs] integer to category cast a0ab226 [Krisztián Szűcs] infer numpy generic types 7ea04f4 [Krisztián Szűcs] factor out ir types' _can_compare methods d97432d [Krisztián Szűcs] factor out _can_implicit_cast and _implicit_cast calls for comparison operations a4f7874 [Krisztián Szűcs] allow double->float because float literal is inferred as double 1aaa292 [Krisztián Szűcs] resolve builtins float name conflict 7b6faf4 [Krisztián Szűcs] fixed failed decimal to floating casts 11ecaf7 [Krisztián Szűcs] multipledispatch to conda build reqs 27471f9 [Krisztián Szűcs] refactored datatype implicit casting d332d8a [Krisztián Szűcs] make multipledispatch mandatory dependency 926943b [Krisztián Szűcs] infer_dtype draft
BUG: Postgres 10 failing on regex_extract
Closes #1276 Author: Phillip Cloud <cpcloud@gmail.com> Closes #1278 from cpcloud/fix-pg-10-regexp-extract and squashes the following commits: 253034e [Phillip Cloud] Add comment with link to bug report 022898f [Phillip Cloud] Fix for real 6fd6ddc [Phillip Cloud] Start manually again ceda828 [Phillip Cloud] Fix f15a5ae [Phillip Cloud] Try again 8122ff2 [Phillip Cloud] Try starting postgres 2313ac0 [Phillip Cloud] Quoting 2 ffef295 [Phillip Cloud] Quoting bc05d06 [Phillip Cloud] See if diagnostics are available 4e64258 [Phillip Cloud] Run postgresql 10 in appveyor 02d57e3 [Phillip Cloud] Postgres 10 failing on regex_extract
ENH/TST/BUG: Add lowest common denominator testing, fill out backend …
…operations This is a large PR that adds a test loop for a large class of operations for every backend with the ultimate goal being to remove the copy pasted execution tests that are sitting inside each individual backends `tests/` directory. I also took the liberty of bringing the backends up to par with each other on many operations (most missing operations were string ops). Notably, we are still missing coverage on window functions; there are tests but they are all xfailed (some backends are xpassing, others truly failing). I need to work out the details of what exactly is missing from analytic function support from each backend. Author: Phillip Cloud <cpcloud@gmail.com> Closes #1256 from cpcloud/test-loop and squashes the following commits: 5dec31f [Phillip Cloud] Address python 2 np.int64 mro 23f995a [Phillip Cloud] Review comments 345c668 [Phillip Cloud] Fix docs 6f4fdc3 [Phillip Cloud] Use the correct error bd3d322 [Phillip Cloud] Raises KeyError not AssertionError 926abf7 [Phillip Cloud] xfail some clickhouse driver stuff 16481e4 [Phillip Cloud] Address comments df24fab [Phillip Cloud] Fix appveyor test running b4d940e [Phillip Cloud] Cleanup c79a071 [Phillip Cloud] Revert "Rename fixtures" d91f6b3 [Phillip Cloud] Run from the installed dir 4d065f6 [Phillip Cloud] Update recipe b935b05 [Phillip Cloud] Rename fixtures 5fb911b [Phillip Cloud] More maketrans compat fb3d16a [Phillip Cloud] Simplify maketrans e06559c [Phillip Cloud] Cleanup c3dd5df [Phillip Cloud] Unicode 9cdeb01 [Phillip Cloud] Add wrapped for py2 compat c77c78f [Phillip Cloud] Cleanup fu 3be5c4a [Phillip Cloud] Fix dependencies 1176804 [Phillip Cloud] Fix maketrans compat 08e2604 [Phillip Cloud] Fix translate 193904e [Phillip Cloud] Add __wrapped__ attribute in PY2 wrapped function for compat with pytest e5db551 [Phillip Cloud] Add absolute_import to ibis/tests/all/config a9e706e [Phillip Cloud] Fixes a373d19 [Phillip Cloud] Clean up test loop 0aa4f9f [Phillip Cloud] Fix escaping 48dde42 [Phillip Cloud] Fix doctest f129283 [Phillip Cloud] Try ignoring tests 9978fa3 [Phillip Cloud] Ignore backend tests. cd37442 [Phillip Cloud] Remove unused test function in init c6564ce [Phillip Cloud] Fix doctests 4f12eb6 [Phillip Cloud] Fix clickhouse translation 5350283 [Phillip Cloud] Fix name ad33851 [Phillip Cloud] Add regex to conda build test a31dc6f [Phillip Cloud] Flake8 fixes 64378ef [Phillip Cloud] Fixes 7cf8a18 [Phillip Cloud] Differentiate array tests 1fc07d6 [Phillip Cloud] Clickhouse array literals d9834f6 [Phillip Cloud] Array configs 8e1717e [Phillip Cloud] Get clickhouse working 507a22f [Phillip Cloud] Ignore the escape parameter in backends that do not implement it f922251 [Phillip Cloud] Make sure backend is the first argument f72b5dd [Phillip Cloud] Default escape implementation for sqlalchemy backends 051a47c [Phillip Cloud] Add escape 2083774 [Phillip Cloud] Remove file 88117c9 [Phillip Cloud] Add ESCAPE clause to LIKE operation 83254f3 [Phillip Cloud] Add new exception for skipping tests when a backend does not implement operations on a certain type dbb8faf [Phillip Cloud] Fix CSV issues c2aefa0 [Phillip Cloud] Move code around 5002081 [Phillip Cloud] Remove unused REGEXP declaration 43de4c8 [Phillip Cloud] Add invalid operations 9b7a67b [Phillip Cloud] Add FindInSet for pandas ed1d017 [Phillip Cloud] Add safe_index function ba9cd25 [Phillip Cloud] Fix pandas string test 378b003 [Phillip Cloud] Implement string like pattern to regex converter for pandas 7787cb9 [Phillip Cloud] Add test for like pattern to regex converter b711302 [Phillip Cloud] Add more complex like tests for pandas 9bd1d3c [Phillip Cloud] Use regex instead of re 018681b [Phillip Cloud] Add regex library to allow posix character classes 7443738 [Phillip Cloud] Add regex library to allow posix character classes c1fd497 [Phillip Cloud] Postgres find_in_set -1 if not found 886e969 [Phillip Cloud] Fix clickhouse 0e55d74 [Phillip Cloud] BigQuery array literals 19f6959 [Phillip Cloud] SQLite does not support arrays 4032442 [Phillip Cloud] Add test_array.py for testing array ops across all backends 761bac2 [Phillip Cloud] Fix bigquery array concat a34ece3 [Phillip Cloud] Clean up cdoe a bit b16c645 [Phillip Cloud] Remove _distinct_from helper function 38508f9 [Phillip Cloud] Fix find_in_set 07f432c [Phillip Cloud] Add test for find_in_set with nonmissing values 65befc2 [Phillip Cloud] Switch argument order for variance 76b53e3 [Phillip Cloud] Change no translator rule test ddd3800 [Phillip Cloud] Use the correct envars 31cdde0 [Phillip Cloud] Ignore exceptions in close_cursor 78fd674 [Phillip Cloud] Fix error de03ef0 [Phillip Cloud] Switch where and how in variance based operations a444f86 [Phillip Cloud] Fix xfails e42b748 [Phillip Cloud] Fix var_samp fee56fc [Phillip Cloud] Proper marks f98ca60 [Phillip Cloud] fix flake8 729a0ff [Phillip Cloud] Copy to prevent mutation 251f812 [Phillip Cloud] Get almost all tests passing for all backends except bigquery-sum 9dae731 [Phillip Cloud] Different table for window functions c943715 [Phillip Cloud] Fix var for compat with bigquery f3d36fe [Phillip Cloud] Any all bigquery b3f7ff4 [Phillip Cloud] Remove pdb 850fcb1 [Phillip Cloud] Allow overriding of functional_alltypes table loading 5cd4dae [Phillip Cloud] Compile does nothing in the parquet backend 8c51a4e [Phillip Cloud] Add ability to specify a column's type 7f9faa8 [Phillip Cloud] Add cumulative ops to postgres 42ee507 [Phillip Cloud] Skip on operationnotdefinederror on execute c595d52 [Phillip Cloud] Show exactly what failed instead trying to guess 19817b1 [Phillip Cloud] OperationNotDefinedError instead of ValueError 5bc519a [Phillip Cloud] Pandas compile does nothing c8d05b0 [Phillip Cloud] Add pandas dialect to csv and parquet clients bfacc8c [Phillip Cloud] Add back window functions 9ba8c55 [Phillip Cloud] Remove window functions for now 15a6f7b [Phillip Cloud] Fixes [ci skip] 2accc97 [Phillip Cloud] Fix sqlite 1e546ce [Phillip Cloud] Fix postgres compiler f6445b3 [Phillip Cloud] Fix regexp extract 56ad4b4 [Phillip Cloud] Use immutable expressions 9714b4c [Phillip Cloud] Use regexp_like 18e7f53 [Phillip Cloud] Generalize flatten f5b8b9c [Phillip Cloud] Clean test_string a bit 3cc629a [Phillip Cloud] Make actual assertions c57e777 [Phillip Cloud] Add dialect to mock f26fac6 [Phillip Cloud] Add dialect to clickhouse fac6566 [Phillip Cloud] Use character classes for LHS regexen 1800bbb [Phillip Cloud] Abstract unordered comparison out 936b696 [Phillip Cloud] Use regexp_like function ae7f79a [Phillip Cloud] Connect to impala 870e2ad [Phillip Cloud] Add aggs to big test loop 7eb024d [Phillip Cloud] Make ValueList behave more list like 146c857 [Phillip Cloud] More concise flatten 5e3dcc4 [Phillip Cloud] Fix parsing in testing 6976a57 [Phillip Cloud] Fix std var ddof and add not any not all 3e112a1 [Phillip Cloud] Add bool to type parser f58d185 [Phillip Cloud] Add minimal list operations to ListExpr 376aa9d [Phillip Cloud] Fill out pandas string operations 2786283 [Phillip Cloud] ListOf returns a ValueList 7f53f72 [Phillip Cloud] Add functools compat d316dbb [Phillip Cloud] Add split and join to postgres a70b569 [Phillip Cloud] Remove unused function 32dd3ad [Phillip Cloud] Fix bigquery ops f56042f [Phillip Cloud] Better test 363227a [Phillip Cloud] Fix docstring ccca715 [Phillip Cloud] Fill out sqlite string ops 18f4089 [Phillip Cloud] Translate compat aeb07e4 [Phillip Cloud] StringSplit operation c53ebb5 [Phillip Cloud] New constants for pandas accb619 [Phillip Cloud] APIs for new string functions c65fabe [Phillip Cloud] Add split and concat 9d1a1ba [Phillip Cloud] Add bigquery implementations 0f55cd5 [Phillip Cloud] BigQuery date cast test 5a49877 [Phillip Cloud] Add string split tests for pandas e34bdb1 [Phillip Cloud] Test refactor fc311bc [Phillip Cloud] Refacto backend test config 54ee48c [Phillip Cloud] Backends can skip operations in special cases 56648a4 [Phillip Cloud] Endow the pandas backend with a dialect b749b9b [Phillip Cloud] Split strings and arrays into separate modules for pandas 798ce3c [Phillip Cloud] Add where to group concat api 0e305ab [Phillip Cloud] Subclass ec25b98 [Phillip Cloud] Spelling ecf8ab3 [Phillip Cloud] Add DataFrame comparison function 437feaa [Phillip Cloud] Fix imports 97b65be [Phillip Cloud] String repeat misspelling 3381ded [Phillip Cloud] Module init f0828b0 [Phillip Cloud] Add 7ea85a0 [Phillip Cloud] Checkpoint [ci skip] 61eb3d1 [Phillip Cloud] Add test column da0733a [Phillip Cloud] Rename ebd337b [Phillip Cloud] TST: Add test loop
BUG/TST: Fix ops.Where import time in the pandas backend
cc @DiegoAlbertoTorres Author: Phillip Cloud <cpcloud@gmail.com> Closes #1304 from cpcloud/fix-where and squashes the following commits: 19075ce [Phillip Cloud] Skip correctly 6470b40 [Phillip Cloud] Importorskip sh for conda build tests 8412439 [Phillip Cloud] Skip on CI a91ef82 [Phillip Cloud] Refactor test a bit ffa5b35 [Phillip Cloud] Simplify ab34bdf [Phillip Cloud] Fix ops.Where import time in the pandas backend 22840aa [Phillip Cloud] Rename
BUG: can't mutate and then drop
closes #1296 The issue is that `_maybe_resolve_exprs` was not ignoring columns that don't exist in a child `Selection` so it would try to resolve them and throw an `IbisTypeError`. This is okay, because the expressions have already been resolved in the `Projector` constructor so trying to project a column that doesn't anywhere would raise this exception long before getting to `_maybe_resolve_exprs`. Author: Phillip Cloud <cpcloud@gmail.com> Closes #1299 from cpcloud/fix-drop and squashes the following commits: 123b156 [Phillip Cloud] If we look for a column that does not exist in the root's selection we cannot fuse so return None 74b2e7f [Phillip Cloud] Cleanup repeated variable use 2b8b34b [Phillip Cloud] Revert 1e31609 [Phillip Cloud] Do something different for drop de1799c [Phillip Cloud] Test stuff 6bffb3a [Phillip Cloud] Actually exercise the bug 719d0f9 [Phillip Cloud] Add test
ENH/TST: Speed up postgres loading
This timed out on a recent build: https://circleci.com/gh/cpcloud/ibis/6792?utm_campaign=vcs- integration-link&utm_medium=referral&utm_source=github-build-link Author: Phillip Cloud <cpcloud@gmail.com> Closes #1305 from cpcloud/speedup-pg-load and squashes the following commits: 6a7d85a [Phillip Cloud] Username not user 192bd55 [Phillip Cloud] Use text for windows c84b970 [Phillip Cloud] psql for appveyor 692dc02 [Phillip Cloud] Port stuff and path stuff bb8c5b4 [Phillip Cloud] Give explicit parameters 66db47f [Phillip Cloud] Do not mutate the params argument 56a95c3 [Phillip Cloud] Try installing the postgresql-client 7e33f2a [Phillip Cloud] Put psql on the PATH on appveyor and use shell pipes and read from STDIN to avoid filesystem permissions issues 03b9920 [Phillip Cloud] One line c6fe1d7 [Phillip Cloud] ENH/TST: Speed up postgres loading
- separated postgres from impala - reduces duplicated service definitions - makes a circleci build reproducible locally - moved every testing/ci resources inder ci directory TODO: - udfs kill impala daemon Author: Krisztián Szűcs <szucs.krisztian@gmail.com> Closes #1311 from kszucs/compose and squashes the following commits: 845d7a1 [Krisztián Szűcs] remove comments from datamgr d85b805 [Krisztián Szűcs] bigquery works 3f1870f [Krisztián Szűcs] bq debug d95e782 [Krisztián Szűcs] appveyor again 199e476 [Krisztián Szűcs] debug bq issue 1 b984bd8 [Krisztián Szűcs] debig bq issue 299adc7 [Krisztián Szűcs] conda forge channel on appveyor ee5c289 [Krisztián Szűcs] no nomkl on windows 059eaf7 [Krisztián Szűcs] pandas from pip 2b042c0 [Krisztián Szűcs] pytest from pip addadd4 [Krisztián Szűcs] compose parameter default value c88cd03 [Krisztián Szűcs] flake8 from pip b234b65 [Krisztián Szűcs] remove python-graphviz from conda install a88628e [Krisztián Szűcs] plumbum pip on appveyor 8e7f7dc [Krisztián Szűcs] plumbum appveyor 1860107 [Krisztián Szűcs] plumbum 0202e66 [Krisztián Szűcs] trying to fix appveyor 4 6244188 [Krisztián Szűcs] trying to fix appveyor 3 c6ac20a [Krisztián Szűcs] trying to fix appveyor 2 3a0d5af [Krisztián Szűcs] trying to fix appveyor 1 bee2b34 [Krisztián Szűcs] trying to fix appveyor 82ffd6c [Krisztián Szűcs] double asv config print 24f8b6b [Krisztián Szűcs] add certs to dockerfile f2083e0 [Krisztián Szűcs] correct benchmark command 1e9c2ec [Krisztián Szűcs] fix cci syntax 24cfbfc [Krisztián Szűcs] appveyor port as string 8878e9b [Krisztián Szűcs] benchmark instead of build b90fd0b [Krisztián Szűcs] ensure dir d53ffc5 [Krisztián Szűcs] flake8 fb542d8 [Krisztián Szűcs] benchmark db8b705 [Krisztián Szűcs] change sqlite's database path aa1dfc7 [Krisztián Szűcs] unverbose pytest 985225a [Krisztián Szűcs] fix appveyor paths 599ba4f [Krisztián Szűcs] unnecessary identation in requirements file 7661c6a [Krisztián Szűcs] panic in case of failed data loading b6b3d7f [Krisztián Szűcs] temp skip udf 410555f [Krisztián Szűcs] sourcing a832a7f [Krisztián Szűcs] run all tests 974e194 [Krisztián Szűcs] setuppy c329a0e [Krisztián Szűcs] reenable d0fd9ac [Krisztián Szűcs] per version f44d0c5 [Krisztián Szűcs] all-in-one 414500f [Krisztián Szűcs] eee d3de7a0 [Krisztián Szűcs] pip 4ada4c9 [Krisztián Szűcs] conda nightmare 1969890 [Krisztián Szűcs] quiet c814138 [Krisztián Szűcs] conda dep update 5fdca1f [Krisztián Szűcs] install conda build from defaults 64d7ab8 [Krisztián Szűcs] sqlite database path 1f7f72e [Krisztián Szűcs] pin pytz on py34 2087b4f [Krisztián Szűcs] verbose pytest e8786ef [Krisztián Szűcs] envenvenv+1 d19dae4 [Krisztián Szűcs] envenvenv 914441f [Krisztián Szűcs] make 471c080 [Krisztián Szűcs] setup.py develop e53ff38 [Krisztián Szűcs] dont cleanup conda env f7b8d2a [Krisztián Szűcs] tests again 2b9ca42 [Krisztián Szűcs] add gcc to the dockerfile 7705aa8 [Krisztián Szűcs] try to alter python pkg before updating the env 823ee59 [Krisztián Szűcs] quiet d964845 [Krisztián Szűcs] ee aeef44b [Krisztián Szűcs] ee 675decf [Krisztián Szűcs] remove cmakecache 7f8d79c [Krisztián Szűcs] remove cmakefiles eea361a [Krisztián Szűcs] fix conda build 9ef5c05 [Krisztián Szűcs] load udfs 137d5bc [Krisztián Szűcs] ee c59283a [Krisztián Szűcs] quiet conda f5eb2e1 [Krisztián Szűcs] re-enable builds b148cc7 [Krisztián Szűcs] install to root conda env 29b06cc [Krisztián Szűcs] postgres host to doctest 6370a32 [Krisztián Szűcs] fix cci syntax 17473b4 [Krisztián Szűcs] fix cci synta fd66087 [Krisztián Szűcs] build steps 9003ed5 [Krisztián Szűcs] test bigquey a003a0e [Krisztián Szűcs] test bigquey f9d6c98 [Krisztián Szűcs] test bigquey c5b2619 [Krisztián Szűcs] test bigquey 56dfe30 [Krisztián Szűcs] test bigquey 6b6770d [Krisztián Szűcs] test bigquey 1042013 [Krisztián Szűcs] flake8 4a98604 [Krisztián Szűcs] correct backend test configs cde8b05 [Krisztián Szűcs] correct postgres config 9d82063 [Krisztián Szűcs] correct env variables ce13cca [Krisztián Szűcs] fix test command b391fc2 [Krisztián Szűcs] run tests c3d0e5f [Krisztián Szűcs] cleanup 016cca0 [Krisztián Szűcs] links are not required a016e88 [Krisztián Szűcs] wait retry interval 60b2c2d [Krisztián Szűcs] named steps 5795b18 [Krisztián Szűcs] flake8 a6640d5 [Krisztián Szűcs] attempt89 676a6ee [Krisztián Szűcs] pass pgpassword as env variable 9013200 [Krisztián Szűcs] attempt88 ec5f5b2 [Krisztián Szűcs] attempt87 1b6e722 [Krisztián Szűcs] attempt65 9ce25b8 [Krisztián Szűcs] attempt85 5440a0b [Krisztián Szűcs] attempt84 744d361 [Krisztián Szűcs] attempt83 8944781 [Krisztián Szűcs] attempt82 17f0a3c [Krisztián Szűcs] attempt81 f782b70 [Krisztián Szűcs] attempt80 93082ba [Krisztián Szűcs] attempt79 c66fbe6 [Krisztián Szűcs] attempt78 055fee8 [Krisztián Szűcs] attempt77 0f12d1e [Krisztián Szűcs] attempt76 214af89 [Krisztián Szűcs] attempt75 bd52735 [Krisztián Szűcs] attempt74 74f7f2f [Krisztián Szűcs] attempt73 036f432 [Krisztián Szűcs] attempt72 2292390 [Krisztián Szűcs] attempt71 2d9a235 [Krisztián Szűcs] attempt70 7c4d806 [Krisztián Szűcs] attempt69 dc26dff [Krisztián Szűcs] attempt68 9bd4eea [Krisztián Szűcs] attempt67 6c5409b [Krisztián Szűcs] attempt66 c897c6f [Krisztián Szűcs] attempt65 a616d98 [Krisztián Szűcs] attempt64 83004de [Krisztián Szűcs] attempt63 797ac93 [Krisztián Szűcs] attempt62 8b03e98 [Krisztián Szűcs] attempt61 e19e07f [Krisztián Szűcs] attempt60 1203075 [Krisztián Szűcs] attempt59 47f4b55 [Krisztián Szűcs] attempt58 d389b15 [Krisztián Szűcs] attempt57 1279f02 [Krisztián Szűcs] attempt56 52f4cbf [Krisztián Szűcs] attempt55 da21eca [Krisztián Szűcs] attempt54 5f7d13d [Krisztián Szűcs] attempt53 34362a0 [Krisztián Szűcs] attempt52 82561d7 [Krisztián Szűcs] attempt51 b503461 [Krisztián Szűcs] attempt50 c8d70d0 [Krisztián Szűcs] attempt49 fe7c7d3 [Krisztián Szűcs] attempt48 e41de87 [Krisztián Szűcs] attempt47 9191184 [Krisztián Szűcs] attempt46 af90336 [Krisztián Szűcs] attempt45 44b078d [Krisztián Szűcs] attempt44 b25481d [Krisztián Szűcs] attempt43 15e3ad0 [Krisztián Szűcs] attempt42 a4bd253 [Krisztián Szűcs] attempt41 80144d1 [Krisztián Szűcs] attempt40 93363ba [Krisztián Szűcs] attempt40 45c37fa [Krisztián Szűcs] attempt39 bf546f7 [Krisztián Szűcs] attempt38 46fcb1d [Krisztián Szűcs] attempt37 0c2a51d [Krisztián Szűcs] attempt36 5d6168e [Krisztián Szűcs] attempt35 9a77347 [Krisztián Szűcs] attempt34 d21eaf0 [Krisztián Szűcs] attempt33 1abf100 [Krisztián Szűcs] attempt32 453c257 [Krisztián Szűcs] attempt31 eab5150 [Krisztián Szűcs] attempt30 0031410 [Krisztián Szűcs] attempt29 90ec56b [Krisztián Szűcs] attempt28 8f2116c [Krisztián Szűcs] attempt27 2e88620 [Krisztián Szűcs] attempt26 efa9c4c [Krisztián Szűcs] attempt25 1972597 [Krisztián Szűcs] attempt24 db200a4 [Krisztián Szűcs] attempt23 9099efe [Krisztián Szűcs] attempt22 65c4d3c [Krisztián Szűcs] attempt21 7ecf5b0 [Krisztián Szűcs] attempt20 ca08c02 [Krisztián Szűcs] attempt19 68bfd73 [Krisztián Szűcs] attempt18 8fab976 [Krisztián Szűcs] attempt17 f0e959f [Krisztián Szűcs] attempt16 0163e6f [Krisztián Szűcs] attempt15 2d17338 [Krisztián Szűcs] attempt14 2e27d56 [Krisztián Szűcs] attempt13 847016d [Krisztián Szűcs] docker layer caching 1975685 [Krisztián Szűcs] attempt12 9ca1e17 [Krisztián Szűcs] attempt11 899d710 [Krisztián Szűcs] attempt10 a70d5f1 [Krisztián Szűcs] attempt9 3a4108f [Krisztián Szűcs] attempt8 a053b75 [Krisztián Szűcs] attempt7 1890d9e [Krisztián Szűcs] attempt6 62d7c5b [Krisztián Szűcs] attempt5 b648aa7 [Krisztián Szűcs] attempt4 936f211 [Krisztián Szűcs] attempt3 a3dea2a [Krisztián Szűcs] attempt2 2405584 [Krisztián Szűcs] attempt1 29f6d0c [Krisztián Szűcs] attempt0
ENH: Add PARTITIONTIME column to BigQuery backend
Author: dlovell <tsdlovell@twosigma.com> Author: Phillip Cloud <cpcloud@gmail.com> Closes #1322 from cpcloud/has-partitions and squashes the following commits: 12a5cf3 [Phillip Cloud] Fix test 14c2cf6 [Phillip Cloud] Conftest for partitioned bq tables 3d353bb [Phillip Cloud] Add tests 3e35502 [dlovell] BUG: typo 7c71ac9 [dlovell] BUG: make pairs a list so it can be appended to 5f1e010 [dlovell] ENH: register partition time col name a1d429f [dlovell] REF: use a variable for the magical PARTITIONTIME string 1c834c7 [dlovell] ENH: add cpcloud's PARITIONTIME mutation 54ba2be [dlovell] BUG: properly detect _PARTITIONTIME column e34d4e6 [Phillip Cloud] Stuff
Ensure that BigQuery queries run until they are complete, instead of …
closes #1291 Author: Krisztián Szűcs <szucs.krisztian@gmail.com> Closes #1302 from kszucs/addcol and squashes the following commits: cab4f2e [Krisztián Szűcs] use pytest deprecated_call instead 5e18732 [Krisztián Szűcs] flake8 8c8e8e0 [Krisztián Szűcs] configure pytest to not filter deprication warning 07ed602 [Krisztián Szűcs] depricate add_column 1a3b045 [Krisztián Szűcs] remove accidental comment 77b360e [Krisztián Szűcs] add_column proxies to mutate, resolves #1291
DOC: Setup automated doc builds
Author: Phillip Cloud <cpcloud@gmail.com> Closes #1306 from cpcloud/doc-stuff and squashes the following commits: 3dc491a [Phillip Cloud] Specific branch ee3e6c0 [Phillip Cloud] Add other builds back 1163d83 [Phillip Cloud] Remove unused import 1e80f8e [Phillip Cloud] DOC: Setup automated doc builds
