Author: Krisztián Szűcs <szucs.krisztian@gmail.com> Closes #1366 from kszucs/prainbow and squashes the following commits: 63f0786 [Krisztián Szűcs] missing import 80b85c4 [Krisztián Szűcs] missing newlines eed2ec7 [Krisztián Szűcs] fixing custom operation notebook d292008 [Krisztián Szűcs] syntax error in release notes 3b6b9c4 [Krisztián Szűcs] release notes ba36429 [Krisztián Szűcs] reintroduce value expr imports in public api 3356427 [Krisztián Szűcs] test nullable propagation d47849d [Krisztián Szűcs] flake8 9892db5 [Krisztián Szűcs] TypeSignature.from_dtypes for pandas UDFs 16dbb4e [Krisztián Szűcs] resolve conflicts; simplified table rule 332fa50 [Krisztián Szűcs] remove todo note c7805e7 [Krisztián Szűcs] pickle with protocol 2 for objects with __slots__ defined f9e28c3 [Krisztián Szűcs] remove specific pickle protocol be5968b [Krisztián Szűcs] remove operation is_ancestor b7048b4 [Krisztián Szűcs] rebase 0297cd9 [Krisztián Szűcs] remove unnecessary constructors d129e76 [Krisztián Szűcs] typo 6bbc822 [Krisztián Szűcs] signature optional property 5a059c5 [Krisztián Szűcs] fix a couple of review issues 1dad6bf [Krisztián Szűcs] resolve conflicts; added tests using singleton dt.null 68d6243 [Krisztián Szűcs] fix bigquery tests 7ce42d0 [Krisztián Szűcs] fix optional 99d8180 [Krisztián Szűcs] fix tests 45c8b5a [Krisztián Szűcs] factor out signature 0b1bcd3 [Krisztián Szűcs] shape_like for impala udfs 9fb8915 [Krisztián Szűcs] shape_like b3a60dc [Krisztián Szűcs] renames for backward compat 4c3089b [Krisztián Szűcs] fix doc test c5ba16b [Krisztián Szűcs] value count test b8a70aa [Krisztián Szűcs] unnecessary ternary operator 3e94580 [Krisztián Szűcs] doc 229b847 [Krisztián Szűcs] doc dc36d3c [Krisztián Szűcs] format tests f822eae [Krisztián Szűcs] review e1612a6 [Krisztián Szűcs] review 0728a2b [Krisztián Szűcs] review 600b2d5 [Krisztián Szűcs] fix signed <-> unsigned implicit casting; fix a couple of review issues 544910a [Krisztián Szűcs] refactor impala udfs to support new signatures 35c7e20 [Krisztián Szűcs] use __slots__ for operations to provide argument order in a compatible manner acfeaa8 [Krisztián Szűcs] fix formatting 7567fb9 [Krisztián Szűcs] flake8 ed47c7a [Krisztián Szűcs] cleanups 06e687b [Krisztián Szűcs] rebase 7ae1613 [Krisztián Szűcs] fix temporal tests 5b270da [Krisztián Szűcs] flake8 d0750b7 [Krisztián Szűcs] fixing test cases 18bee0a [Krisztián Szűcs] fix coalesce for impala and bigquery 1010d38 [Krisztián Szűcs] fix timezone infer issue 7536468 [Krisztián Szűcs] flake8 6e550ac [Krisztián Szűcs] fix a couple of interval tests bef5c96 [Krisztián Szűcs] pandas stringconcat stringjoin b4eea2d [Krisztián Szűcs] alchemy fixes 6358244 [Krisztián Szűcs] fix a couple of test cases b315efa [Krisztián Szűcs] drop node descriptors edccf4e [Krisztián Szűcs] rename back to float/double 757d81a [Krisztián Szűcs] flake8 fbcd1ac [Krisztián Szűcs] fix quantile cdf3fc0 [Krisztián Szűcs] fixed tests 1b23802 [Krisztián Szűcs] minor fixes b9b6dab [Krisztián Szűcs] refactored promotions; removed old rules 8a8f05c [Krisztián Szűcs] extended shapeof rule ccb4d19 [Krisztián Szűcs] removed a couple of previous rules fb36534 [Krisztián Szűcs] arg corrections 486d508 [Krisztián Szűcs] arguments for analytic ops d81cc15 [Krisztián Szűcs] remove interval type properties 4eaf1f7 [Krisztián Szűcs] refactor types f4c4c5e [Krisztián Szűcs] wip 0049403 [Krisztián Szűcs] output types 4e4f9c9 [Krisztián Szűcs] reorganize 270d237 [Krisztián Szűcs] todo notes ca9f9fd [Krisztián Szűcs] todo notes 0fbb9be [Krisztián Szűcs] move all Node and ValueOp to operations 2ef6522 [Krisztián Szűcs] enum validator abae059 [Krisztián Szűcs] a couple of additiional rule tests 2813d49 [Krisztián Szűcs] rules tests; remove old counterparts 15029a5 [Krisztián Szűcs] tests for new rules 5cd0b38 [Krisztián Szűcs] remove is_table, is_array, is_scalar, is_collection from rules 3c0171a [Krisztián Szűcs] cleanup commented parts from operations aee5af8 [Krisztián Szűcs] heavily wip, just before first cleanup
ENH: Allow execution of just bound parameters in the pandas backend
ENH: Add JavaScript UDFs for BigQuery
Author: Phillip Cloud <cpcloud@gmail.com> Closes #1377 from cpcloud/bigquery-udfs and squashes the following commits: 6e3766a [Phillip Cloud] Skip yield from test in py2 b9e20c0 [Phillip Cloud] Fix import 357ecc9 [Phillip Cloud] Remove 63ce871 [Phillip Cloud] Add more tests d49fd08 [Phillip Cloud] Use the parent expr 9d4a473 [Phillip Cloud] More descriptive docs f38470c [Phillip Cloud] ENH: Add JavaScript UDFs for BigQuery
1. Clone 2. Render 3. Update version and source in recipe 4. Calls out to conda build Closes #1402 Author: Krisztián Szűcs <szucs.krisztian@gmail.com> Closes #1405 from kszucs/feedstock_update and squashes the following commits: 42303d5 [Krisztián Szűcs] fix py2 unicode error 11c7666 [Krisztián Szűcs] updated circleci config 9ec9cc5 [Krisztián Szűcs] CI script to download update and test conda forge recipe
BigQuery UDF tests were not actually running on CI, due to some incorrectly written code that ignored imports on Python 2. This PR fixes that. Author: Phillip Cloud <cpcloud@gmail.com> Closes #1421 from cpcloud/fix-bigquery-udf-tests and squashes the following commits: 76273ff [Phillip Cloud] BUG/TST: Fix BigQuery UDFs
Author: Krisztián Szűcs <szucs.krisztian@gmail.com> Closes #1420 from kszucs/set and squashes the following commits: 12f80fa [Krisztián Szűcs] unused import e6b2d52 [Krisztián Szűcs] resolve review issues 3517b93 [Krisztián Szűcs] fix order of values in test cases bfd4d5f [Krisztián Szűcs] fix mysql setscalar formatting 9d9858d [Krisztián Szűcs] fix one length tuple formatting for clickhouse 58409e3 [Krisztián Szűcs] all backend param test 875c5b2 [Krisztián Szűcs] set type implementation
ENH: Add to_timestamp support in BigQuery and Pandas
- [x] Update our testing data for parquet Author: Phillip Cloud <cpcloud@gmail.com> Closes #1410 from cpcloud/timestamp-from-unix-bigquery and squashes the following commits: a7a4392 [Phillip Cloud] Remove dead code 23ef10c [Phillip Cloud] More readable error message 8e685c7 [Phillip Cloud] Formatting 7ac4485 [Phillip Cloud] Add an error doctest 3080adf [Phillip Cloud] code formatting fd70bed [Phillip Cloud] Fix a bug where slicing was incorrect a024734 [Phillip Cloud] Do not convert if the units are the same cba682a [Phillip Cloud] Add doctest example ef2c514 [Phillip Cloud] fix impala compiler tests 7ea1ac9 [Phillip Cloud] Remove copyright header b8df182 [Phillip Cloud] Docstring for convert_unit 2f5f953 [Phillip Cloud] Test udf context 7aa4a45 [Phillip Cloud] Call dtype 855197f [Phillip Cloud] Factor out convert_unit b82330c [Phillip Cloud] use named args 9b7ec4b [Phillip Cloud] Remove string dispatches 59ad6ac [Phillip Cloud] Docstring for type translation b6cb3a0 [Phillip Cloud] Explain the need to translate int64 to float64 in udfs 2741254 [Phillip Cloud] Rename types.py to datatypes.py 888b32a [Phillip Cloud] ENH: Add to_timestamp support in BigQuery and Pandas
ENH: Abstract out the table_class and table_expr_class properties
Removed dead code that had a bug in it. Author: Phillip Cloud <cpcloud@gmail.com> Closes #1436 from cpcloud/fix-formatting-bug and squashes the following commits: 554b856 [Phillip Cloud] BUG: Fix format string bug
ENH: Refactor the pandas backend
This is a major refactor of the pandas backend to support new functionality as well as to clean up and simplify the architecture. 1. `execute_first` is removed: This was originally useful for implementing the ``WindowOp`` rule, but is made obsolete by allowing ``ibis.client.Client``s to be data arguments in ``execute_node`` rules. 1. `data_preload` is removed: This was originally use to load in data from the pandas backend specifically, and was very much a special snowflake in the execute loop. This is also made obsolete by allowing ``ibis.client.Client``s to be data arguments in ``execute_node`` rules. 1. ``execute_literal`` was introduced to avoid the performance overhead of dispatching a call to ``execute_node`` for the few cases we have of evaluating ``ops.Literal`` nodes. 1. I've introduced a new ``inputs`` property to the ``Node`` class, which is the sequence of arguments required to evaluate an expression without any additional context. Currently, only ``WindowOp`` requires something different because it needs access to the column underlying the ``expr`` since you cannot blindly evaluate ``expr`` and get the correct result, all other ``Node`` subclasses define ``inputs`` as ``return self.args``. 1. Finally, I've added support for expressions in the `preceding` and `following` arguments of `WindowOp`. Author: Phillip Cloud <cpcloud@gmail.com> Closes #1441 from cpcloud/pandas-execute-refactor and squashes the following commits: 168e016 [Phillip Cloud] ENH: Refactor the pandas backend
ENH: Fill out more pandas backend operations
Also fixes an xfailed test because of postgres Author: Phillip Cloud <cpcloud@gmail.com> Closes #1423 from cpcloud/pandas-operations and squashes the following commits: 0d3722e [Phillip Cloud] chunksize of 1 5ec6318 [Phillip Cloud] Chunksize for database insertion 93bf5fe [Phillip Cloud] Repeat for 0.23 because series constructor behavior changed eaf0fa7 [Phillip Cloud] Repeat quantile for 0.23 b8b0026 [Phillip Cloud] Cannot yet infer decimal d386e9a [Phillip Cloud] Install older sqlalchemy for now c2301b3 [Phillip Cloud] Compat ab9836b [Phillip Cloud] Shorter 7810ec3 [Phillip Cloud] PY2 places bf60041 [Phillip Cloud] Call to super a0656ec [Phillip Cloud] Give sign its own test c0f16b1 [Phillip Cloud] Fix bigquery sign aee877a [Phillip Cloud] Bring back sign 8f01ace [Phillip Cloud] Fix sign b3a6a94 [Phillip Cloud] cOmment out sign 7a7ae0c [Phillip Cloud] Remove license headers ccd8144 [Phillip Cloud] Remove license headers ed10af1 [Phillip Cloud] Dtype on modulus is not easily predictable 35f4434 [Phillip Cloud] Remove smallint dispatch 8ae3180 [Phillip Cloud] Dispatch for mysql integer types b781929 [Phillip Cloud] Add mysql types a7deee7 [Phillip Cloud] Remove header d44b494 [Phillip Cloud] Flake8 b56e8c9 [Phillip Cloud] Clean up test 6b3c450 [Phillip Cloud] Add backend property for floating modulus 6684a14 [Phillip Cloud] Further separate modulus with floating point operand 72670cd [Phillip Cloud] Fix args a577ec5 [Phillip Cloud] Forward the dialect 46ad7e1 [Phillip Cloud] Remove unnecessary test 3c09526 [Phillip Cloud] Breakout test_mod to special case clickhouse b315ba6 [Phillip Cloud] Use python mod for sqite mod bb1b166 [Phillip Cloud] Use allclose for testing floating point equality 011990e [Phillip Cloud] Use param for better naming fa0f16e [Phillip Cloud] Cast in log and use DOUBLE_PRECISION in type map 06db221 [Phillip Cloud] Remove unused code 8b4d219 [Phillip Cloud] Add mysql double 2bf133c [Phillip Cloud] Pass dialect when getting the type a756da8 [Phillip Cloud] Fix postgres numerics 5df050a [Phillip Cloud] Fix sql builtins ada5413 [Phillip Cloud] Pandas ops 8e84d31 [Phillip Cloud] Impala should conform to ibis not the other way around 67df787 [Phillip Cloud] Fix the output type of sign 4178420 [Phillip Cloud] Implement rounding behavior tests d38b1ac [Phillip Cloud] Move rounding behavior comment d97c05c [Phillip Cloud] Break out rounding test 3faab88 [Phillip Cloud] Comment about postgres rounding behavior b06edfd [Phillip Cloud] ENH: Add scalar math functions to pandas
ENH: Add array_of rule to allow arbitrary value_type validation
Author: Phillip Cloud <cpcloud@gmail.com> Closes #1453 from cpcloud/array-of-rule and squashes the following commits: 2838849 [Phillip Cloud] Slow down data loading on windows only 053bd6d [Phillip Cloud] More informative error message 106582a [Phillip Cloud] Shorten 20a23dc [Phillip Cloud] ENH: Add `array_of` rule to allow arbitrary value_type validation
BUG/CI/TST: Fix compatibility with pandas 0.23
Pandas 0.23 changed the default behavior of the `chunksize` argument which breaks SQLite when loading a large dataset because SQLite has an upper bound on the number of rows in a single insert statement. This PR works around that our `ci/datamgr.py` script so that we can get master back to green. Author: Phillip Cloud <cpcloud@gmail.com> Closes #1458 from cpcloud/fix-build and squashes the following commits: 453fc5b [Phillip Cloud] Tests should no assume __del__ will be run e8faf15 [Phillip Cloud] Fix impala tests a8d83dd [Phillip Cloud] Fix pandas doctest 6cafcb0 [Phillip Cloud] Linting c8e33f6 [Phillip Cloud] Remove the UTC argument to the Timestamp constructor ecdaf91 [Phillip Cloud] Fix clickhouse 1519430 [Phillip Cloud] Gross 50ab7d4 [Phillip Cloud] Try turning off layer caching e76aaed [Phillip Cloud] workaround pandas 0.23 e2aa7ad [Phillip Cloud] Try direct insertion 8469da7 [Phillip Cloud] BUG/CI: Insert chunksize for sqlite should be 1
Append common actions to the PandasClient
This pull request adds list_tables, load_data, and create_table attributes for the PandasClient. Author: tonyfast <tony.fast@gmail.com> Closes #1464 from tonyfast/pandas-ddl and squashes the following commits: cd045b5 [tonyfast] Fix flake spacings fd92308 [tonyfast] Merge remote-tracking branch 'ibis/master' into pandas-ddl 9e0f582 [tonyfast] Include a get_schema attribute and tests for create_table 9490ddd [tonyfast] Merge remote-tracking branch 'ibis/master' into pandas-ddl dc3bb12 [tonyfast] A simpler exists_table method 5647eea [tonyfast] Resolve some flake8 issues 88997b1 [tonyfast] Add a test for list_tables(like="") 4dd37af [tonyfast] Add a get_schema method to the pandas client and fix for review 9db2db3 [tonyfast] Add spaces after function definition and simplify list_tables test. def4e16 [tonyfast] Include tests for loading data with the pandas backend. 9d7aad7 [tonyfast] Add list_tables, load_data, and create_table to the pandas client
Closes #1463 Author: Scott Lustig <scott.lustig@twosigma.com> Closes #1467 from missing-semicolon/bigquery-schema and squashes the following commits: 53266ab [Scott Lustig] Merge branch 'master' of https://github.com/ibis-project/ibis into bigquery-schema 50b08a1 [Scott Lustig] Implement _get_schema_using_query and _adapt_types 4d8e3e9 [Scott Lustig] Implement client.sql test
Implement Strptime in BigQuery
Closes #1455 As implemented, the string method `to_datetime` requires a format string argument and accepts an optional argument for timezones. All comments appreciated! Author: Scott Lustig <scott.lustig@twosigma.com> Author: Scott Lustig <s.j.lustig@gmail.com> Closes #1457 from missing-semicolon/implement-strpdate and squashes the following commits: 1f0d899 [Scott Lustig] PEP8 47fda25 [Scott Lustig] StringToTimestamp returns UTC timestamp cfbad90 [Scott Lustig] Merge branch 'master' into implement-strpdate 93983b0 [Scott Lustig] Merge branch 'master' of https://github.com/ibis-project/ibis into implement-strpdate 10afd94 [Scott Lustig] Address additional comments 882ed16 [Scott Lustig] Add execution test. d331261 [Scott Lustig] Add compiler test 7f9bf4b [Scott Lustig] Edits from feedback 2dde1e2 [Scott Lustig] Fix formatting on docstring 8fe1a73 [Scott Lustig] Add to_datetime to Docs 003f628 [Scott Lustig] Implement bigquery to_datetime
WIP: ENH/CI: Add pandas 0.20, 0.22 to ci builds
Depends on #1458 Closes #1465 Author: Phillip Cloud <cpcloud@gmail.com> Closes #1466 from cpcloud/add-pandas-versions and squashes the following commits: 4a6a9ba [Phillip Cloud] Strings for paths on python2 and older pandas 3f8ad28 [Phillip Cloud] Revert compat changes and fix lint issues 8f08b81 [Phillip Cloud] Use item to get the python value of a numpy scalar out 53b079d [Phillip Cloud] ENH/CI: Add pandas 0.20, 0.22 to ci builds
Enable ibis.literal to detect and preserve when datetime has a timezone.
Closes #1459 Author: Scott Lustig <scott.lustig@twosigma.com> Closes #1473 from missing-semicolon/timestamp-tz-literal and squashes the following commits: 0d5339b [Scott Lustig] Update test for literal timezone attr as string 1c8cf5c [Scott Lustig] Cast tzinfo to string f2d7e1a [Scott Lustig] Merge branch 'master' of https://github.com/ibis-project/ibis into timestamp-tz-literal 02d2f94 [Scott Lustig] Implement and Test ibis.literal maintains timestamp's timezone
ENH/BUG: Enable multiple definitions as well as fix duplicate definit…
…ions Closes #1472 Author: Phillip Cloud <cpcloud@gmail.com> Closes #1474 from cpcloud/fix-bq-udf-multiple-calls and squashes the following commits: 42b67f9 [Phillip Cloud] Better test name e5ca5c2 [Phillip Cloud] Lint 3232625 [Phillip Cloud] Fix test c70904b [Phillip Cloud] Add libraries argument docstring 11c2d80 [Phillip Cloud] Add back libraries argument after the rebase ce0336d [Phillip Cloud] No need for a lambda 083749b [Phillip Cloud] More accurate comment b009ee5 [Phillip Cloud] Better comment f069014 [Phillip Cloud] ENH/BUG: Enable multiple definitions as well as fix duplicate definitions
BUG: Disable creation of UDFs that contain integer signatures
Closes #1469 Author: Phillip Cloud <cpcloud@gmail.com> Closes #1478 from cpcloud/fix-bq-udf-int64 and squashes the following commits: 3812eb7 [Phillip Cloud] Remove note 18ab8a9 [Phillip Cloud] Fix duplicate definition 55902b0 [Phillip Cloud] Fix doctest d9e895b [Phillip Cloud] Fix imports abcd0b5 [Phillip Cloud] Lint b07df49 [Phillip Cloud] Add tests 294d341 [Phillip Cloud] Move UDFContext into an easier to find module f62698f [Phillip Cloud] Compile the UDF signature before compiling the function 85e2368 [Phillip Cloud] Add tests
Support component accessors on time type: e.g., hour, minute, second
resolves #1486 Author: Krisztián Szűcs <szucs.krisztian@gmail.com> Closes #1487 from kszucs/1486 and squashes the following commits: 2f08ce0 [Krisztián Szűcs] correct pytest skipif marker 290b36d [Krisztián Szűcs] skip tests on py2 due to time is unsupported 8785ead [Krisztián Szűcs] extract time from timestemp instead of casting 64288ee [Krisztián Szűcs] time field accessors
