TST: WebHDFS parameters are now configurable
Adds two additional environment variables * `IBIS_TEST_HDFS_HOST` * `IBIS_TEST_WEBHDFS_PORT` that default to a localhost dev environment. Fixes #324.
Add logging per #287 and overwrite=True implementation per #296 Author: Wes McKinney <wes@cloudera.com> Closes #333 from wesm/hdfs-get-overwrite and squashes the following commits: f983d94 [Wes McKinney] Test directory overwrite and add basic logging per #287 and #296 1afb05f [Wes McKinney] Add overwrite support to HDFS.get for single files
BUG: setup.py no longer imports numpy at top level
Before the fix, this required the user to have numpy installed, even if they were not going to use the comms extension.
ENH: make Ibis compatible with Python 2.6
Changes are: * Fixing lots of strings for .format() to include the explicit arg number. * Using the set() constructor instead of set literals * Added argparse to requirements if installing on py2.6 * Refactoring out the use of subprocess.check_output() * Import unittest2 when using py2.6 * Added ibis.compat for the new unittest Fixes #215.
Implement remaining Impala string built-in functions
Implements all string features except 'space'. Discussed and decided 'space' was not neccesary. per #256 Author: Meghana Vuyyuru <megvuyyuru@gmail.com> Closes #338 from megvuyyuru/str-blitz and squashes the following commits: 7aaaacd [Meghana Vuyyuru] TST: More robust checking of builtin string,int, and double functions c243154 [Meghana Vuyyuru] ENH: All input arguments and return values now 0-indexed edd2327 [Meghana Vuyyuru] ENH: Changed find_in_set to take a list of strings c9dcc25 [Meghana Vuyyuru] STY: flake8 cleanup 41d78ea [Meghana Vuyyuru] ENH: String join method that mirrors python's join function. Removes concat and concat_ws 3444c4c [Meghana Vuyyuru] STY: Reduces code duplication in SQL generation and improves some docstrings bdd9749 [Meghana Vuyyuru] STY: Moving code around for better flow f495abd [Meghana Vuyyuru] Updating avro_schema test 63b8250 [Meghana Vuyyuru] STY: Moving string functions to proper dict. Closes #256 2a4fa5b [Meghana Vuyyuru] ENH: String function 're_extract', 're_replace' added 864b1e9 [Meghana Vuyyuru] ENH: string function concat, concat_ws added d3203a9 [Meghana Vuyyuru] ENH, STY: Adds lpad, rpad. Cleans up test file 9d84b3e [Meghana Vuyyuru] ENH: String function 'translate', 'locate', 'find_in_set' now implemented and tested 6842ed5 [Meghana Vuyyuru] ENH: String function instr now implemented and tested. 325a37c [Meghana Vuyyuru] CLN: Creates and uses _int_output function to mirror _string_output paradigm 0edc24d [Meghana Vuyyuru] ENH String functions 'reverse', 'repeat' added 8d4f4c8 [Meghana Vuyyuru] ENH: String functions added: trim, ltrim, trim ce50b51 [Meghana Vuyyuru] ENH: String function 'ascii' added
TST: fix failing test_create_database_with_location
Test now uses an HDFS path that the user "guarantees" is writable. Fixes #339
Ibis developer API for operation type signatures, associated refactoring
Adds a set of input type declaration API that validate operation types. Removed as many operation constructors as I could and simplified some code in `ibis/sql/exprs.py`. Some bugs were uncovered in the process and can be addressed separately. Some related issues: #331, #305, #285, #345, #346 Author: Wes McKinney <wes@cloudera.com> Closes #347 from wesm/type-validation and squashes the following commits: 1423839 [Wes McKinney] Type decl for timedelta 87acc6c [Wes McKinney] Joint type checker for checking array[boolean] e458e5b [Wes McKinney] Varargs validation, more arg types and common code use c165f83 [Wes McKinney] Data type name validator 8f28dbf [Wes McKinney] Get rid of ensure_array cruft 546b70f [Wes McKinney] Implement ListOf validator. Raise exceptions out of type validation 916f9da [Wes McKinney] Argument validators as classes a3273a8 [Wes McKinney] Move more user APIs to api.py 3383748 [Wes McKinney] Handle int expr for pos in locate 82f93b3 [Wes McKinney] More type cleaning. Fix up analytics.py 9b34a3b [Wes McKinney] Bunch of refactoring and work on type validation, miscellanea ad5746d [Wes McKinney] Initial type validation experiment and some light refactoring e62fd5c [Wes McKinney] Stubs 1f73b9e [Wes McKinney] Remove some redundant root_tables and type checking functions 485cf40 [Wes McKinney] Refactoring some code into rules.py. Remove MultiExprNode and generalize root analysis for ValueNode
TST: fix failing test_set_database (#340)
Creates a new ImpalaClient object without ibis_testing as default db. Test will fail if the default database happens to have a table called `functional_alltypes`. Seems unlikely, though. Fixes #340.
Work around isnull decimal return type issue
Impala has some inconsistent type output in `isnull`. In the meantime, I'm inserting an explicit cast hack. per #345 Author: Wes McKinney <wes@cloudera.com> Closes #357 from wesm/isnull-decimal-type-issue and squashes the following commits: b9b53db [Wes McKinney] Little decimal cast hack to work around Impala issue 54dc026 [Wes McKinney] Enable argument checkers to see all arguments
TST: fix failing test_query_parquet_file_like_table
Accidentally referenced `tpch.region` instead of `tpch_region`. The schema for `tpch_region` is now manually specified to `int16` in the data-loading script bc schema inference assumes it's `int32` bc of parquet. Fixes #341.
BUG: fix avro e2e test post refactor. Add CLI to load_test_data.pyto …
…specify a local directory vs pulling from S3 per #289
Adds remaining Impala string builtin functions
Adds two Impala built-in string functions. Refactors some functions to better mirror Python string functions. #316 Author: Meghana Vuyyuru <megvuyyuru@gmail.com> Closes #356 from megvuyyuru/str-cleanup and squashes the following commits: 218e95c [Meghana Vuyyuru] STY: Reduced duplicate code fb2dc75 [Meghana Vuyyuru] ENH: Added capitalize, parse_url
Implement TRUNC for timestamp values
Closes #303. Set of supported units to `trunc` and their appearance in Ibis should be reviewed at some point. Author: Wes McKinney <wes@cloudera.com> Closes #368 from wesm/timestamp-truncate and squashes the following commits: c8f7807 [Wes McKinney] Finish basic truncate implementation and tests c5cab51 [Wes McKinney] Initial truncate scaffold