Support bitwise operators

[davidchall]

This PR adds support for the base R functions: bitwNot(), bitwAnd(), bitwOr(), bitwXor(), bitwShiftL(), bitwShiftR().

I've added support for most backends based on their documentation and some manual testing. However, I didn't consider the ODBC and Access backends - are these also needed?

Also, is there some standard way to systematically compare the results produced by each db connection type to those produced by dplyr? I saw the figure from #131 and was wondering if this was produced by a standard script?

[edgararuiz-zz]

HI @davidchall , thank you, this PR is looking great to me! The way are are running systematic tests against individual database back ends is via infrastructure and a testing package that @colearendt has created.

@colearendt - Do you think we may be able to run this PR through your dbtest setup?

[colearendt]

Sure thing! @davidchall do you have some examples of the kinds of mutate, filter, or select statements that might be used with these verbs? I have to admit I am not super familiar with them. The input for the tests is a YAML file with these types of statements. I think some of the currently open PRs have some examples of this YAML syntax, but I am happy to build the file if you have some handy expressions that are good test cases.

[hadley]

Looking great!

Can you please add a bullet to NEWS? It should briefly describe the change and end with (@yourname, #issuenumber).

[davidchall]

Thank you all for your feedback! Let's wait until I've resolved the final comment from @hadley before running dbtest.

@colearendt Looking at your examples, I came up with the following YAML file. I've not seen the values in the fld_integer column, so it's possible that some of the hardcoded values are inappropriate.

Also, it's important to note that the results can depend on whether we're using a signed or unsigned integer data type. Will be interesting to see if anything is flagged due to this.

- bitwNot:
    mutate: bitwNot(fld_integer)
    filter: bitwNot(fld_integer) < 0
    summarize: n_distinct(bitwNot(fld_integer))
    group_by: bitwNot(fld_integer)
    arrange: bitwNot(fld_integer)
- bitwAnd:
    mutate: bitwAnd(fld_integer, 8L)
    filter: bitwAnd(fld_integer, 8L) > 0
    summarize: n_distinct(bitwAnd(fld_integer, 8L))
    group_by: bitwAnd(fld_integer, 8L)
    arrange: bitwAnd(fld_integer, 8L)
- bitwOr:
    mutate: bitwOr(fld_integer, 8L)
    filter: bitwOr(fld_integer, 8L) > 0
    summarize: n_distinct(bitwOr(fld_integer, 8L))
    group_by: bitwOr(fld_integer, 8L)
    arrange: bitwOr(fld_integer, 8L)
- bitwXor:
    mutate: bitwXor(fld_integer, 8L)
    filter: bitwXor(fld_integer, 8L) > 0
    summarize: n_distinct(bitwXor(fld_integer, 8L))
    group_by: bitwXor(fld_integer, 8L)
    arrange: bitwXor(fld_integer, 8L)
- bitwShiftL:
    mutate: bitwShiftL(fld_integer, 1L)
    filter: bitwShiftL(fld_integer, 1L) > 10
    summarize: n_distinct(bitwShiftL(fld_integer, 1L))
    group_by: bitwShiftL(fld_integer, 1L)
    arrange: bitwShiftL(fld_integer, 1L)
- bitwShiftR:
    mutate: bitwShiftR(fld_integer, 1L)
    filter: bitwShiftR(fld_integer, 1L) > 10
    summarize: n_distinct(bitwShiftR(fld_integer, 1L))
    group_by: bitwShiftR(fld_integer, 1L)
    arrange: bitwShiftR(fld_integer, 1L)

[davidchall]

@colearendt Ok, I think it should be good to test now - and I've updated the YAML file above to account for the latest changes. Thanks!

[hadley]

Thanks!

[colearendt]

@davidchall Thanks for writing the YAML file! I will include that into our dbtest suite. Overall, it looks really good. Just a few problems (on Postgres / MSSQL, which is what I currently have handy to test):

Looks like semantics of "where" clauses are different in SQL vs. R. In particular, "WHERE integer" does not get handled appropriately. I don't know what would possibly fix this. It's almost like you need a context aware translation - didn't you run into this somewhere @edgararuiz ?

> t %>% filter(bitwAnd(fld_integer, 8L))
Error in new_result(connection@ptr, statement) : 
  nanodbc/nanodbc.cpp:1344: 42804: [RStudio][PostgreSQL] (30) Error occurred while trying to execute a query: [SQLState 42804] ERROR:  argument of WHERE must be type boolean, not type integer
LINE 3: WHERE ("fld_integer" & 8)
               ^
 
> t %>% filter(bitwAnd(fld_integer, 8L)) %>% show_query()
<SQL>
SELECT *
FROM "blah"
WHERE ("fld_integer" & 8)

The other MSSQL "failures" are just "not available on this SQL variant"

[davidchall]

@colearendt Ah, this does make sense. The result of bitwise operations on integers is also an integer, but the WHERE clause expects to receive a boolean value. I've adjusted the YAML file to account for this.