Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

str_detect always treats pattern as fixed string #168

Closed
dlindelof opened this issue Sep 26, 2018 · 7 comments
Closed

str_detect always treats pattern as fixed string #168

dlindelof opened this issue Sep 26, 2018 · 7 comments
Milestone

Comments

@dlindelof
Copy link

@dlindelof dlindelof commented Sep 26, 2018

When str_detect is used in a call to filter on a tbl, I expect pattern to be treated as a regular expression. Instead, it seems that it is treated as a fixed pattern:

> MASTER %>% filter(str_detect(FOO, "pen$")) %>% show_query
<SQL>
SELECT *
FROM "MASTER"
WHERE (INSTR("FOO", 'pen$') > 0)

The SQL INSTR function performs a case-insensitive search, and does not treat its second argument as a regex.

@colearendt

This comment has been minimized.

Copy link
Collaborator

@colearendt colearendt commented Oct 10, 2018

What database are you using here?

@kmace

This comment has been minimized.

Copy link

@kmace kmace commented Oct 11, 2018

I have this issue too. I am using teradata.

@PNorvaisas

This comment has been minimized.

Copy link

@PNorvaisas PNorvaisas commented Oct 11, 2018

I have the same problem with MySQL database. Need to collect data to be able to use patterns in str_detect.

@yutannihilation

This comment has been minimized.

Copy link
Member

@yutannihilation yutannihilation commented Oct 12, 2018

str_detect() can be translated as INSTR(), CHARINDEX(), or CHARINDEX(), but neither of them is a regex function.

dbplyr/R/db-postgres.r

Lines 68 to 70 in 79d2a03

str_detect = function(string, pattern){
sql_expr(strpos(!!string, !!pattern) > 0L)
}

dbplyr/R/db-odbc-mssql.R

Lines 93 to 96 in 79d2a03

str_detect = function(string, pattern){
build_sql(
"CHARINDEX(", pattern, ", ", string, ") > 0"
)}

According to #151 (comment), this seems a design decision.

There's unfortunately no way that dbplyr can translate every stringr feature into every database backend.

You can directly specify a function particular to the backend, though. For example (this is only for illustration purpose and the SQLs are invalid for SQLite):

library(DBI)
library(dplyr, warn.conflicts = FALSE)
library(dbplyr, warn.conflicts = FALSE)

con <- dbConnect(RSQLite::SQLite(), ":memory:")
copy_to(con, iris)

# MySQL has REGEXP
tbl(con, "iris") %>%
  filter(FOO %regexp% "pen$") %>%
  show_query
#> <SQL>
#> SELECT *
#> FROM `iris`
#> WHERE (`FOO` REGEXP 'pen$')

# Teradata has REGEXP_LIKE
tbl(con, "iris") %>%
  filter(regexp_like(FOO, "pen$")) %>%
  show_query
#> <SQL>
#> SELECT *
#> FROM `iris`
#> WHERE (REGEXP_LIKE(`FOO`, 'pen$'))

Created on 2018-10-12 by the reprex package (v0.2.1)

@yutannihilation

This comment has been minimized.

Copy link
Member

@yutannihilation yutannihilation commented Oct 12, 2018

@kmace
BTW, for tidyverse/stringr#262, you might feel you were using regex, but it was, sadly, just a bug (c.f. #81)... The dev version of dbplyr will give you 0 rows with the very same code.

@kmace

This comment has been minimized.

Copy link

@kmace kmace commented Oct 12, 2018

@yutannihilation
Oh wow. Thanks for that heads up.

@hadley

This comment has been minimized.

Copy link
Member

@hadley hadley commented Jan 3, 2019

@colearendt do you want to take this one on too?

@hadley hadley added this to the v1.4.0 milestone Jan 9, 2019
@hadley hadley added the wip label Mar 13, 2019
@hadley hadley closed this in 785d106 Mar 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.