Error: not compatible with STRSXP

[SamoPP]

Below worked perfectly a few days ago. I can not remember which package I have updated so this is now broken...

baseIQUrl <- "https://www.iqpagedoesnotexist.com/" # This page dies not exist since I can not disclose which page I am logging into

iqPageSession <- html_session(baseIQUrl)
iqPageSession <- iqPageSession %>% follow_link("Log In")
Navigating to https://www.iqpagedoesnotexist.com/my-account
iqLogInForm <- iqPageSession %>% html_node("form") %>% html_form()
Warning message:
In node_find_one(x$node, x$doc, xpath = xpath, nsMap = ns) :
3 matches for .//form: using first
iqLogInForm <- set_values(iqLogInForm, username="mysupersecretusername", password="mysupersecretpassword)
iqPageSession <- submit_form(iqPageSession, iqLogInForm, submit="login")
Error: not compatible with STRSXP
traceback()
3: stop(list(message = "not compatible with STRSXP", call = NULL,
cppstack = NULL)) at RcppExports.R#110
2: xml2::url_absolute(form$url, session$url)
1: submit_form(iqPageSession, iqLogInForm, submit = "login")
sessionInfo()
R version 3.2.0 (2015-04-16)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.2 LTS

locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=en_US.UTF-8
[9] LC_ADDRESS=en_US.UTF-8 LC_TELEPHONE=en_US.UTF-8
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] rvest_0.2.0.9000 xml2_0.1.0.9000
[3] listviewer_0.1 KMData_0.0.1
[5] OpeningGap_0.0.1 RQuantLib_0.4.0
[7] qmao_1.6.10 RCurl_1.95-4.5
[9] bitops_1.0-6 stringr_1.0.0
[11] plyr_1.8.2 gdata_2.16.1
[13] timeDate_3012.100 lubridate_1.3.3
[15] data.table_1.9.5 XML_3.98-1.1
[17] httr_0.6.1.9000 twsInstrument_1.4-10
[19] FinancialInstrument_1.2.0 quantmod_0.4-4
[21] TTR_0.22-0 IBrokers_0.9-10
[23] xts_0.9.874 zoo_1.7-10
[25] jsonlite_0.9.16 rj_2.0.3-2

loaded via a namespace (and not attached):
[1] Rcpp_0.11.6 tools_3.2.0 digest_0.6.8 rj.gd_2.0.0-1
[5] memoise_0.2.1 lattice_0.20-31 shiny_0.12.0.9002 curl_0.6.9000
[9] htmlwidgets_0.4.1 gtools_3.4.2 grid_3.2.0 R6_2.0.0.9000
[13] RJSONIO_1.0-3 selectr_0.2-3 pander_0.5.1 magrittr_1.5
[17] htmltools_0.2.6 mime_0.3 xtable_1.7-3 httpuv_1.3.2
[21] stringi_0.4-1 chron_2.3-45

[jeffthompson1971]

Seeing the same issue. Is this being worked?

[jeroen]

Can you please clean up your post and include a reproducible example? If you can not disclose which page you are logging into, try to find another example. There is no way we can debug this.

[lazarillo]

I have a similar problem. I will try to provide a reproducible example that's easier to work with:

park_url <- "https://reservations.eparks.net/"
park_session <- html_session(park_url)
park_form <- html_form(read_html(park_session))[[1]]

(This next part, with set_values actually does not work, either. I determined this was because rvest does not work (or at least not consistently) with select form fields, but only with input fields.

set_values(park_form, arrive_month = "12",
           arrive_day = "10",
           arrive_year = "2015",
           res_length = "1",
           tent_trailer = "2",
           park_id = "5")

(The form does not have an official submit button, but rather a javascript call-style button, which calls a validation js script. Since it is effectively working like a submit button, I rename the type, as this successful suggestion.)

park_form$fields$btn_search$type <- "submit"

(Here I am setting the values I actually want, since as I said above, rvest's set_values does not work for select fields, at least sometimes).

park_form$fields$arrive_month$value <- "12"
park_form$fields$arrive_day$value <- "10"
park_form$fields$arrive_year$value <- "2015"
park_form$fields$res_length$value <- "1"
park_form$fields$tent_trailer$value <- "2"
park_form$fields$park_id$value <- "5"

Now all the submission form looks good / OK.

I run the submission below, and it fails

submit_form(park_session, park_form)

with this error:

Error in eval(substitute(expr), envir, enclos) : 
  not compatible with STRSXP

---

I then did some debugging to understand where the problem is. As I stepped through it, within the rvest function submit_form, the first line was able to run well:

request <- submit_request(form, submit)

Peering into the request object, I see:

Browse[3]> request
$method
[1] "POST"

$encode
[1] "form"

$url
NULL

$values
$values$actiontype
[1] "search"

$values$CalendarCurrentDate
[1] "10/7/2015"

$values$CalendarLastBookableDate
[1] "10/6/2016"

$values$arrive_date
[1] ""

$values$use_type
[1] ""

$values$arrive_month
[1] "12"

$values$arrive_day
[1] "10"

$values$arrive_year
[1] "2015"

$values$res_length
[1] "1"

$values$tent_trailer
[1] "2"

$values$park_id
[1] "5"

$values$btn_search
[1] "Check Availability"

$values$search_type
[1] "Flash"

---

The next line in submit_form says:

url <- xml2::url_absolute(form$url, session$url)

When this function is run, it fails (with the error Error in eval(substitute(expr), envir, enclos) : not compatible with STRSXP ) in the line:

getExportedValue(pkg, name)

If I look at the values of pkg & name, I see:

Browse[4]> pkg
[1] "xml2"

Browse[4]> name
[1] "url_absolute"

---

I can burrow deeper into the errors that I see, for instance when looking at getExportedValue, but this is just getting into the nitty gritty of namespaces and, I think, pointers / references. It seems unlikely to be the actual root cause of the error.

I hope this further detail helps!

As the original poster had said, it is unclear whether this is happening due to issues in rvest or xml2, but I am only posting here, as per your request not to duplicate.

[lazarillo]

Hmmm... after all that work putting through the details, I just figured out the problem:

The form being passed to url_absolute has a null value for its url field.

However, I still do not understand why that information is being lost, but it does mean this should be with rvest, not xml2. I hope that helps!

[lazarillo]

I discovered the problem:

When the html form has no more specific url where the form submission occurs, then the url_absolute function attempts to combine the NULL (present in the relative HTML form path (e.g., if I have an object my_form returned from html_form, it may have a NULL value in my_form$url.

This then kills the attempt to make that relative path. The solution could be to either:

1. Don't allow for NULL values in the object field created from html_form$url, and instead make it an empty character string "".
2. Allow for xml2::url_absolute() to handle NULL values by coercing them into empty strings.

I personally think the former is better, since I'm a fan of letting mis-matched values throw an error. It makes it easier to track the bugs. But you better understand the bigger picture of how all these libraries work together, obviously! ;-p

[AndreeaGotca]

I have this issue. How should I go about solution no. 1 you mentioned?

stat <- accountj %>% follow_link("Stats")

Navigating to Stats.php

stat_f <- stat %>%
html_nodes("form") %>%
extract2(1) %>%
html_form() %>%

set_values("mydate"="2016-01-15","mydate2"="2016-01-15")

'' (GET )
'mydate': 2016-01-15
'': Go
'mydate2': 2016-01-15
>

data_y <- submit_form(stat,stat_f)
Submitting with 'NULL'
Error: not compatible with STRSXP