html_table() error

[jubjubbc]

I'm somewhat new to scraping with R, but I'm getting an error message that I can't make sense of. A commenter on stackoverflow said that html_table() isn't handling the colspan logic correctly and suggested I file a bug report.

My code:

 url <- "https://en.wikipedia.org/wiki/California_State_Legislature,_2017%E2%80%9318_session"

leg <- read_html(url)

testdata <- leg %>% 
  html_nodes('table') %>% 
  .[6] %>% 
  html_table()

To which I get the response:

Error in out[j + k, ] : subscript out of bounds

[lifesabirch]

This error is the result of assuming that the html is formed properly and that a row span does not go beyond the end of the table.

read_html("<table><tr><td rowspan='7'>foo</td></tr></table>") %>% html_table()

If you add a check to see if you're at the end of the table, it works

for (k in seq_len(rowspan - 1)) {
  #ADD the following line it works just fine 
  if (j+k > nrow(out)) break;
  l <- utils::head(out[j+k, ], i-1)
  r <- utils::tail(out[j+k, ], maxp-i+1)
  out[j + k, ] <- utils::head(c(l, out[j, i], r), maxp)
}

[sreineri]

Hi, I have the same problem, but I couldnt fix it with the help above.

library('rvest')

url <- 'http://www2.bmf.com.br/pages/portal/bmfbovespa/lumis/lum-taxas-referenciais-bmf-ptBR.asp'

#Le o codigo HTML da url indicada
site <- read_html(url)
lista_tabela <- site %>%
html_nodes("table") %>%
html_table(fill=TRUE)

Error in out[j + k, ] : subscript out of bounds

Can you please help me?
Tks

[lifesabirch]

create your own version of html_table.xml_node and put in: this line: for (k in seq_len(rowspan - 1)) {
#ADD the following line it works just fine
if (j+k > nrow(out)) break;
l <- utils::head(out[j+k, ], i-1)
r <- utils::tail(out[j+k, ], maxp-i+1)
out[j + k, ] <- utils::head(c(l, out[j, i], r), maxp)
}

[sreineri]

I am sorry, I need more details. What is this rowspan? Can you please send me the whole code that you have tested and worked?

Thank you very much