Error in a dataset with 78550 rows 10 columns

[joniarroba]

While running the full dataset it gives the following errors.

datafrem characteristics:
df = 78550 rows 10 columns

str(df)
Classes ‘spec_tbl_df’, ‘tbl_df’, ‘tbl’ and 'data.frame': 78550 obs. of 10 variables:
$ date : Date, format: "2015-02-15" "2015-02-15" "2015-02-15" ...
$ author : Factor w/ 5 levels "‎ vazio","joni",..: 2 2 3 3 2 3 3 2 3 2 ...
$ message : chr "Oi Nubi" "Bom dia" "Bom dia!" "\U0001f60a" ...
$ msn_lengh : int 7 7 9 1 52 30 11 34 24 31 ...
$ day : int 15 15 15 15 15 15 15 15 15 15 ...
$ week : num 7 7 7 7 7 7 7 7 7 7 ...
$ month : num 2 2 2 2 2 2 2 2 2 2 ...
$ year : num 2015 2015 2015 2015 2015 ...
$ question_flag: chr "N" "N" "N" "N" ...
$ laughs : chr "N" "N" "N" "N" ...

Erros message

inspect_cat(df)

Column (2/5): authorError: Tibble columns must have consistent lengths, only values of length one are recycled:

• Length 6: Column value
• Length 11: Column prop
  Call rlang::last_error() to see a backtrace

When I sample it to 10k rows it works. Still looking around over the problem.

[alastairrushworth]

Hi @joniarroba thanks for the feedback.

It looks like the column author is causing the issue. So I can identify the issue, could you paste the output of the following commands:

• table(df$author, useNA = 'ifany')
• levels(df$author)
• rlang::last_error()

Thanks.

[joniarroba]

Hi @alastairrushworth , thanks for the support, here we go with the messages!

First

table(df$author, useNA = 'ifany')

‎ vazio joni nubia ‎vazio vazio
0 42333 36195 0 0

Second

levels(df$author)
[1] "‎ vazio" "joni" "nubia" "‎vazio" "vazio"

Third

rlang::last_error()

message: Tibble columns must have consistent lengths, only values of length one are recycled:

• Length 8: Column value
• Length 12: Column prop
  class: rlang_error
  backtrace:

1. inspectdf::inspect_cat(whats)
2. inspectdf:::fast_table(df_cat[[i]], show_na = TRUE, show_cnt = TRUE)
3. tibble::tibble(value = vals, prop = freq/length(v))
4. tibble:::lst_to_tibble(xlq$output, .rows, .name_repair, lengths = xlq$lengths)
5. tibble:::recycle_columns(x, .rows, lengths)
   Call rlang::last_trace() to see the full backtrace

In the mean I while find a way to solve it!

[alastairrushworth]

Hi @joniarroba thanks for the update! Glad you've found a way to solve this.

I think the problem is because there are duplicated factor levels in df$author - vazio appears twice in levels(df$author). I think think you can fix this by resetting the levels using something like

levels(df$author) <- unique(levels(df$author))

In the meantime, I'll open a new issue to handle this internally with a warning in inspectdf.

Thanks!

[alastairrushworth]

I've added a factor check to the GH version of the package. So hopefully this fixes the problem you were having. Thanks!