Add experimental custom autocompletion

[sorhawell]

Close #580

• implement custom Rstudio code completion
• native R / radian code completion, to be as good as Rstudio completion or what possible.
• support vscode-R completion via their webserver feature
• add pl$polars_code_completion()

also to support completion:

• added $columns()-method and names() for GroupBy and LazyGroupBy
• improved "method not found" error message
• added missing .DollarNames.RLazyGroupBy method
• deprecated pl$extra_auto_completion() the old way to activate native R completion.

[sorhawell]

completions for chains, icons shows if method / sub-namespace ("list")

... and assignment (columns<-)

Auto completions can be used to "tab-start" an expression from a column name

works for lazy too

[sorhawell]

interactive syntax explaining via messages. In ipython-polars I miss this feature. Sometimes I tab-complete and nothing happens and I don't know why. Most often because some earlier error in expression.

alias is missing an argument

[sorhawell]

@eitsupi @etiennebacher

It is possible to make very sweet Rstudio auto completion.
I would be possible to also add interactive help-docs or table views of DataFrame.

So far only data.table, dplyr and reticulate and maybe 3 other packages have their completions in the rstudio-source code. At some point we may apply to get our completions included. I guess it will be denied as we use full evaluation of the entire syntax which could slow down the IDE. I have a 3 seconds timeout to block any unintended eager evaluation. R timeout cannot block rust code, before returning to R.

Here I make use the completions methods can be wrapped with custom handlers and injected into the rstudio ide environment. In theory multiple packages could all wrap the same rstudio completion without conflicting.

This approach is using the private api of Rstudio-api, but judging from the blame history. This code part is quite stable on a yearly basis.

[sorhawell]

to activate auto-completion run this

source("~/Documents/projs/r-polars/R/rstudio_completion.R")
.dev$activate_polars_rstudio_completion()
library(polars)

the weird .dev-prefix would not stay. It is just a convenient way to update completion functions without restarting session or building polars package to do faster "trial and error".

[sorhawell]

We also have some reasonable code completion for native R terminals which could be further refined. In the past I was discouraged how bad the native R utils completion worked combined with Rstudio IDE. But that is no longer a problem with this PR. We can detect what IDE is being used.

[etiennebacher]

Love this, the examples look amazing! I'll try a bit more later.

[etiennebacher]

I don't know how to test completion in "native R". I ran vanilla R in bash and in Powershell and couldn't get any completion (for any function, not just polars)

[eitsupi]

Thanks.

I am wondering if there is any worth in the option to enable/disable this.

[etiennebacher]

I am wondering if there is any worth in the option to enable/disable this.

I wanted to enable this automatically at first, but I'm not confident on how it will interact with other packages that might add their own custom completion (also @sorhawell wrote most of this, I'm not so familiar with the autocompletion internals). I'd rather keep this experimental and opt-in for now

[sorhawell]

@eitsupi first time using task to update deps and build. That was a very out-of-the-box experience. Nice!

@etiennebacher
native R terminal has no drop down for multiple suggestions, and there is something annoying where it will
delete what alraady has been written. Right now it works best for the very final completion.

> library(polars)
> pl$code_completion()
Using code completion in 'native' mode.
> # writes this and tab:  pl$col()$ab
> pl$col()$abs() # completes s()
polars Expr: col("").abs()
> rc.status()$comps # use rc.status()$comps for unit testing
[1] "$abs()"
>

it works much better with the radian terminal

writes this and tab: pl$col()$a

[sorhawell]

I found this for unit testing of native-native + native-radian

> utils:::.assignToken("pl$col()$a")
> utils:::.completeToken()
> utils:::.retrieveCompletions()
 [1] "pl$col()$abs()"             "pl$col()$add()"             "pl$col()$agg_groups()"      "pl$col()$alias()"          
 [5] "pl$col()$all()"             "pl$col()$and()"             "pl$col()$any()"             "pl$col()$append()"         
 [9] "pl$col()$approx_n_unique()" "pl$col()$arccos()"          "pl$col()$arccosh()"         "pl$col()$arcsin()"         
[13] "pl$col()$arcsinh()"         "pl$col()$arctan()"          "pl$col()$arctanh()"         "pl$col()$arg_max()"        
[17] "pl$col()$arg_min()"         "pl$col()$arg_sort()"        "pl$col()$arg_unique()"      "pl$col()$argsort()"        
[21] "pl$col()$arr()"   

[eitsupi]

Greate, let's merge.