Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI][R][Docs] Docs jobs failing due to R Markdown error #39942

Closed
AlenkaF opened this issue Feb 5, 2024 · 4 comments · Fixed by #39977
Closed

[CI][R][Docs] Docs jobs failing due to R Markdown error #39942

AlenkaF opened this issue Feb 5, 2024 · 4 comments · Fixed by #39977
Assignees
@paleolimbot
Labels
Component: Continuous Integration Component: Documentation Component: R Type: bug
Milestone
15.0.1

Comments

@AlenkaF
Copy link
Member

AlenkaF commented Feb 5, 2024 •
edited
Loading

Describe the bug, including details regarding any error messages, version, and platform.

The docs jobs have started failing with a R Markdown error:

-- RMarkdown error -------------------------------------------------------------

Quitting from lines 192-195 [unnamed-chunk-14] (arrow.Rmd)
Warning message:
The vignette title specified in \VignetteIndexEntry{} is different from the title in the YAML metadata. The former is "", and the latter is "Get started with Arrow". If that is intentional, you may set options(rmarkdown.html_vignette.check_title = FALSE) to suppress this check. 
--------------------------------------------------------------------------------
---
Standard error:
Attaching package: 'arrow'

The following object is masked from 'package:utils':

    timestamp

Warning message:
In process_conditional_examples(x) :
  @examplesIf condition `arrow_with_dataset() && identical(Sys.getenv("NOT_CRAN"), "true")` is FALSE
---
Backtrace:
1. pkgdown::build_site(install = FALSE)
2. pkgdown:::build_site_external(pkg = pkg, examples = examples, run_dont_run = run_d...
3. callr::r(function(..., cli_colors, pkgdown_internet) { ...
4. callr:::get_result(output = out, options)
5. callr:::throw(callr_remote_error(remerr, output), parent = fix_msg(remerr[[3]]))

See:

@assignUser @paleolimbot do you have any insights into what should be updated/fixed?

cc @kou

Component(s)

Continuous Integration, R
@AlenkaF AlenkaF mentioned this issue Feb 5, 2024
Open
@jorisvandenbossche
Copy link
Member

FWIW I assume it's not related to the preview, it's just the doc build in general that is failing? (also on the main branch, the last commits have red CI for docs)

@AlenkaF AlenkaF changed the title [CI][R][Docs] Docs preview jobs failing due to R Markdown error [CI][R][Docs] Docs jobs failing due to R Markdown error Feb 5, 2024
@AlenkaF
Copy link
Member Author

AlenkaF commented Feb 5, 2024

Yes, true! Changed the description and title.

@raulcd raulcd mentioned this issue Feb 5, 2024
Merged
@kou
Copy link
Member

kou commented Feb 5, 2024

Quitting from lines 192-195 [unnamed-chunk-14] (arrow.Rmd)

arrow/r/vignettes/arrow.Rmd

Lines 192 to 195 in ed78986

library(reticulate)
sw_table_python <- r_to_py(sw_table)
```

This error happen since 2024-02-01: https://github.com/apache/arrow/actions/runs/7742113156/job/21110485230

reticulate 1.35.0 released on 2024-02-01: https://github.com/rstudio/reticulate/releases/tag/v1.35.0

It seems that reticulate 1.35.0 is related. #39933 may be caused by the same problem.

@paleolimbot
Copy link
Member

I can take a stab at fixing this...it looks like they updated how Python capsules and R external pointers are converted. Basically, I think we need to use strings instead of capsules but my memory of rewriting that is fuzzy.

@kou kou mentioned this issue Feb 5, 2024
Closed
@github-actions github-actions bot mentioned this issue Feb 7, 2024
Merged
paleolimbot added a commit that referenced this issue Feb 9, 2024
@paleolimbot @pitrou @jorisvandenbossche
GH-39942: [Python] Make capsule name check more lenient (#39977)
abf4fbf 
### Rationale for this change

While #39969 fixed the immediate issue caused by the update of the capsule name used by reticulate whilst converting an R "external pointer", it will still result in an error if somebody is using an older version of the Arrow R package.

### What changes are included in this PR?

The pyarrow Cython code was modified to accept capsules with the name NULL or "r_extptr".

### Are these changes tested?

Not sure where the best place for this is, but:

CRAN arrow + released pyarrow + new reticulate (errors):

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.35.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "15.0.0"

reticulate::r_to_py(arrow::int32())
#> PyCapsule_GetPointer called with incorrect name
```

CRAN arrow + pyarrow from this PR + old reticulate:

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.34.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "16.0.0.dev92+geafcff7a5"

reticulate::r_to_py(arrow::int32())
#> DataType(int32)
```

CRAN arrow + pyarrow from this PR + new reticulate:

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.35.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "16.0.0.dev92+geafcff7a5"

reticulate::r_to_py(arrow::int32())
#> DataType(int32)
```

### Are there any user-facing changes?

No
* Closes: #39942

Lead-authored-by: Dewey Dunnington <dewey@fishandwhistle.net>
Co-authored-by: Dewey Dunnington <dewey@voltrondata.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
@paleolimbot paleolimbot added this to the 16.0.0 milestone Feb 9, 2024
dgreiss pushed a commit to dgreiss/arrow that referenced this issue Feb 19, 2024
@paleolimbot @pitrou
@jorisvandenbossche @dgreiss
apacheGH-39942: [Python] Make capsule name check more lenient (apache…
eb209f3 
…#39977)

### Rationale for this change

While apache#39969 fixed the immediate issue caused by the update of the capsule name used by reticulate whilst converting an R "external pointer", it will still result in an error if somebody is using an older version of the Arrow R package.

### What changes are included in this PR?

The pyarrow Cython code was modified to accept capsules with the name NULL or "r_extptr".

### Are these changes tested?

Not sure where the best place for this is, but:

CRAN arrow + released pyarrow + new reticulate (errors):

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.35.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "15.0.0"

reticulate::r_to_py(arrow::int32())
#> PyCapsule_GetPointer called with incorrect name
```

CRAN arrow + pyarrow from this PR + old reticulate:

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.34.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "16.0.0.dev92+geafcff7a5"

reticulate::r_to_py(arrow::int32())
#> DataType(int32)
```

CRAN arrow + pyarrow from this PR + new reticulate:

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.35.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "16.0.0.dev92+geafcff7a5"

reticulate::r_to_py(arrow::int32())
#> DataType(int32)
```

### Are there any user-facing changes?

No
* Closes: apache#39942

Lead-authored-by: Dewey Dunnington <dewey@fishandwhistle.net>
Co-authored-by: Dewey Dunnington <dewey@voltrondata.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
@assignUser assignUser modified the milestones: 16.0.0, 15.0.1 Feb 21, 2024
raulcd pushed a commit that referenced this issue Feb 22, 2024
@paleolimbot @pitrou
@jorisvandenbossche @raulcd
GH-39942: [Python] Make capsule name check more lenient (#39977)
fa2b3c1 
### Rationale for this change

While #39969 fixed the immediate issue caused by the update of the capsule name used by reticulate whilst converting an R "external pointer", it will still result in an error if somebody is using an older version of the Arrow R package.

### What changes are included in this PR?

The pyarrow Cython code was modified to accept capsules with the name NULL or "r_extptr".

### Are these changes tested?

Not sure where the best place for this is, but:

CRAN arrow + released pyarrow + new reticulate (errors):

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.35.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "15.0.0"

reticulate::r_to_py(arrow::int32())
#> PyCapsule_GetPointer called with incorrect name
```

CRAN arrow + pyarrow from this PR + old reticulate:

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.34.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "16.0.0.dev92+geafcff7a5"

reticulate::r_to_py(arrow::int32())
#> DataType(int32)
```

CRAN arrow + pyarrow from this PR + new reticulate:

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.35.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "16.0.0.dev92+geafcff7a5"

reticulate::r_to_py(arrow::int32())
#> DataType(int32)
```

### Are there any user-facing changes?

No
* Closes: #39942

Lead-authored-by: Dewey Dunnington <dewey@fishandwhistle.net>
Co-authored-by: Dewey Dunnington <dewey@voltrondata.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
raulcd pushed a commit that referenced this issue Feb 22, 2024
@paleolimbot @pitrou
@jorisvandenbossche @raulcd
GH-39942: [Python] Make capsule name check more lenient (#39977)
8c69590 
### Rationale for this change

While #39969 fixed the immediate issue caused by the update of the capsule name used by reticulate whilst converting an R "external pointer", it will still result in an error if somebody is using an older version of the Arrow R package.

### What changes are included in this PR?

The pyarrow Cython code was modified to accept capsules with the name NULL or "r_extptr".

### Are these changes tested?

Not sure where the best place for this is, but:

CRAN arrow + released pyarrow + new reticulate (errors):

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.35.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "15.0.0"

reticulate::r_to_py(arrow::int32())
#> PyCapsule_GetPointer called with incorrect name
```

CRAN arrow + pyarrow from this PR + old reticulate:

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.34.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "16.0.0.dev92+geafcff7a5"

reticulate::r_to_py(arrow::int32())
#> DataType(int32)
```

CRAN arrow + pyarrow from this PR + new reticulate:

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.35.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "16.0.0.dev92+geafcff7a5"

reticulate::r_to_py(arrow::int32())
#> DataType(int32)
```

### Are there any user-facing changes?

No
* Closes: #39942

Lead-authored-by: Dewey Dunnington <dewey@fishandwhistle.net>
Co-authored-by: Dewey Dunnington <dewey@voltrondata.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
zanmato1984 pushed a commit to zanmato1984/arrow that referenced this issue Feb 28, 2024
@paleolimbot @pitrou
@jorisvandenbossche @zanmato1984
apacheGH-39942: [Python] Make capsule name check more lenient (apache…
a45a1fb 
…#39977)

### Rationale for this change

While apache#39969 fixed the immediate issue caused by the update of the capsule name used by reticulate whilst converting an R "external pointer", it will still result in an error if somebody is using an older version of the Arrow R package.

### What changes are included in this PR?

The pyarrow Cython code was modified to accept capsules with the name NULL or "r_extptr".

### Are these changes tested?

Not sure where the best place for this is, but:

CRAN arrow + released pyarrow + new reticulate (errors):

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.35.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "15.0.0"

reticulate::r_to_py(arrow::int32())
#> PyCapsule_GetPointer called with incorrect name
```

CRAN arrow + pyarrow from this PR + old reticulate:

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.34.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "16.0.0.dev92+geafcff7a5"

reticulate::r_to_py(arrow::int32())
#> DataType(int32)
```

CRAN arrow + pyarrow from this PR + new reticulate:

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.35.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "16.0.0.dev92+geafcff7a5"

reticulate::r_to_py(arrow::int32())
#> DataType(int32)
```

### Are there any user-facing changes?

No
* Closes: apache#39942

Lead-authored-by: Dewey Dunnington <dewey@fishandwhistle.net>
Co-authored-by: Dewey Dunnington <dewey@voltrondata.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
thisisnic pushed a commit to thisisnic/arrow that referenced this issue Mar 8, 2024
@paleolimbot @pitrou
@jorisvandenbossche @thisisnic
apacheGH-39942: [Python] Make capsule name check more lenient (apache…
5b879a1 
…#39977)

### Rationale for this change

While apache#39969 fixed the immediate issue caused by the update of the capsule name used by reticulate whilst converting an R "external pointer", it will still result in an error if somebody is using an older version of the Arrow R package.

### What changes are included in this PR?

The pyarrow Cython code was modified to accept capsules with the name NULL or "r_extptr".

### Are these changes tested?

Not sure where the best place for this is, but:

CRAN arrow + released pyarrow + new reticulate (errors):

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.35.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "15.0.0"

reticulate::r_to_py(arrow::int32())
#> PyCapsule_GetPointer called with incorrect name
```

CRAN arrow + pyarrow from this PR + old reticulate:

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.34.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "16.0.0.dev92+geafcff7a5"

reticulate::r_to_py(arrow::int32())
#> DataType(int32)
```

CRAN arrow + pyarrow from this PR + new reticulate:

``` r
library(arrow, warn.conflicts = FALSE)
reticulate::use_virtualenv("~/Desktop/rscratch/arrow/.venv")
packageVersion("arrow")
#> [1] '14.0.0.2'
packageVersion("reticulate")
#> [1] '1.35.0'
pa <- reticulate::import("pyarrow")
pa[["__version__"]]
#> [1] "16.0.0.dev92+geafcff7a5"

reticulate::r_to_py(arrow::int32())
#> DataType(int32)
```

### Are there any user-facing changes?

No
* Closes: apache#39942

Lead-authored-by: Dewey Dunnington <dewey@fishandwhistle.net>
Co-authored-by: Dewey Dunnington <dewey@voltrondata.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@paleolimbot paleolimbot

Labels
Component: Continuous Integration Component: Documentation Component: R Type: bug
Projects
None yet
Milestone
15.0.1
Development

Successfully merging a pull request may close this issue.

GH-39942: [Python] Make capsule name check more lenient paleolimbot/arrow
5 participants
@kou @jorisvandenbossche @paleolimbot @assignUser @AlenkaF