Skip to content

[SPARK-56540][INFRA][4.0] Add libuv1-dev to CI Docker images to fix R package fs build failure#55439

Closed
LuciferYang wants to merge 1 commit intoapache:branch-4.0from
LuciferYang:SPARK-56540-4.0
Closed

[SPARK-56540][INFRA][4.0] Add libuv1-dev to CI Docker images to fix R package fs build failure#55439
LuciferYang wants to merge 1 commit intoapache:branch-4.0from
LuciferYang:SPARK-56540-4.0

Conversation

@LuciferYang
Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Add libuv1-dev to the apt-get install list in all five CI Dockerfiles that install R packages:

  • dev/spark-test-image/sparkr/Dockerfile
  • dev/spark-test-image/lint/Dockerfile
  • dev/spark-test-image/docs/Dockerfile
  • dev/infra/Dockerfile
  • dev/create-release/spark-rm/Dockerfile.base

Why are the changes needed?

The R package fs now requires system library libuv (uv.h). fs is a transitive dependency of both rmarkdown (via sassbslib) and testthat (via pkgload).

When the Docker layer cache is valid, the pre-built R packages are reused and everything works fine. But when the cache is invalidated (e.g. base image digest changes), the R packages are compiled from source. Without libuv1-dev installed, fs fails to configure:

Configuration failed because libuv was not found. Try installing:
 * deb: libuv1-dev (Debian, Ubuntu, etc)
...
<stdin>:1:10: fatal error: uv.h: No such file or directory
ERROR: configuration failed for package 'fs'

This triggers a cascade:

fs (missing libuv) → FAIL
  └→ sass → FAIL
       └→ bslib → FAIL
            └→ rmarkdown → FAIL
  └→ pkgload → FAIL
       └→ testthat → FAIL

The image build itself still "succeeds" because install.packages() in R does not return a non-zero exit code when individual packages fail to install -- it logs warnings but continues. So the Docker image gets pushed with rmarkdown and testthat missing, and the SparkR job fails at test time:

Error in library(testthat) : there is no package called 'testthat'
Error: processing vignette 'sparkr-vignettes.Rmd' failed with diagnostics:
there is no package called 'rmarkdown'

Example failure: https://github.com/LuciferYang/spark/actions/runs/24612026450/job/71969572946

Does this PR introduce any user-facing change?

No

How was this patch tested?

  • Pass Github Actions

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code

…age fs build failure

### What changes were proposed in this pull request?
Add `libuv1-dev` to the `apt-get install` list in all five CI Dockerfiles that install R packages:

- `dev/spark-test-image/sparkr/Dockerfile`
- `dev/spark-test-image/lint/Dockerfile`
- `dev/spark-test-image/docs/Dockerfile`
- `dev/infra/Dockerfile`
- `dev/create-release/spark-rm/Dockerfile.base`

### Why are the changes needed?
The R package `fs` now requires system library `libuv` (`uv.h`). `fs` is a transitive dependency of both `rmarkdown` (via `sass` → `bslib`) and `testthat` (via `pkgload`).

When the Docker layer cache is valid, the pre-built R packages are reused and everything works fine. But when the cache is invalidated (e.g. base image digest changes), the R packages are compiled from source. Without `libuv1-dev` installed, `fs` fails to configure:

```
Configuration failed because libuv was not found. Try installing:
 * deb: libuv1-dev (Debian, Ubuntu, etc)
...
<stdin>:1:10: fatal error: uv.h: No such file or directory
ERROR: configuration failed for package 'fs'
```

This triggers a cascade:

```
fs (missing libuv) → FAIL
  └→ sass → FAIL
       └→ bslib → FAIL
            └→ rmarkdown → FAIL
  └→ pkgload → FAIL
       └→ testthat → FAIL
```

The image build itself still "succeeds" because `install.packages()` in R does not return a non-zero exit code when individual packages fail to install -- it logs warnings but continues. So the Docker image gets pushed with `rmarkdown` and `testthat` missing, and the SparkR job fails at test time:

```
Error in library(testthat) : there is no package called 'testthat'
```

```
Error: processing vignette 'sparkr-vignettes.Rmd' failed with diagnostics:
there is no package called 'rmarkdown'
```

Example failure: https://github.com/LuciferYang/spark/actions/runs/24612026450/job/71969572946

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
- Pass Github Actions

### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code

Closes apache#55414 from LuciferYang/fix-sparkr-dockerfile-libuv.

Authored-by: YangJie <yangjie01@baidu.com>
Signed-off-by: yangjie01 <yangjie01@baidu.com>
@LuciferYang
Copy link
Copy Markdown
Contributor Author

cc @sarutak and @yaooqinn

pan3793 pushed a commit that referenced this pull request Apr 21, 2026
… package fs build failure

### What changes were proposed in this pull request?
Add `libuv1-dev` to the `apt-get install` list in all five CI Dockerfiles that install R packages:

- `dev/spark-test-image/sparkr/Dockerfile`
- `dev/spark-test-image/lint/Dockerfile`
- `dev/spark-test-image/docs/Dockerfile`
- `dev/infra/Dockerfile`
- `dev/create-release/spark-rm/Dockerfile.base`

### Why are the changes needed?
The R package `fs` now requires system library `libuv` (`uv.h`). `fs` is a transitive dependency of both `rmarkdown` (via `sass` → `bslib`) and `testthat` (via `pkgload`).

When the Docker layer cache is valid, the pre-built R packages are reused and everything works fine. But when the cache is invalidated (e.g. base image digest changes), the R packages are compiled from source. Without `libuv1-dev` installed, `fs` fails to configure:

```
Configuration failed because libuv was not found. Try installing:
 * deb: libuv1-dev (Debian, Ubuntu, etc)
...
<stdin>:1:10: fatal error: uv.h: No such file or directory
ERROR: configuration failed for package 'fs'
```

This triggers a cascade:

```
fs (missing libuv) → FAIL
  └→ sass → FAIL
       └→ bslib → FAIL
            └→ rmarkdown → FAIL
  └→ pkgload → FAIL
       └→ testthat → FAIL
```

The image build itself still "succeeds" because `install.packages()` in R does not return a non-zero exit code when individual packages fail to install -- it logs warnings but continues. So the Docker image gets pushed with `rmarkdown` and `testthat` missing, and the SparkR job fails at test time:

```
Error in library(testthat) : there is no package called 'testthat'
```

```
Error: processing vignette 'sparkr-vignettes.Rmd' failed with diagnostics:
there is no package called 'rmarkdown'
```

Example failure: https://github.com/LuciferYang/spark/actions/runs/24612026450/job/71969572946

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
- Pass Github Actions

### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code

Closes #55439 from LuciferYang/SPARK-56540-4.0.

Authored-by: YangJie <yangjie01@baidu.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
@pan3793
Copy link
Copy Markdown
Member

pan3793 commented Apr 21, 2026

thanks, merged to branch-4.0

@pan3793 pan3793 closed this Apr 21, 2026
@LuciferYang
Copy link
Copy Markdown
Contributor Author

Thank you @pan3793 @sarutak @yaooqinn and @HyukjinKwon

@LuciferYang LuciferYang deleted the SPARK-56540-4.0 branch April 22, 2026 06:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants