Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process isolated workers always use system default encoding for file names #13843

Closed
ghale opened this issue Jul 17, 2020 · 3 comments · Fixed by #25319
Closed

Process isolated workers always use system default encoding for file names #13843

ghale opened this issue Jul 17, 2020 · 3 comments · Fixed by #25319
Assignees
Milestone

Comments

@ghale
Copy link
Member

ghale commented Jul 17, 2020

When we spawn worker processes when using process isolation with the worker API, the environment of the spawned process is scrubbed. We do set the file.encoding system property by default, but this only affects the contents of files. File names, on the other hand, are interpreted by the underlying native library of the JVM which relies on the locale (e.g. "LANG" environment variable) to define the effective charset. In the absence of any environment variables, it will use whatever the default is for the operating system and file system.

When passing a file property as a parameter and the file name contains UTF-8 characters, if locale environment variables are set for the build process, this can lead to a mismatch between the encoding used when interpreting the filename between the build process and the worker process leading to a FileNotFoundException when the worker process tries to access the file.

Explicitly setting the LANG environment variable in the worker process configuration resolves the issue. So, one solution would be to set the LANG environment variable in worker processes to whatever has been set in the build process.

Expected Behavior

Worker processes should use the same encoding for file names as the build process unless explicitly changed by the user.

Current Behavior

Worker processes may or may not use the same encoding for file names as the build process.

Context

This came up in the context of the Android team configuring their Jacoco task to use the worker api. They are passing the name of the file to instrument as a worker api file property parameter. They have a test that uses non-ASCII characters in the filename. On Linux, the build process has UTF-8 encoding, the worker process gets spawned with a null value for the LANG environment variable, and so the filename gets interpreted with the default encoding, which is something else (might be ASCII, might be ISO-8859, not sure) leading to a FileNotFoundException.

Steps to Reproduce

See 2ce3189

ghale added a commit that referenced this issue Jul 17, 2020
bjornvester added a commit to bjornvester/xjc-gradle-plugin that referenced this issue Mar 17, 2021
…s versions of Java, Gradle and interoperability with the Groovy DSL (and plain Java for that matter).

Also added a work-around for gradle/gradle#13843
bjornvester added a commit to bjornvester/xjc-gradle-plugin that referenced this issue Mar 17, 2021
…s versions of Java, Gradle and interoperability with the Groovy DSL (and plain Java for that matter).

Also added a work-around for gradle/gradle#13843
bjornvester added a commit to bjornvester/xjc-gradle-plugin that referenced this issue Mar 17, 2021
…s versions of Java, Gradle and interoperability with the Groovy DSL (and plain Java for that matter).

Also added a work-around for gradle/gradle#13843
bjornvester added a commit to bjornvester/xjc-gradle-plugin that referenced this issue Mar 17, 2021
…s versions of Java, Gradle and interoperability with the Groovy DSL (and plain Java for that matter).

Also added a work-around for gradle/gradle#13843
bjornvester added a commit to bjornvester/xjc-gradle-plugin that referenced this issue Mar 17, 2021
…s versions of Java, Gradle and interoperability with the Groovy DSL (and plain Java for that matter).

Also added a work-around for gradle/gradle#13843
bjornvester added a commit to bjornvester/xjc-gradle-plugin that referenced this issue Mar 17, 2021
…s versions of Java, Gradle and interoperability with the Groovy DSL (and plain Java for that matter).

Added a work-around for gradle/gradle#13843

Rolled back test for Java 16 due to an issue with Kotlin, which has not yet been fixed in the version that is bundled with Gradle.
bjornvester added a commit to bjornvester/xjc-gradle-plugin that referenced this issue Mar 17, 2021
…s versions of Java, Gradle and interoperability with the Groovy DSL (and plain Java for that matter).

Added a work-around for gradle/gradle#13843

Rolled back test for Java 16 due to an issue with Kotlin, which has not yet been fixed in the version that is bundled with Gradle.
@stale
Copy link

stale bot commented Apr 17, 2022

This issue has been automatically marked as stale because it has not had recent activity. Given the limited bandwidth of the team, it will be automatically closed if no further activity occurs. If you're interested in how we try to keep the backlog in a healthy state, please read our blog post on how we refine our backlog. If you feel this is something you could contribute, please have a look at our Contributor Guide. Thank you for your contribution.

@stale stale bot added the stale label Apr 17, 2022
@stale
Copy link

stale bot commented Jun 12, 2022

This issue has been automatically closed due to inactivity. If you can reproduce this on a recent version of Gradle or if you have a good use case for this feature, please feel free to to let know so we can reopen the issue. Please try to provide steps to reproduce, a quick explanation of your use case or a high-quality pull request.

@stale stale bot closed this as completed Jun 12, 2022
@eskatos eskatos removed the stale label Jun 14, 2022
@eskatos eskatos reopened this Jun 14, 2022
@lptr lptr added this to the 8.3 RC1 milestone Jun 14, 2023
@lptr lptr self-assigned this Jun 14, 2023
@lptr
Copy link
Member

lptr commented Jun 14, 2023

Fixed by #25319

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants