Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Java] Build memory-core and memory-unsafe as JPMS modules #38998

Closed
jduo opened this issue Nov 30, 2023 · 0 comments · Fixed by #39011
Closed

[Java] Build memory-core and memory-unsafe as JPMS modules #38998

jduo opened this issue Nov 30, 2023 · 0 comments · Fixed by #39011
Assignees
@jduo
Labels
Breaking Change Includes a breaking change to the API Component: Java Type: enhancement
Milestone
16.0.0

Comments

@jduo
Copy link
Member

jduo commented Nov 30, 2023 •
edited
Loading

Describe the enhancement requested

This depends on #39134 .

Build memory-core and memory-unsafe as JPMS modules.

  • Refactor the memory module's structures to faciliate this.

These modules use Unsafe and reflection and require adding --add-opens command line flags since they are now named modules.

Note that memory-netty is omitted because it requires a bigger change to the user's command-line.

Component(s)

Java
This was referenced Nov 30, 2023
Open
Open
@github-actions github-actions bot mentioned this issue Nov 30, 2023
Merged
@jduo jduo mentioned this issue Dec 7, 2023
Closed
@lidavidm lidavidm added the Breaking Change Includes a breaking change to the API label Jan 5, 2024
lidavidm pushed a commit that referenced this issue Jan 10, 2024
@jduo
GH-38998: [Java] Build memory-core and memory-unsafe as JPMS modules (#…
e7ab540 
…39011)

Depends on #39134 

### Rationale for this change
Part of modularizing Arrow. Allows these components to be used by tools that require JPMS modules such as jlink.

### What changes are included in this PR?
- Refactor memory modules so that all netty code is in netty-memory.
- All code being injected into Netty packages and have dependencies on Netty's package-private code go into a new module memory-netty-buffer-patch.
- The surefire command line has been changed to allow reflection on unsafe to be used by arrow-memory-core
- Add module-info files for arrow-memory-core and arrow-memory-unsafe

### Are these changes tested?
Yes, existing tests work when run with modules.

### Are there any user-facing changes?
Yes, users now need to put --add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED on their java command line instead of --add-opens=java.base/java.nio=ALL-UNNAMED since memory-core is now a named module instead of unnamed.

 **This PR includes breaking changes to public APIs.** 
Yes, users now need to put --add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED on their java command line instead of --add-opens=java.base/java.nio=ALL-UNNAMED since memory-core is now a named module instead of unnamed.
* Closes: #38998

Authored-by: James Duong <james.duong@improving.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
@lidavidm lidavidm added this to the 16.0.0 milestone Jan 10, 2024
clayburn pushed a commit to clayburn/arrow that referenced this issue Jan 23, 2024
@jduo @clayburn
apacheGH-38998: [Java] Build memory-core and memory-unsafe as JPMS mo…
e4a4604 
…dules (apache#39011)

Depends on apache#39134 

### Rationale for this change
Part of modularizing Arrow. Allows these components to be used by tools that require JPMS modules such as jlink.

### What changes are included in this PR?
- Refactor memory modules so that all netty code is in netty-memory.
- All code being injected into Netty packages and have dependencies on Netty's package-private code go into a new module memory-netty-buffer-patch.
- The surefire command line has been changed to allow reflection on unsafe to be used by arrow-memory-core
- Add module-info files for arrow-memory-core and arrow-memory-unsafe

### Are these changes tested?
Yes, existing tests work when run with modules.

### Are there any user-facing changes?
Yes, users now need to put --add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED on their java command line instead of --add-opens=java.base/java.nio=ALL-UNNAMED since memory-core is now a named module instead of unnamed.

 **This PR includes breaking changes to public APIs.** 
Yes, users now need to put --add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED on their java command line instead of --add-opens=java.base/java.nio=ALL-UNNAMED since memory-core is now a named module instead of unnamed.
* Closes: apache#38998

Authored-by: James Duong <james.duong@improving.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
dgreiss pushed a commit to dgreiss/arrow that referenced this issue Feb 19, 2024
@jduo @dgreiss
apacheGH-38998: [Java] Build memory-core and memory-unsafe as JPMS mo…
396cb1a 
…dules (apache#39011)

Depends on apache#39134 

### Rationale for this change
Part of modularizing Arrow. Allows these components to be used by tools that require JPMS modules such as jlink.

### What changes are included in this PR?
- Refactor memory modules so that all netty code is in netty-memory.
- All code being injected into Netty packages and have dependencies on Netty's package-private code go into a new module memory-netty-buffer-patch.
- The surefire command line has been changed to allow reflection on unsafe to be used by arrow-memory-core
- Add module-info files for arrow-memory-core and arrow-memory-unsafe

### Are these changes tested?
Yes, existing tests work when run with modules.

### Are there any user-facing changes?
Yes, users now need to put --add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED on their java command line instead of --add-opens=java.base/java.nio=ALL-UNNAMED since memory-core is now a named module instead of unnamed.

 **This PR includes breaking changes to public APIs.** 
Yes, users now need to put --add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED on their java command line instead of --add-opens=java.base/java.nio=ALL-UNNAMED since memory-core is now a named module instead of unnamed.
* Closes: apache#38998

Authored-by: James Duong <james.duong@improving.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
zanmato1984 pushed a commit to zanmato1984/arrow that referenced this issue Feb 28, 2024
@jduo @zanmato1984
apacheGH-38998: [Java] Build memory-core and memory-unsafe as JPMS mo…
06a5c3e 
…dules (apache#39011)

Depends on apache#39134 

### Rationale for this change
Part of modularizing Arrow. Allows these components to be used by tools that require JPMS modules such as jlink.

### What changes are included in this PR?
- Refactor memory modules so that all netty code is in netty-memory.
- All code being injected into Netty packages and have dependencies on Netty's package-private code go into a new module memory-netty-buffer-patch.
- The surefire command line has been changed to allow reflection on unsafe to be used by arrow-memory-core
- Add module-info files for arrow-memory-core and arrow-memory-unsafe

### Are these changes tested?
Yes, existing tests work when run with modules.

### Are there any user-facing changes?
Yes, users now need to put --add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED on their java command line instead of --add-opens=java.base/java.nio=ALL-UNNAMED since memory-core is now a named module instead of unnamed.

 **This PR includes breaking changes to public APIs.** 
Yes, users now need to put --add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED on their java command line instead of --add-opens=java.base/java.nio=ALL-UNNAMED since memory-core is now a named module instead of unnamed.
* Closes: apache#38998

Authored-by: James Duong <james.duong@improving.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jduo jduo

Labels
Breaking Change Includes a breaking change to the API Component: Java Type: enhancement
Projects
None yet
Milestone
16.0.0
Development

Successfully merging a pull request may close this issue.

GH-38998: [Java] Build memory-core and memory-unsafe as JPMS modules Bit-Quill/arrow
2 participants
@lidavidm @jduo