Add windows JNI support #44257

erip · 2020-09-06T00:27:43Z

dr-ci · 2020-09-06T00:38:38Z

💊 CI failures summary and remediations

As of commit e62256e (more details on the Dr. CI page):

4/4 failures possibly* introduced in this PR
- 2/4 non-CircleCI failure(s)---

2 failures not recognized by patterns:

Job	Step	Action
^{docker-pytorch-linux-xenial-py3-clang5-android-ndk-r19c}	^{Check if image should be built}	🔁 rerun
^{pytorch_linux_xenial_py3_6_gcc5_4_build}	^Build	🔁 rerun

🚧 1 ongoing upstream failure:

These were probably caused by upstream breakages that are not fixed yet:

docker-pytorch-linux-xenial-py3-clang5-android-ndk-r19c since Oct 14
- 🔁 rerun

ci.pytorch.org: 2 failed

Failed: pr/caffe2-pytorch-linux-bionic-rocm3.8-py3.6-test
Failed: pr/pytorch-linux-bionic-rocm3.8-py3.6

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 157 times.

erip · 2020-09-06T13:40:52Z

I'm not positive if there's something special I need to do to ensure a JDK is installed on the CI runners, but the usual suspects on Windows (C:\Program Files\Java, C:\Program Files\AdoptOpenJDK) aren't resolving to JAVA_HOMEs. 😬

.circleci/scripts/binary_populate_env.sh

mszhanyi

sorry, ignore it

mszhanyi

Fixes #44363

erip · 2020-09-17T10:36:19Z

~~This is not quite ready, so I'd remove approval @mszhanyi. The fbjni folks have also been responding, so this may be moving a bit more soon. 🙌~~

mszhanyi · 2020-09-17T10:45:04Z

I'm not positive if there's something special I need to do to ensure a JDK is installed on the CI runners, but the usual suspects on Windows (C:\Program Files\Java, C:\Program Files\AdoptOpenJDK) aren't resolving to JAVA_HOMEs. 😬
No problem. I just don't find how to remove me from the reviewer list.

erip · 2020-09-25T01:57:16Z

Lingering TODOs:

conditionally link optional libraries on Windows.
Find out why libtorch build is failing on Linux. 🤔

erip · 2020-09-25T13:49:02Z

cc @dreiss do these fbjni errors look familiar to you? They they all look related and this is the history of SimpleFixedString.h - not sure why it only fails in these limited cases... It seems like SimpleFixedString is claiming that the strings aren't null-terminated which is causing an (intentional) compilation error.

Edit: I think the fix is to update Meta.h L306 like what's demonstrated in this godbolt: https://godbolt.org/z/sYffn6

WDYT @dreiss?

erip · 2020-09-25T17:11:04Z

@mszhanyi I don't imagine this PR is going to change dramatically by fixing the remaining two build errors, so if you'd like to review it I think this PR is quite close.

erip · 2020-09-25T17:57:00Z

Waiting for this PR to be merged.

erip · 2020-09-26T15:21:16Z

I think these errors are a fixture of flaky builds and upstream breakages. I'll try to rebuild a bit later.

erip · 2020-09-26T18:55:20Z

5600e82 is a temporary commit while we wait for the commits to land in fbjni.

peterjc123

These changes LGTM.

codecov · 2020-09-27T18:22:10Z

Codecov Report

Merging #44257 into master will decrease coverage by 0.04%.
The diff coverage is 60.31%.

@@            Coverage Diff             @@
##           master   #44257      +/-   ##
==========================================
- Coverage   68.31%   68.26%   -0.05%     
==========================================
  Files         410      410              
  Lines       53582    53614      +32     
==========================================
- Hits        36602    36598       -4     
- Misses      16980    17016      +36

Impacted Files	Coverage Δ
torch/nn/quantized/modules/__init__.py	`97.22% <ø> (ø)`
torch/testing/_internal/expecttest.py	`78.57% <ø> (+1.02%)`	⬆️
...orch/testing/_internal/distributed/rpc/rpc_test.py	`26.39% <14.28%> (-0.07%)`	⬇️
.../testing/_internal/distributed/rpc/jit/rpc_test.py	`28.97% <18.18%> (-0.17%)`	⬇️
torch/futures/__init__.py	`84.61% <50.00%> (-2.89%)`	⬇️
torch/jit/annotations.py	`91.78% <50.00%> (-0.39%)`	⬇️
torch/serialization.py	`86.87% <75.00%> (+0.56%)`	⬆️
torch/quantization/fx/quantize.py	`96.24% <90.90%> (-0.17%)`	⬇️
torch/optim/_multi_tensor/adam.py	`94.87% <100.00%> (ø)`
torch/optim/_multi_tensor/adamw.py	`93.50% <100.00%> (ø)`
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2d6fd22...e62256e. Read the comment docs.

erip · 2020-10-13T22:36:37Z

@peterjc123 @mszhanyi I think this is good once CI passes. WDYT?

android/pytorch_android/CMakeLists.txt

…glesource.com/platform/libnativehelper/+/brillo-m9-dev/include/nativehelper/jni.h) with one difference: JNIEXPORT is defined per the Windows JDK.

peterjc123

Again, this LGTM.

peterjc123 · 2020-10-15T13:47:39Z

Please merge this pr. @ezyang @malfet

facebook-github-bot

@ezyang has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

erip · 2020-10-15T15:25:11Z

These errors seem to be related to git funniness because of a rebase - seems to be a fluke and should be OK to merge.

facebook-github-bot · 2020-10-15T18:14:06Z

@ezyang merged this pull request in b547973.

pytorchbot added the open source label Sep 6, 2020

peterjc123 reviewed Sep 7, 2020

View reviewed changes

.circleci/scripts/binary_populate_env.sh Outdated Show resolved Hide resolved

erip mentioned this pull request Sep 10, 2020

no pytorch_jni in java.library.path windows #44363

Closed

mszhanyi self-requested a review September 17, 2020 03:05

mszhanyi reviewed Sep 17, 2020

View reviewed changes

mszhanyi requested review from mszhanyi and removed request for mszhanyi September 17, 2020 03:23

mszhanyi approved these changes Sep 17, 2020

View reviewed changes

mszhanyi requested review from mszhanyi and removed request for mszhanyi September 17, 2020 10:45

erip changed the title ~~[WIP] Add windows JNI support~~ Add windows JNI support Sep 25, 2020

zou3519 added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Sep 25, 2020

erip requested a review from mszhanyi September 25, 2020 17:14

erip requested a review from peterjc123 September 26, 2020 17:43

peterjc123 approved these changes Sep 27, 2020

View reviewed changes

erip mentioned this pull request Sep 28, 2020

add fix for GCC. facebookincubator/fbjni#40

Closed

mszhanyi approved these changes Oct 1, 2020

View reviewed changes

t-vi reviewed Oct 15, 2020

View reviewed changes

android/pytorch_android/CMakeLists.txt Outdated Show resolved Hide resolved

erip and others added 7 commits October 15, 2020 09:34

pull most recent fbjni commits.

4d3951a

add the JNI as primarily defined by Android (see: https://android.goo…

9df0a10

…glesource.com/platform/libnativehelper/+/brillo-m9-dev/include/nativehelper/jni.h) with one difference: JNIEXPORT is defined per the Windows JDK.

use the custom JNI in CI.

1bdd0e1

conditionally link optional libraries for Windows linking.

b2f9f1a

skip windows-jni workaround on non-Windows builds.

cf2af85

add most recent fbjni.

b6fe841

replace find and append with consistent USE_* flags

e62256e

peterjc123 approved these changes Oct 15, 2020

View reviewed changes

facebook-github-bot reviewed Oct 15, 2020

View reviewed changes

facebook-github-bot closed this in b547973 Oct 15, 2020

erip mentioned this pull request Oct 15, 2020

*NNPACK are required to build Java bindings #43909

Closed

erip deleted the feature/add-windows-jni-support branch October 15, 2020 18:12

facebook-github-bot added the Merged label Oct 15, 2020

erip mentioned this pull request Oct 16, 2020

[META] Assert that expected libraries appear in libtorch #46464

Open

erip mentioned this pull request Jan 14, 2021

update to reflect merged Java runtime on Windows pytorch/pytorch.github.io#559

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add windows JNI support #44257

Add windows JNI support #44257

erip commented Sep 6, 2020 •

edited

dr-ci bot commented Sep 6, 2020 •

edited by facebook-github-bot

erip commented Sep 6, 2020 •

edited

mszhanyi left a comment •

edited

mszhanyi left a comment

erip commented Sep 17, 2020 •

edited

mszhanyi commented Sep 17, 2020

erip commented Sep 25, 2020 •

edited

erip commented Sep 25, 2020 •

edited

erip commented Sep 25, 2020

erip commented Sep 25, 2020

erip commented Sep 26, 2020

erip commented Sep 26, 2020 •

edited

peterjc123 left a comment

codecov bot commented Sep 27, 2020 •

edited

erip commented Oct 13, 2020

peterjc123 left a comment

peterjc123 commented Oct 15, 2020

facebook-github-bot left a comment

erip commented Oct 15, 2020 •

edited

facebook-github-bot commented Oct 15, 2020

Add windows JNI support #44257

Add windows JNI support #44257

Conversation

erip commented Sep 6, 2020 • edited

dr-ci bot commented Sep 6, 2020 • edited by facebook-github-bot

💊 CI failures summary and remediations

2 failures not recognized by patterns:

🚧 1 ongoing upstream failure:

ci.pytorch.org: 2 failed

erip commented Sep 6, 2020 • edited

mszhanyi left a comment • edited

Choose a reason for hiding this comment

mszhanyi left a comment

Choose a reason for hiding this comment

erip commented Sep 17, 2020 • edited

mszhanyi commented Sep 17, 2020

erip commented Sep 25, 2020 • edited

erip commented Sep 25, 2020 • edited

erip commented Sep 25, 2020

erip commented Sep 25, 2020

erip commented Sep 26, 2020

erip commented Sep 26, 2020 • edited

peterjc123 left a comment

Choose a reason for hiding this comment

codecov bot commented Sep 27, 2020 • edited

Codecov Report

erip commented Oct 13, 2020

peterjc123 left a comment

Choose a reason for hiding this comment

peterjc123 commented Oct 15, 2020

facebook-github-bot left a comment

Choose a reason for hiding this comment

erip commented Oct 15, 2020 • edited

facebook-github-bot commented Oct 15, 2020

erip commented Sep 6, 2020 •

edited

dr-ci bot commented Sep 6, 2020 •

edited by facebook-github-bot

erip commented Sep 6, 2020 •

edited

mszhanyi left a comment •

edited

erip commented Sep 17, 2020 •

edited

erip commented Sep 25, 2020 •

edited

erip commented Sep 25, 2020 •

edited

erip commented Sep 26, 2020 •

edited

codecov bot commented Sep 27, 2020 •

edited

erip commented Oct 15, 2020 •

edited