Flex deployment fails for Windows 10 users. #937

patflynn · 2016-09-27T13:36:27Z

OS: Windows 10
Java: 1.8.0_112-release
IJ: 2016.2.4
plugin: 1.0-BETA-0.2

NONE_
error.message
null
error.stacktrace
java.lang.UnsupportedOperationException
at java.nio.file.Files.setPosixFilePermissions(Files.java:2044)
at com.google.cloud.tools.intellij.appengine.cloud.AppEngineFlexibleStage.stage(AppEngineFlexibleStage.java:63)
at com.google.cloud.tools.intellij.appengine.cloud.AppEngineFlexibleDeployTask.execute(AppEngineFlexibleDeployTask.java:68)
at com.google.cloud.tools.intellij.appengine.cloud.AppEngineExecutor.run(AppEngineExecutor.java:37)
at com.intellij.openapi.application.TransactionGuardImpl$6.run(TransactionGuardImpl.java:291)
at com.intellij.openapi.application.impl.LaterInvocator$FlushQueue.runNextEvent(LaterInvocator.java:326)
at com.intellij.openapi.application.impl.LaterInvocator$FlushQueue.run(LaterInvocator.java:310)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
at com.intellij.ide.IdeEventQueue.defaultDispatchEvent(IdeEventQueue.java:793)
at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.java:629)
at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:385)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

patflynn · 2016-09-27T13:36:41Z

Alex can you take a look and check out the severity of this?

aslo · 2016-09-27T14:02:20Z

Sure, I can take a look this afternoon.

loosebazooka · 2016-09-27T14:24:58Z

This is coming from the common library though. Are we not detecting windows
correctly?

On Sep 27, 2016 10:02, "Alex Sloan" notifications@github.com wrote:

Sure, I can take a look this afternoon.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#937 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABPo-ot1WB0Da4hh-1r5ditY5dCQMQC7ks5quSHtgaJpZM4KHrKQ
.

joaoandremartins · 2016-09-27T20:02:11Z

fyi, haven't been able to stage Flex successfully on Windows yet. tried on a gce instance and the test laptop.

patflynn · 2016-09-27T20:09:00Z

With this error?

On Tue, Sep 27, 2016 at 4:02 PM, João Martins notifications@github.com
wrote:

fyi, haven't been able to stage Flex successfully on Windows yet. tried on
a gce instance and the test laptop.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#937 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHf5HRaXJTUAWaBGBSWVCuHlL-qMupTWks5quXZEgaJpZM4KHrKQ
.

joaoandremartins · 2016-09-27T20:17:28Z

Yes

patflynn · 2016-09-27T20:21:37Z

Do the maven and gradle plugins work on Windows 10?

On Tue, Sep 27, 2016 at 4:17 PM, João Martins notifications@github.com
wrote:

Yes

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#937 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHf5Haqxf69B5Yor5XRfOvTl73HYG1I2ks5quXnZgaJpZM4KHrKQ
.

joaoandremartins · 2016-09-27T20:38:43Z

Possibly, I doubt they're calling Files.setPosixFilePermissions().
https://github.com/GoogleCloudPlatform/google-cloud-intellij/blob/fa4eef8c8d2ffcb147c83c733533601b07784d19/google-cloud-tools-plugin/src/com/google/cloud/tools/intellij/appengine/cloud/AppEngineFlexibleStage.java#L63

What's causing this on Windows is that we're trying to get a PosixFileAttributeView from WindowsFileSystemProvider, and it can't return it.
http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/sun/nio/fs/WindowsFileSystemProvider.java#156

I don't know of another way to set permissions on a file using Files. I might try to not set any permissions and see if that works.

joaoandremartins · 2016-09-27T21:51:44Z

In fact, this is failing for ALL windows users. Files.setPosixFilePermissions doesn't work in Windows, and I haven't been able to discover a way to modify file permissions on Windows, without having to run console commands..

I just verified that the copied file has owner and group read, which should be enough for our case, unless I'm missing anything?

etanshaul · 2016-09-27T21:53:15Z

What determines the default permissions? And is it system dependent?

joaoandremartins · 2016-09-27T22:00:48Z

As I understand, in java.nio there isn't a way to specify file permissions at copy time, you need to do it after copying, through Files.setPosixFilePermissions. However, this doesn't work on Windows, and I don't know of any other way other than running system commands.
Files.copy takes CopyOptions that don't seem to be what we need.

I removed the setPosixFilePermissions call on Windows and the whole deployment succeeded. Originally, I had only introduced the call to assure backwards compatibility with the previous call. However, unless there is an edge case that I can't see right now, we don't need this call.

joaoandremartins · 2016-09-27T22:07:00Z

FWIW, on Windows, after the file is copied, SYSTEM, Administrators and the current user are given full control of the file. I'm running as a machine administrator, but for any user that has similar permissions to run a deployment, the same should still apply.

patflynn · 2016-09-27T22:15:46Z

We did push this down:
https://github.com/GoogleCloudPlatform/appengine-plugins-core/blob/master/src/main/java/com/google/cloud/tools/appengine/cloudsdk/CloudSdkAppEngineFlexibleStaging.java

But doesn't that beg the question why it works for Maven/Gradle?

On Tue, Sep 27, 2016 at 6:07 PM, João Martins notifications@github.com
wrote:

FWIW, on Windows, after the file is copied, SYSTEM, Administrators and the
current user are given full control of the file. I'm running as a machine
administrator, but for any user that has similar permissions to run a
deployment, the same should still apply.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#937 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHf5HbNiKvTBLfDxpFDA_Fb7OnSwOTHSks5quZOFgaJpZM4KHrKQ
.

etanshaul · 2016-09-27T22:18:33Z

What I meant in my previous comment is that intellij isn't consuming this yet.

patflynn · 2016-09-27T22:20:38Z

Aha! In any case, both codepaths would share the same problem.

On Tue, Sep 27, 2016 at 6:18 PM, Etan Shaul notifications@github.com
wrote:

What I meant in my previous comment is that intellij isn't consuming this
yet.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#937 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHf5Hbj0BCJb9paQOQBIJZD4mFZ5VXWiks5quZY6gaJpZM4KHrKQ
.

etanshaul · 2016-09-27T22:22:08Z

Yeah, I'd be surprised if they didn't since the common lib is also using the posix permission setter mechanism.

joaoandremartins · 2016-09-27T22:22:26Z

It appears that someone on the library side was cunning enough to add a check:

        if (!System.getProperty("os.name").contains("Windows")) {
          Set<PosixFilePermission> permissions = Sets.newHashSet();
          permissions.add(PosixFilePermission.OWNER_READ);
          permissions.add(PosixFilePermission.OWNER_WRITE);
          permissions.add(PosixFilePermission.GROUP_READ);
          permissions.add(PosixFilePermission.OTHERS_READ);

          Files.setPosixFilePermissions(destination, permissions);
        }

I can go ahead and replicate this on our side? Although I think I was just able to run Jetty without it

etanshaul · 2016-09-27T22:23:27Z

ah yes. I didn't notice that check. Perhaps in the interim, we should do the same thing in Intellij, and then get rid of this code once we defer to the common-lib for flex staging

etanshaul · 2016-09-27T22:25:13Z

I do have an issue to track that too: #717

joaoandremartins · 2016-09-27T22:27:54Z

I wasn't able to reproduce any issue without this piece of code, running the war on Jetty.
I would prefer to disable such unsightly code instead of adding it and YAGNI

patflynn · 2016-09-27T22:30:38Z

+1

On Tue, Sep 27, 2016 at 6:25 PM, Etan Shaul notifications@github.com
wrote:

I do have an issue to track that too: #717
#717

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#937 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHf5Hdh04r3aCXUgsKu0N10iyf69PqFqks5quZfKgaJpZM4KHrKQ
.

etanshaul · 2016-09-27T22:31:37Z

Ok. @chanseokoh can you elaborate on your comment about jetty and the file persmissions relating to this to make sure we aren't missing something? is this still reproducible?

joaoandremartins · 2016-09-27T22:34:55Z

I was just able to reproduce the issue @chanseokoh was referring to. When the app is deployed to Flex, no instances will be spun up. Thanks @chanseokoh !

joaoandremartins · 2016-09-27T22:53:34Z

Actually, the version was stopped, which is why I couldn't reach the endpoint..
Deploying a pure Flex war artifact with -rw-r----- is allowing me to get good responses.

joaoandremartins · 2016-09-27T22:56:28Z

Unless @chanseokoh can still reproduce the issue with a different setup to mine, I suggest to go ahead with this code, and possibly delete the other one from the library.

etanshaul · 2016-09-28T02:12:14Z

STGM

On Tue, Sep 27, 2016 at 6:56 PM João Martins notifications@github.com
wrote:

Unless @chanseokoh https://github.com/chanseokoh can still reproduce
the issue with a different setup to mine, I suggest to go ahead with this
code, and possibly delete the other one from the library.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#937 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABp8QFDYrt5xwGMw1buhTbYnqEuW3Pvxks5quZ8cgaJpZM4KHrKQ
.

chanseokoh · 2016-09-28T03:03:37Z

I just noticed that flex deploy works fine without the others read permission. I did some investigation and learned something funny:

This permission issue in the flex runtime image is more or less "fixed" by myself, inadvertently. When I was fixing the broken Cloud Debugger in the flex image, I decided to explicitly unpack a war (which is done by root) and change the owner of the unpacked contents to the user jetty (PR here). Never knew at that time this would also take care of the permission issue.

patflynn · 2016-09-28T15:40:32Z

But what about users of custom images? Couldn't they still have this issue?

On Wed, Sep 28, 2016 at 10:57 AM, João Martins notifications@github.com
wrote:

Closed #937
#937
via #938
#938.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#937 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHf5HUHooCzEoXBbTd1dZm-Vb7CApvDKks5quoBEgaJpZM4KHrKQ
.

patflynn · 2016-09-29T01:10:09Z

For posterity: GoogleCloudPlatform/appengine-plugins#224 describes why this code change is safe.

anhld4 · 2016-10-05T17:32:16Z

Hello, Can i fix this bug with plugin 1.0-BETA-0.3 ?

patflynn · 2016-10-05T17:33:30Z

@anhld4 unfortunately not. We have a release planned tomorrow that'll include a fix for this.

### What is this PR for? Fixing ZEPPELIN-2382: A fresh install of Zeppelin 0.7.1 failed to start on a Windows due to the use of [getPosixFilePermissions](https://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#getPosixFilePermissions(java.nio.file.Path,%20java.nio.file.LinkOption...)) This method fails on Windows with an UnsupportedOperationException - however, recovering (that is ignoring the exception) appears to have no negative effects. (For a discussion on this related to a different project see, e.g., GoogleCloudPlatform/cloud-code-intellij#937 ) ### What type of PR is it? [Bug Fix] ### Todos * [x] - Have the binary distribution checked by the reporter of [ZEPPELIN-2382] ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2382 ### How should this be tested? Check if clean distribution runs on Windows. Important: Ensure that HADOOP_HOME points to a folder containing bin/winutils.exe since this pull request does not include a fix for https://issues.apache.org/jira/browse/ZEPPELIN-2438 (which is part of a separate pull request). ### Screenshots (if appropriate) N/A ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Christian Fries <email@christian-fries.de> Closes #2305 from cfries/windows-compatiblity-interpreter-binding and squashes the following commits: e3b5738 [Christian Fries] Added graceful exception handling if posix permission is not available 096fed6 [Christian Fries] Merge branch 'master' into apache/master 3635ed8 [Christian Fries] [ZEPPELIN-2375]: Avoid modification of CLASSPATH variable. (cherry picked from commit fb80bf7) Signed-off-by: Lee moon soo <moon@apache.org>

### What is this PR for? Fixing ZEPPELIN-2382: A fresh install of Zeppelin 0.7.1 failed to start on a Windows due to the use of [getPosixFilePermissions](https://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#getPosixFilePermissions(java.nio.file.Path,%20java.nio.file.LinkOption...)) This method fails on Windows with an UnsupportedOperationException - however, recovering (that is ignoring the exception) appears to have no negative effects. (For a discussion on this related to a different project see, e.g., GoogleCloudPlatform/cloud-code-intellij#937 ) ### What type of PR is it? [Bug Fix] ### Todos * [x] - Have the binary distribution checked by the reporter of [ZEPPELIN-2382] ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2382 ### How should this be tested? Check if clean distribution runs on Windows. Important: Ensure that HADOOP_HOME points to a folder containing bin/winutils.exe since this pull request does not include a fix for https://issues.apache.org/jira/browse/ZEPPELIN-2438 (which is part of a separate pull request). ### Screenshots (if appropriate) N/A ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Christian Fries <email@christian-fries.de> Closes #2305 from cfries/windows-compatiblity-interpreter-binding and squashes the following commits: e3b5738 [Christian Fries] Added graceful exception handling if posix permission is not available 096fed6 [Christian Fries] Merge branch 'master' into apache/master 3635ed8 [Christian Fries] [ZEPPELIN-2375]: Avoid modification of CLASSPATH variable.

### What is this PR for? Fixing ZEPPELIN-2382: A fresh install of Zeppelin 0.7.1 failed to start on a Windows due to the use of [getPosixFilePermissions](https://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#getPosixFilePermissions(java.nio.file.Path,%20java.nio.file.LinkOption...)) This method fails on Windows with an UnsupportedOperationException - however, recovering (that is ignoring the exception) appears to have no negative effects. (For a discussion on this related to a different project see, e.g., GoogleCloudPlatform/cloud-code-intellij#937 ) ### What type of PR is it? [Bug Fix] ### Todos * [x] - Have the binary distribution checked by the reporter of [ZEPPELIN-2382] ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2382 ### How should this be tested? Check if clean distribution runs on Windows. Important: Ensure that HADOOP_HOME points to a folder containing bin/winutils.exe since this pull request does not include a fix for https://issues.apache.org/jira/browse/ZEPPELIN-2438 (which is part of a separate pull request). ### Screenshots (if appropriate) N/A ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Christian Fries <email@christian-fries.de> Closes apache#2305 from cfries/windows-compatiblity-interpreter-binding and squashes the following commits: e3b5738 [Christian Fries] Added graceful exception handling if posix permission is not available 096fed6 [Christian Fries] Merge branch 'master' into apache/master 3635ed8 [Christian Fries] [ZEPPELIN-2375]: Avoid modification of CLASSPATH variable.

patflynn added the kind/bug label Sep 27, 2016

patflynn assigned aslo Sep 27, 2016

patflynn changed the title ~~User reported error~~ Flex deployment fails for Windows 10 users. Sep 27, 2016

patflynn added high priority google-cloud-tools-plugin area/app-engine labels Sep 27, 2016

patflynn added this to the 1.0-BETA-0.3 milestone Sep 27, 2016

joaoandremartins mentioned this issue Sep 27, 2016

Remove setPosixFilePermissions. #938

Merged

joaoandremartins closed this as completed in #938 Sep 28, 2016

joaoandremartins mentioned this issue Sep 28, 2016

Remove unnecessary code. GoogleCloudPlatform/appengine-plugins#224

Merged

joaoandremartins assigned joaoandremartins and unassigned aslo Sep 28, 2016

patflynn modified the milestones: 1.0-GA, 1.0-BETA-0.3 Oct 5, 2016

chanseokoh mentioned this issue Oct 12, 2016

Intelij plugin deploy to App engine: UnsupportedOperationException: null GoogleCloudPlatform/getting-started-java#136

Closed

cfries mentioned this issue Apr 30, 2017

[ZEPPELIN-2382] Fixed windows compatibility of interpreter binding. apache/zeppelin#2305

Closed

1 task

muga mentioned this issue Apr 20, 2018

Remove setPosixFilePermissions method call for Windows treasure-data/digdag#765

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Flex deployment fails for Windows 10 users. #937

Flex deployment fails for Windows 10 users. #937

patflynn commented Sep 27, 2016

patflynn commented Sep 27, 2016

aslo commented Sep 27, 2016

loosebazooka commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

patflynn commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

patflynn commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

etanshaul commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

patflynn commented Sep 27, 2016

etanshaul commented Sep 27, 2016

patflynn commented Sep 27, 2016

etanshaul commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016 •

edited by chanseokoh

Loading

etanshaul commented Sep 27, 2016

etanshaul commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

patflynn commented Sep 27, 2016

etanshaul commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

etanshaul commented Sep 28, 2016

chanseokoh commented Sep 28, 2016 •

edited

Loading

patflynn commented Sep 28, 2016

patflynn commented Sep 29, 2016

anhld4 commented Oct 5, 2016

patflynn commented Oct 5, 2016

Flex deployment fails for Windows 10 users. #937

Flex deployment fails for Windows 10 users. #937

Comments

patflynn commented Sep 27, 2016

patflynn commented Sep 27, 2016

aslo commented Sep 27, 2016

loosebazooka commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

patflynn commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

patflynn commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

etanshaul commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

patflynn commented Sep 27, 2016

etanshaul commented Sep 27, 2016

patflynn commented Sep 27, 2016

etanshaul commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016 • edited by chanseokoh Loading

etanshaul commented Sep 27, 2016

etanshaul commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

patflynn commented Sep 27, 2016

etanshaul commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

joaoandremartins commented Sep 27, 2016

etanshaul commented Sep 28, 2016

chanseokoh commented Sep 28, 2016 • edited Loading

patflynn commented Sep 28, 2016

patflynn commented Sep 29, 2016

anhld4 commented Oct 5, 2016

patflynn commented Oct 5, 2016

joaoandremartins commented Sep 27, 2016 •

edited by chanseokoh

Loading

chanseokoh commented Sep 28, 2016 •

edited

Loading