Support multi-plugin project

[zolotov]

There is a workaround for that: use the same sandbox directory in each module and make :prepareSandbox tasks depend on each other. But this way requires a lot of non-obvious configuration. There should be a way to do it easier.

[patflynn]

Hi Alexander,

How do you think this should work? I've found it's a bit tricky to use the proposed strategy because prepareSandbox from multiple plugins pointed at the same folder seems to blow away the depended on plugin (I am no gradle expert so maybe I configure the dependency wrong).

I was looking into how we could define it in the context of this plugin and was wondering what the best model might be. At first I thought allowing project(':dep-plugin') as one of the inputs to intellij.plugins would be the obvious solution, but I'm starting to think a ideaPlugin 'dependencies' configuration might fit the Gradle model a little better.

e.g.

intellij {
   pluginName = 'my-plugin'
}

dependencies {
   ideaPlugin 'junit'
   ideaPlugin 'Groovy'
   ideaPlugin(project(':my-dep-plugin')
}

Thoughts?

[zolotov]

Hi Patrick,

The point of suggested strategy is that if you have several plugin sub-project, then prepareSandbox tasks will populate shared sandbox directory with plugins from each module. And I think it should work in 0.0.29 version (I've published it a moment ago)

I don't think it will work for projects that depend on other plugin-subprojects. Do you really need this?

As for 'junit' and 'Groovy' plugin dependencies, please take a look at readme, you can attach it with intellij.plugins ['junit', 'Groovy']

[patflynn]

Hey Alexander,

Thanks for getting back to me. The 0.0.29 update does fix the issue of the plugin sub-projects cleaning out the full sandbox directory so that you couldn't effectively share the same directory across dependent plugin sub-projects for running. Thanks!

I did see you can declare the intellij sdk packaged plugins as deps using intellij.plugins, which is how we've tried to configure out plugin here:
https://github.com/GoogleCloudPlatform/gcloud-intellij/blob/move_to_gradle_intellij_plugin/core-plugin/build.gradle

This works fine. I was just wondering, if you eventually support multiple plugin sub-projects with dependencies between them, how you would declare those dependencies in the build.gradle. We've used the normal Java deps way of dependencies { compile(project('dep-plugin') }, but that has the unfortunate consequence of including the dependent plugin as a jar in the plugins lib during packaging, which breaks things in the class loader when you runIdea. (It's a shame Gradle doesn't have an included 'provided' scope!)

My initial snippet (from my first comment) is a suggestion to add a special intellij plugin dep configuration, and if you did so you could also move your intellij.plugins declaration to use the same mechanism. But really any way to fix the problem including being able to declare sub-project deps in the intellij extension would be welcome.

Thanks again for working on this plugin! It's exactly what we needed.

[zolotov]

We've used the normal Java deps way of dependencies { compile(project('dep-plugin') }, but that has the unfortunate consequence of including the dependent plugin as a jar in the plugins lib during packaging, which breaks things in the class loader when you runIdea.

I see. Well, I think I can handle it too. And if I fail, I'll try to implement the custom syntax you suggested. Give me couple more days and I'll return to this.

Thank you for feedback. I'm very glad that you're using this plugin. Actually I didn't expect that someone will use it until its "official" announcement or until documentation is published :)

[patflynn]

Ha! Sorry about that. If you build it, they will come. :)

You're addressing a pressing need for us.

[patflynn]

FYI one thing to keep in mind is how to deal with the third party classes exported in a Plugin API. Without an explicit Gradle provided scope it's tricky not to accidentally package those jars with the client plugin, which would cause the same classloader issues as bundling the dependent plugin jar in the client plugin package.

[zolotov]

Please try 0.0.31. Now all dependency projects that looks like intellij plugins and its dependencies won't be included in target distribution

[zolotov]

@patflynn hi! How is it going? Does 0.0.31 version works for you?

[patflynn]

@zolotov I've tried this on our project, and it does look as though our plugin dep is no longer bundled into the lib, however the shared dependencies are still there. Without an explicit way to declare which third party libraries are exported as part of the dependent plugin API, it's not clear to me how you would know what not to package.

[zolotov]

@patflynn And who provides these libraries in runtime? A 'parent' plugin? Please describe a simple project layout to recreate the issue and I'll try to handle it.

[zolotov]

@patflynn I see that you worked it around with following code:

project.configurations.provided.each {
        intellij.intellijFiles.add(it)
}

It seems that I can do the same in plugin. Let's say that we use only runtime and compile dependencies for distribution. What do you think?

[patflynn]

Makes sense to me. It's a shame that gradle doesn't have provided as a
first class concept.

On Mon, Nov 30, 2015 at 5:52 PM, Alexander Zolotov <notifications@github.com

wrote:

@patflynn https://github.com/patflynn I see that you worked it around
with following code:

project.configurations.provided.each {
intellij.intellijFiles.add(it)
}

It seems that I can do the same in plugin. Let's say that we use only
runtime and compile dependencies for distribution. What do you think?

—
Reply to this email directly or view it on GitHub
#17 (comment)
.

[zolotov]

@patflynn but the question is still open. How user's IDE will get google.gdt.eclipse.login.common.jar and google-api-java-client-min-repackaged-1.20.0.jar in runtime?

[patflynn]

The simplified scenario is this:

Plugin B:
includes C.jar
C.jar includes a class C
Plugin B has a public class B which includes a public method getC() which
returns an object of type C

Plugin A:
Depends on Plugin B
Plugin A has a class A which calls Plugin B's B.getC() and uses the
returned C object for whatever.

For Plugin A to compile it needs to include C.jar on it's compile
classpath. However if C.jar is included in its lib at runtime on the
IntelliJ platform, IntelliJ will complain that the type returned by
B.getC() is not the same as the type of the C it is assigning to (class
cast exception I think). I believe this is because Plugin A and Plugin B
are using two different classloaders.

The way we addressed this is by removing C.jar from Plugin A's library and
relying on the fact that IntelliJ will look for classes in Plugin B if
they're not found in Plugin A since Plugin A 'depends on' Plugin B.

This is not a robust solution but it works, and I'm not aware of any
support in IntelliJ for anything better (but you may know better!).

On Tue, Dec 1, 2015 at 10:33 AM, Alexander Zolotov <notifications@github.com

wrote:

@patflynn https://github.com/patflynn but the question is still open.
How user's IDE will get google.gdt.eclipse.login.common.jar and
google-api-java-client-min-repackaged-1.20.0.jar in runtime?

—
Reply to this email directly or view it on GitHub
#17 (comment)
.

[zolotov]

I believe this is because Plugin A and Plugin B are using two different classloaders.

That's right.

The way we addressed this is by removing C.jar from Plugin A's library and relying on the fact that IntelliJ will look for classes in Plugin B if they're not found in Plugin A since Plugin A 'depends on' Plugin B.

Sure, if B plugin has C.jar in its /lib directory and A depends on B, then classes from C.jar will be available in A. Actually IDEA uses exactly the same classloader for B classes and classes from /lib/*.jars.
So there is no needed to explicitly add C.jar as a dependency for project A. I believe you can just do something like:

runtime project(':google-account-plugin')

Offtopic:

I've looked at gcloud intellij plugin and found some issues. First of all, AppEngineUpdateDialog has unicode symbols, but source encoding is ASCII, I think you should to add to build.gradle something like

tasks.withType(JavaCompile) { options.encoding = 'UTF-8' }

Also, the plugin depends on Git4Idea and this is quite strange for gcloud plugin. I think it could be better to have Git4Idea as an optional dependency. To do that just replace <depends>Git4Idea</depends> with <depends optional="true" config-file="my-git-features.xml">Git4Idea</depends> and create my-git-features.xml file with describing git-specific extensions.

In the third. IDEA provides junit, commons-io and guava dependencies, no need to explicitly depend on them.

[patflynn]

Thank you for taking a look at our project, and sorry for the slow
response!

On Tue, Dec 1, 2015 at 1:25 PM, Alexander Zolotov notifications@github.com
wrote:

I believe this is because Plugin A and Plugin B are using two different
classloaders.

That's right.

The way we addressed this is by removing C.jar from Plugin A's library and
relying on the fact that IntelliJ will look for classes in Plugin B if
they're not found in Plugin A since Plugin A 'depends on' Plugin B.

Sure, if B plugin has C.jar in its /lib directory and A depends on B, then
classes from C.jar will be available in A. Actually IDEA uses exactly the
same classloader for B classes and classes from /lib/*.jars.
So there is no needed to explicitly add C.jar as a dependency for project
A. I believe you can just do something like:

runtime project(':google-account-plugin')

runtime did not work, but I guess that makes sense as I think that
precludes it from being on the classpath at compile time. But you were
right that it works if I turn off transitive deps for the modules.

Offtopic:

I've looked at gcloud intellij plugin and found some issues. First of all,
AppEngineUpdateDialog has unicode symbols, but source encoding is ASCII,
I think you should to add to build.gradle something like

tasks.withType(JavaCompile) { options.encoding = 'UTF-8' }

Good to know. I kind of assumed UTF-8 was the default.

Also, the plugin is depends on Git4Idea and this is quite strange for
gcloud plugin. I think it could be better to have Git4Idea as an optional
dependency. To do that just replace Git4Idea with
<depends

optional="true" config-file="my-git-features.xml">Git4Idea and
create my-git-features.xml file with describing git-specific extensions.

This we're aware of. I think we'll eventually tease out our dependencies
so that we can target a wider set of plugins. For now we're focused on CE
and UE and I believe they both have Git4Idea. I did not add this dep
myself so I'll probably need to take a look at some point and see why it's
there.

In the third. IDEA provides junit, commons-io and guava dependencies, no
need to explicitly depend on them.

Yeah we were aware, but hadn't cleaned things up, this actually ended up
biting us today because of conflicting versions between our deps and IDEA
SDKS apache HttpClient.

—
Reply to this email directly or view it on GitHub
#17 (comment)
.

[zolotov]

@patflynn btw, gradle-intellij-plugin is able to add version to plugin.xml file (https://github.com/GoogleCloudPlatform/gcloud-intellij/blob/master/build.gradle#L57)

[patflynn]

Nice!

[aslo]

It appears that @patflynn 's earlier issue - that prepareSandbox blows away the entire sandbox directory when building multiple dependant plugins - exists once again in version 0.1.9. This is preventing us from updating the Google Cloud Platform's build to the latest version of this plugin.

@zolotov Any advice would be much appreciated, thanks!

[zolotov]

@aslo thanks, I'm sure I had a test for this, I'll take a look anyway. How exactly can I recreate this on Google Cloud Platform plugin?

[aslo]

You can recreate by pulling the upgrade-ij-gradleplugin branch in the GoogleCloudPlatform/gcloud-intellij repo. See also this pull request. This project builds two plugins, and if you run the prepareSandbox task, you'll notice that during the second plugin's build, the prior plugin is removed from the build/idea-sandbox/plugins/ dir.

Thanks for taking a look! Let me know if you think there's something on our end that needs to change.

[zolotov]

@aslo cannot recreate, that's what I got on prepareSandbox task:

tree -L 3 ./plugins
./plugins
`-- google-cloud-tools
    |-- META-INF
    |   `-- plugin.xml
    |-- classes
    |   |-- com
    |   |-- config.properties
    |   |-- generation
    |   |-- icons
    |   `-- messages
    `-- lib
        |-- GoogleFeedback.jar
        |-- annotations-2.0.3.jar
        |-- appengine-plugins-core-0.1.0.jar
        |-- common-lib-0.9.7-beta-SNAPSHOT.jar
        |-- google-api-services-clouddebugger-v2-rev7-1.21.0.jar
        |-- google-api-services-cloudresourcemanager-v1beta1-rev12-1.21.0.jar
        |-- google-api-services-source.jar
        |-- hamcrest-core-1.3.jar
        |-- joda-time-2.9.2.jar
        |-- jsr305-3.0.1.jar
        `-- junit-4.12.jar

[zolotov]

Or, I see, both of them should be available in sandbox. Recreated then

[zolotov]

Temporary fix was implemented in 0.1.10. The proper solution will come up with 0.2.0

[aslo]

Thanks for your help! I really appreciate it.

[zolotov]

Since 0.2-SNAPSHOT it's possible to include other plugin in runtime via intellij { plugins = [project('mySecondPlugin')] }.

The second way is to patch prepareSandbox task like this:

dependencies {
    compile project('mySecondPlugin') // if needed
}

prepareSandbox {
    def task = project.tasks.getByPath(':mySecondPlugin:prepareSandbox')
    from(new File(task.destinationDir, task.pluginName)) {
        into task.pluginName
    }
    dependsOn task
}