Dependencies should be pulled during gradle execution time

[nathanabercrombie]

As a best practice, dependencies should be pulled during gradle execution time. This means that this plugin should not download the huge dependency files during configuration.

Downloading at configuration time is bad practice because when a large project configures, it configures every module, regardless of what it is going to run. So if I have a large gradle project, and one of the modules has this plugin applied, but I am building a different module, it will still pull the dependency. This does not scale well at all for large projects.

[zolotov]

As far as I understand in this case none of the tools (IDEs, Review Tools) which relies on gradle project model will be able to properly handle such dependencies. This is unacceptable for this plugin. If you aware of how to deal with it – contributions are welcome.

[nathanabercrombie]

I have not read through the implementation for this project, but the way that plugins should handle injecting dependencies is by:

1. Adding the repository for the dependencies during configuration (plugin.apply)
   project.repositories.maven { url YOURURL }
2. After determining which dependencies, adding them during configuration
   project.dependencies { compile "com.intellij.org:blah:blah" }

Then, if this project needs to be compiled (based on the task given to gradle), only then will the dependencies get pulled. All tools that are based on gradle will work fine. I have built a plugin this way, and it is the recommended way (unfortunately it is closed source, so I cannot show you it).

I'll take a look at this plugin's source when I get the chance.

[zolotov]

@nathanabercrombie thanks for the answer. IntelliJ dependencies are not the maven dependency

What the plugin does during the configuration:

1. it downloads idea.zip, which is the build of particular IDEA version
2. it extracts zip, create local ivy repo with the list of jar (from idea.zip) which should be used as dependencies
3. attaches ivy dependecy via project.dependencies { compile "com.intellij.org:blah:blah" }

So it cannot attach dependencies with project.dependencies { compile "com.intellij.org:blah:blah" } until it downloads idea.zip

[nathanabercrombie]

I suspected that was what was happening. The only solution to this is if you host the ivy repo remotely. I am not sure what Idea's policies are on uploading those jars to something like maven central, but it wouldn't be hard to script it.

This is a deal breaker for my project. I have a work around though: my company provides an internal repository host, so I may just upload the required jars there. There are other really cool features of this plugin, so I was hoping to use it.

[zolotov]

I have a work around though: my company provides an internal repository host, so I may just upload the required jars there

If this a solution for you then you can upload IDEA zips to your internal maven repository. You can use distributions and pom-files from https://www.jetbrains.com/intellij-repository/releases or https://www.jetbrains.com/intellij-repository/snapshots. Then substitute repo host url via intellij.intellijRepo extension property. By default it is https://www.jetbrains.com/intellij-repository.

There are other really cool features of this plugin, so I was hoping to use it.

What kind of features do you mean? I think I can extract them so you'll be able to use them without downloading IDEA distribution.

[nathanabercrombie]

Hey, I figured out a pretty elegant solution. This passes my test of:

• gradle tasks should not download any compile time dependencies
• gradle assemble should compile with no issues

I did this experiment with the java plugin, but I am sure it can be adapted to your plugin

repositories {
    // We pull from the same repository as before
    maven { url "https://www.jetbrains.com/intellij-repository/releases" }
}

configurations {
    // We add a new "configuration" which lets us do some crazy magic
    ideaPluginPrep
}

// Of course, get ideaVersion however you want
ext.ideaVersion = "145.2070.6"

dependencies {
    // Our new configuration is dependent on the specified idea version
    ideaPluginPrep "com.jetbrains.intellij.idea:ideaIC:$ideaVersion"

    // And here is the magic.  The compile dependency depends on all jars found in 'lib' directory
    compile fileTree("$buildDir/ideaLib/$ideaVersion/lib") {
        // and that lib directory is created by this other task
        builtBy 'prepareIntellijLibraryDirectory'
    }
}

// This task unzips the zip downloaded into the build/ideaLib/$version directory
task prepareIntellijLibraryDirectory {
    doLast {
        // the dereference of configuration.ideaPluginPrep must be done in an action, not in the configuration
        // So we put all this in the doLast block
        copy {
            // configurations.ideaPluginPrep.asPath will include all the dependencies defined above, so this
            // may not work if you include multiple dependencies in ideaPluginPrep above.
            from file(configurations.ideaPluginPrep.asPath)
            outputDir = file("${buildDir}/ideaLib/$ideaVersion")
        }
    }
}

[nathanabercrombie]

What kind of features do you mean?

Hopefully you can integrate the above solution, but, the features I was referring to are:

• Automatic modification / packing of plugin.xml
• Publishing to idea repo
• runIdea task and all that good stuff
• any new features you add in the future

[zolotov]

compile.builtBy dependency looks promising, thank you! I'll try to implement something similar.

Hopefully you can integrate the above solution, but, the features I was referring to are:

I'm going to reimplement all tasks one by one as part of #110 issue. So they will be available without IDEA dependency, the will have cleaner configurations (via task extension, not via intellij extension). In the 2.0-SNAPSHOT I reimplemented PublishTask. Building, packing plugin, running IDEA and modification plugin.xml tasks are on the plan.

[zolotov]

I failed to implement it so far. A suggested solution with a fileTree dependency won't allow to attach sources.
Another approach with resolving ivy-dependency on config.resolutionStrategy.eachDependency still leads to downloading idea during gradle tasks task.
I postpone the issue for a while

[raniejade]

Any ETA on this?

[zolotov]

@raniejade no, contributions are welcome

[raniejade]

I don't think this will be solved without adding a new feature into gradle, is there any particular reason we cant publish to a maven repo the jars in idea.zip?

[zolotov]

@raniejade Alexey well described the situation in the related issue at Gradle issue tracker

[oehme]

I thought about this a bit more and there is a new hook that seems like it could give you what you need: Configuration.withDependencies allows you to lazily create dependencies only when that Configuration is actually needed. So you could create your custom repository in such a withDependencies hook. This exists since Gradle 4.6 I think.

[zolotov]

@oehme sounds promising! I'll try it and will tell about the results.

[DreierF]

Has there been any progress on this?

[zolotov]

@DreierF there is zolotov/delay_dependency_resolving branch, you're very welcome to help with it

[raniejade]

@zolotov does this mean that this is fixed? 🤞

[zolotov]

@raniejade yes, should be available in 0.5-SNAPSHOT. In order to see the stacktrace of dependency resolution, run it with --debug

[raniejade]

So good, thanks @zolotov!