Provide support for custom plugin repositories #528

piotrtomiak · 2020-09-04T09:45:09Z

Building Vue.j plugin from sources on master using gradle requires IDEA nightly builds. It was really cumbersome to get everything to compile, so here is PR with support for nightly builds. Example configuration:

repositories {
  mavenCentral()
  maven("https://jetbrains.bintray.com/intellij-third-party-dependencies")
  maven("https://jetbrains.bintray.com/jediterm")
  maven("https://cache-redirector.jetbrains.com/www.myget.org/F/rd-snapshots/maven")
}

intellij {
  pluginName = "Vue.js"
  type = "IU"
  updateSinceUntilBuild = false
  nightlyBuild = "https://..."
  setPlugins("JavaScriptLanguage", "JSIntentionPowerPack", "JavaScriptDebugger", "CSS", "HtmlTools",
             "org.jetbrains.plugins.sass", "org.jetbrains.plugins.less", "org.jetbrains.plugins.stylus",
             "org.intellij.plugins.postcss:${ideaDependency.buildNumber}",
             "com.jetbrains.plugins.Jade:${ideaDependency.buildNumber}",
             "intellij.prettierJS:${ideaDependency.buildNumber}")
}

dependencies {
  testImplementation("com.jetbrains.intellij.javascript:javascript-test-framework:${intellij.ideaDependency.buildNumber}")
  testImplementation("com.jetbrains.intellij.copyright:copyright:${intellij.ideaDependency.buildNumber}")
}

You can of course fix the URL to a particular nightly build. Tested runIde target, which works perfectly.

piotrtomiak · 2020-09-04T09:46:57Z

@zolotov I haven't bumped plugin version nor added documentation. Let me know if any of these is needed.

zolotov · 2020-09-04T09:57:24Z

Thanks. It looks like a solution for #15

I have concerns about the API, though:

it adds yet another option to intellij extension. ideally I'd get rid of the extension completely, I believe all the settings should be configured in particular tasks or in the dependencies block
it reveals intellij build server specifics and is not discoverable or usable for external users
there is also intellij.pluginsRepo settings which is somehow related to the changes

Probably, it makes sense to convert intellij.pluginsRepo into an array setting and automatically detect whether it's an plugins.xml-based repository or maven based (e.g. by downloading plugins.xml files every time).

zolotov · 2020-09-04T09:58:27Z

I haven't bumped plugin version nor added documentation. Let me know if any of these is needed.

No problem, I usually do that right before publishing. I also think that feature should go with 0.5.0

piotrtomiak · 2020-09-04T10:01:18Z

@zolotov I understand all of your concerns, especially point #2. The intellij.pluginsRepo requires Maven repository, that's why I didn't want to reuse it. Let me know if you think we should change something or abandon the idea.

zolotov · 2020-09-04T10:05:55Z

Let me know if you think we should change something or abandon the idea.

I've posted the suggestion :)

Probably, it makes sense to convert intellij.pluginsRepo into an array setting and automatically detect whether it's an plugins.xml-based repository or maven based (e.g. by downloading plugins.xml files every time).

piotrtomiak · 2020-09-04T10:33:25Z

intellij.pluginsRepo into an array setting and automatically detect whether it's an plugins.xml-based repository or maven based

OK, sounds good and more generic! That would however solve just a half of the problem. The other is downloading the latest IDE build. Should we do the same and simply detect if we are dealing with nightly?

zolotov · 2020-09-04T10:57:46Z

The other is downloading the latest IDE build. Should we do the same and simply detect if we are dealing with nightly?

Sorry, didn't get the problem. Could you elaborate,please?

piotrtomiak · 2020-09-04T11:51:35Z

@zolotov Nevermind. I replied too quickly. I'll look into applying your suggestions next week. Do you have an example of a corporate repo layout, so that I can support that as well?

zolotov · 2020-09-04T14:10:46Z

@piotrtomiak as far as I understand, artifacts of nightly teamcity builds is an example of corporate repo layout, so it's just an updatePlugins.xml file and plugin distribution nearby.

piotrtomiak · 2020-09-07T11:25:33Z

@zolotov I've been trying to figure out the best way to keep backward compatibility and provide support for the corporate layout. How about something like this:

intellij {
  ...
  pluginsRepo {
     marketplace() // maven("https://cache-redirector.jetbrains.com/plugins.jetbrains.com/maven")
     maven("https://corpo.net/intellij/maven")
     custom("https://corpo.net/plugins/intellij/updatePlugins.xml")
     custom("https://other-corpo.org/plugins/ij/plugins.xml")
  }
  ...
}

Now, the question is about intellijRepo property and nightlies. Should we require an url to a nightly build snapshot, or should we figure that out from the list of plugins repos. In the first case, an example configuration for nightly would be something like this:

val nightlyBuildUrl = "https://..."
intellij {
  ...
  intellijRepo = "${nightlyBuildUrl}"
  pluginsRepo {
     maven("${nightlyBuildUrl}/maven-artifacts")
     custom("${nightlyBuildUrl}/IU-plugins/plugins.xml")
  }
  ...
}

Let me know what you think!

zolotov · 2020-09-08T16:19:41Z

How about something like this:

Love it

Should we require an url to a nightly build snapshot

I don't think url should be required. It's possible to use local directory as an intellij build without a nightly build. Probably I misunderstood the suggestion.

UPD: ah, you want to download Installers right from the buildserver too. Is there a need for that? From the JB network you can access to them without patching an intellijRepo url.

piotrtomiak · 2020-09-08T17:33:37Z

Love it

Awesome! I'll go with it than :)

src/main/groovy/org/jetbrains/intellij/Utils.groovy

zolotov

LGTM, would be nice to have a test for custom repository. see org.jetbrains.intellij.DownloadIntelliJPluginsSpec

src/main/groovy/org/jetbrains/intellij/IntelliJPluginExtension.groovy

piotrtomiak · 2020-09-09T12:16:36Z

LGTM, would be nice to have a test for custom repository. see org.jetbrains.intellij.DownloadIntelliJPluginsSpec

I'll need to include an xml and single jar in a test directory, unless you are aware of a public and stable custom plugin repository. Sounds good?

zolotov · 2020-09-09T12:53:53Z

I'll need to include an xml and single jar in a test directory, unless you are aware of a public and stable custom plugin repository. Sounds good?

@piotrtomiak sure

Also, upgrade documentr to newer version, as the old one fails to generate output with cryptic message. Fixes JetBrains#15

zolotov · 2020-09-10T09:21:16Z

Thank you very much! Merged. I’ll publish 0.5 as soon as I investigate problems with kotlin-serialization 1.4

Undin · 2020-09-13T10:44:37Z

Awesome!
@piotrtomiak thank you!
Сan't wait for these changes in the release

rachelcarmena · 2020-09-22T12:41:30Z

Hi! Please, is this feature deployed? Thanks in advance!

zolotov · 2020-09-22T13:00:19Z

it's in 0.5-SNAPSHOT only so far, will publish as soon as #537 gets fixed

zolotov · 2020-09-22T13:00:52Z

here is the instruction on how to use snapshot version https://github.com/JetBrains/gradle-intellij-plugin#snapshot-version

rachelcarmena · 2020-09-22T13:29:08Z

Awesome, thank you so much @zolotov 🙌

Gradle works and accepts the new DSL however It's not finding the plugin.

I'm following this schema: https://jetbrains.org/intellij/sdk/docs/basics/getting_started/update_plugins_format.html#format-of-updatepluginsxml-file

though the test is using this schema: https://github.com/piotrtomiak/gradle-intellij-plugin/blob/9e6031e05d9ad5656c934410c497eba5aab88c7f/src/test/resources/custom-repo/plugins.xml

Please, did I check the wrong documentation for the private repository? Thanks in advance!!

zolotov · 2020-09-22T18:19:06Z

Please, did I check the wrong documentation for the private repository?

That is the right documentation.

Probably running gradle with --info will give some clues. Please attach the output.

cc @piotrtomiak

rachelcarmena · 2020-09-23T13:23:48Z

Thank you so much @zolotov !

No clues in the console output with the different Gradle options.

After:

[gradle-intellij-plugin :ide-plugin] Configuring plugin io.arrow-kt.arrow:1.4.10-SNAPSHOT

it appears:

Could not determine the dependencies of task ':ide-plugin:prepareSandbox'.
> Cannot resolve plugin io.arrow-kt.arrow version 1.4.10-SNAPSHOT

and I think the reason is looking for download-url:

        String downloadUrl = pluginsXml.category."idea-plugin"
                .find { it.id.text().equalsIgnoreCase(id) && it.version.text() == version }
                ?."download-url"?.text()

and nothing happens if it's not found:

        if (downloadUrl == null) return null

download-url doesn't exist in the schema that I followed: https://jetbrains.org/intellij/sdk/docs/basics/getting_started/update_plugins_format.html#format-of-updatepluginsxml-file

Thanks again!!

rachelcarmena · 2020-09-23T13:25:44Z

And <idea-plugin> doesn't exist but <plugin>.

piotrtomiak · 2020-09-23T13:28:40Z

@rachelcarmena It turns out that I was following a different schema. I will update the code to support both.

rachelcarmena · 2020-09-23T13:31:08Z

That would be awesome @piotrtomiak , thank you so much!! 👏 💃

…Brains#528

piotrtomiak requested a review from zolotov September 4, 2020 09:46

piotrtomiak force-pushed the nightly-support branch from d8f8bec to b3ea92d Compare September 9, 2020 10:23

piotrtomiak changed the title ~~Provide support for IDEA nightly builds through "nightlyBuild" property~~ Provide support for custom plugin repositories Sep 9, 2020

piotrtomiak force-pushed the nightly-support branch 2 times, most recently from 8f8e6a6 to de9c1d5 Compare September 9, 2020 10:45

zolotov reviewed Sep 9, 2020

View reviewed changes

src/main/groovy/org/jetbrains/intellij/Utils.groovy Outdated Show resolved Hide resolved

zolotov requested changes Sep 9, 2020

View reviewed changes

src/main/groovy/org/jetbrains/intellij/IntelliJPluginExtension.groovy Outdated Show resolved Hide resolved

Provide support for corporate repositories.

9e6031e

Also, upgrade documentr to newer version, as the old one fails to generate output with cryptic message. Fixes JetBrains#15

piotrtomiak force-pushed the nightly-support branch from de9c1d5 to 9e6031e Compare September 9, 2020 15:55

piotrtomiak requested a review from zolotov September 9, 2020 18:36

zolotov merged commit d4604b5 into JetBrains:master Sep 10, 2020

piotrtomiak added a commit to piotrtomiak/gradle-intellij-plugin that referenced this pull request Sep 28, 2020

Provide support for custom plugin repositories in official format Jet…

59957be

…Brains#528

zolotov pushed a commit that referenced this pull request Oct 1, 2020

Provide support for custom plugin repositories in official format #528

9f7a1fb

zolotov added this to the 0.5.0 milestone Oct 1, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Provide support for custom plugin repositories #528

Provide support for custom plugin repositories #528

piotrtomiak commented Sep 4, 2020 •

edited

Loading

piotrtomiak commented Sep 4, 2020

zolotov commented Sep 4, 2020 •

edited

Loading

zolotov commented Sep 4, 2020

piotrtomiak commented Sep 4, 2020

zolotov commented Sep 4, 2020

piotrtomiak commented Sep 4, 2020

zolotov commented Sep 4, 2020

piotrtomiak commented Sep 4, 2020

zolotov commented Sep 4, 2020

piotrtomiak commented Sep 7, 2020 •

edited

Loading

zolotov commented Sep 8, 2020 •

edited

Loading

piotrtomiak commented Sep 8, 2020

zolotov left a comment

piotrtomiak commented Sep 9, 2020

zolotov commented Sep 9, 2020

zolotov commented Sep 10, 2020

Undin commented Sep 13, 2020

rachelcarmena commented Sep 22, 2020

zolotov commented Sep 22, 2020

zolotov commented Sep 22, 2020

rachelcarmena commented Sep 22, 2020

zolotov commented Sep 22, 2020

rachelcarmena commented Sep 23, 2020

rachelcarmena commented Sep 23, 2020

piotrtomiak commented Sep 23, 2020

rachelcarmena commented Sep 23, 2020

Provide support for custom plugin repositories #528

Provide support for custom plugin repositories #528

Conversation

piotrtomiak commented Sep 4, 2020 • edited Loading

piotrtomiak commented Sep 4, 2020

zolotov commented Sep 4, 2020 • edited Loading

zolotov commented Sep 4, 2020

piotrtomiak commented Sep 4, 2020

zolotov commented Sep 4, 2020

piotrtomiak commented Sep 4, 2020

zolotov commented Sep 4, 2020

piotrtomiak commented Sep 4, 2020

zolotov commented Sep 4, 2020

piotrtomiak commented Sep 7, 2020 • edited Loading

zolotov commented Sep 8, 2020 • edited Loading

piotrtomiak commented Sep 8, 2020

zolotov left a comment

Choose a reason for hiding this comment

piotrtomiak commented Sep 9, 2020

zolotov commented Sep 9, 2020

zolotov commented Sep 10, 2020

Undin commented Sep 13, 2020

rachelcarmena commented Sep 22, 2020

zolotov commented Sep 22, 2020

zolotov commented Sep 22, 2020

rachelcarmena commented Sep 22, 2020

zolotov commented Sep 22, 2020

rachelcarmena commented Sep 23, 2020

rachelcarmena commented Sep 23, 2020

piotrtomiak commented Sep 23, 2020

rachelcarmena commented Sep 23, 2020

piotrtomiak commented Sep 4, 2020 •

edited

Loading

zolotov commented Sep 4, 2020 •

edited

Loading

piotrtomiak commented Sep 7, 2020 •

edited

Loading

zolotov commented Sep 8, 2020 •

edited

Loading