Use Update Sites as Bundle Source

[jGleitz]

I am very sorry if I’m missing a major point here. But I’ve read all of this project’s wiki pages and tried a lot of things, but still can figure it out:

In our MANIFEST.MF, we require the Apache Commons:

Require-Bundle: org.apache.commons.lang3;bundle-version="3.1.0"

If we try to build that using wuff, we get:

* What went wrong:
Could not resolve all dependencies for configuration ':Prototypes/Eclipse Build:compile'.
> Could not find eclipse-luna-sr2:org.apache.commons.lang3:3.1.0.
  Searched in the following locations:
      https://repo1.maven.org/maven2/eclipse-luna-sr2/org.apache.commons.lang3/3.1.0/org.apache.commons.lang3-3.1.0.pom
      https://repo1.maven.org/maven2/eclipse-luna-sr2/org.apache.commons.lang3/3.1.0/org.apache.commons.lang3-3.1.0.jar
      file:/home/josh/.wuff/m2_repository/eclipse-luna-sr2/org.apache.commons.lang3/3.1.0/org.apache.commons.lang3-3.1.0.pom
      file:/home/josh/.wuff/m2_repository/eclipse-luna-sr2/org.apache.commons.lang3/3.1.0/org.apache.commons.lang3-3.1.0.jar
  Required by:
      Beagle:Prototypes/Eclipse Build:0.0.1

Now on one hand it’s totally clear what’s happening here: lang3 is not part of a default Eclipse installation. But: How do we solve this?
We do not want to include the jar of the Apache Commons in our project, because we believe this is not how it’s supposed to work. We want to declare the dependency in MANIFEST.ML.

I expected to be able to point wuff to a target platform definition so it can retrieve further plugins from there, but I found no such option?

How do we point wuff where to get this plugin from?

This might be related to #75 and #98

[jGleitz]

For anybody interested, I managed to add update sites to look for dependencies like this: (heavily inspired by #9):

buildscript {
    repositories {
        mavenLocal()
        jcenter()
    }

    dependencies {
        classpath group: 'org.akhikhl.wuff', name: 'wuff-plugin', version: '+'
        classpath group: 'de.undercouch', name: 'gradle-download-task', version: '+'
        classpath group: 'org.apache.ivy', name: 'ivy', version: '+' 
    }
}

import org.akhikhl.wuff.EclipseBundlePlugin
import de.undercouch.gradle.tasks.download.DownloadTaskPlugin
import org.apache.ivy.util.url.ApacheURLLister

apply plugin: EclipseBundlePlugin
apply plugin: DownloadTaskPlugin

def lister = new ApacheURLLister()
def downloadUpdatesite

/**
 * Recursivly downloads an update site.
 */
downloadUpdatesite = { URL url, destinationDir ->
    file(destinationDir).mkdirs()
    project.download {
        src lister.listFiles(url)
        dest destinationDir
    }
    for (folder in lister.listDirectories(url)) {
        def destPart = url.toURI().relativize(folder.toURI()).toString()
        downloadUpdatesite folder, "$destinationDir/$destPart" 
    }
}

/**
 * Downloads an update site and provides a path to it to be used by wuff.
 * 
 * @param url The update site’s url.
 */
def updatesite = { url ->
    // create a folder name out of the URL
    def dirName = url.replaceAll("[^a-zA-Z0-9.-]", "_")
    def wuffDir = project.wuff.wuffDir ?: System.getProperty('user.home') + '/.wuff'

    // imitate unpuzzle’s checksum mechanism
    def checksumFile = file("$wuffDir/downloaded-checksums/${dirName}.md5")
    checksumFile.parentFile.mkdirs()
    def destinationDir = "$wuffDir/unpacked/$dirName"

    if (!checksumFile.exists() || !file(destinationDir).isDirectory()) {
        downloadUpdatesite new URL(url), destinationDir
        // unpuzzle’s dummy checksum
        checksumFile.text = 'deadbea1'
    }
    return "file://$destinationDir"
}

project.wuff {
    selectedEclipseVersion = '4.5'

    eclipseVersion('4.5') {
        sources {
            /*
             * Configure all update sites to get dependency plugins from here.
             */
            source updatesite("https://sdqweb.ipd.kit.edu/eclipse/palladiosimulator/nightly/aggregate/")
        }
    }
}

It simply downloads the whole update site and makes it available to wuff.

Obviously, this is far from perfect. If adding update sites for bundle sources would be supported (or somebody told me how to do this) I’d be very happy.

[izreal]

@jGleitz your solution works pretty well when your P2 update site exposes index information for lister to consume. However, many update sites do not return index information and we fail with a "Please provide a download source" message.

For instance http://download.eclipse.org/eclipse/updates/4.5 fails with your code.

[jGleitz]

@izreal: I know that. But there is even a hack for the hack!

• You need to get the site’s artefacts.jar. In the case of http://download.eclipse.org/eclipse/updates/4.5, you can hit “Show Directory Contents” to navigate to R-4.5.1-201509040015/artefacts.jar
• download and unpack the jar by hand
• Open the contained artefacts.xml and search for a property p2.mirrorsURL.
• Open the referenced XML file. For the Mars 4.5 Updates Site it’s http://www.eclipse.org/downloads/download.php?format=xml&file=/eclipse/updates/4.5/R-4.5.1-201509040015
• Pick one mirror from the list there that supports directory listing (most do, so I always pick one that’s close to me)

I know that this is super dirty – but it works.