Skip to content
Github mirror of "analytics/refinery/source" - our actual code is hosted with Gerrit (please see https://www.mediawiki.org/wiki/Developer_access for contributing
Scala Java Other
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
guard
refinery-camus
refinery-cassandra
refinery-core
refinery-hive
refinery-job
refinery-spark
refinery-tools
.gitignore
.gitmodules
.gitreview
LICENSE.md
README.md
changelog.md
pom.xml

README.md

refinery/source

This repository should contain only source code that is used for building artifacts for the Wikimedia Analytics Refinery.

refinery-core

Core source code for the Wikimeida Analytics Refinery.

refinery-hive

Hive UDFs and other Hive related code.

refinery-tools

Handy Tools used for dealing with Wikimedia Analytics data.

Archiva

WMF uses Archiva as its sole build dependency and deployment artifact repository. You can read more about this here: https://wikitech.wikimedia.org/wiki/Archiva.

Dependencies

All third party dependencies should be mirrored in the WMF Archiva instance at http://archiva.wikimedia.org/repository/mirrored. You can follow the instructions at https://wikitech.wikimedia.org/wiki/Archiva#Uploading_Dependency_Artifacts to upload dependencies to Archiva. If you don't have login access to Archiva, ask a WMF Archiva admin to do this for you.

Development

When working on this repository, building, running tests, or using an IDE, make sure you're using Java 8 (aka Java 1.8). For example, on Ubuntu, you have to use update-alternatives to configure java and javac

Releases and Deployment

Follow the instructions at https://wikitech.wikimedia.org/wiki/Archiva#Deploying_to_Archiva to set up your ~/.m2/settings.xml with proper deployment credentials. Ask a WMF Archiva admin if you don't have these but think you should.

Snapshots

At any given time, the refinery ${project.version} on the master branch should be X.X.X-SNAPSHOT. This will allow you to deploy snapshot builds to Archiva to share with other developers by just running mvn deploy.

Releases

To upload a new version of refinery/source artifacts to Archiva:

First run mvn release:prepare. This will:

  • Check that there are no uncommitted changes in the sources
  • Check that there are no SNAPSHOT dependencies
  • Change the version in the POMs from x-SNAPSHOT to a new version (you will be prompted for the versions to use)
  • Transform the git information in the POM to include the final destination of the tag
  • Run the project tests against the modified POMs to confirm everything is in working order
  • Commit the modified POMs and push
  • Tag the code in the git with a version name
  • Bump the version in the POMs to a new value y-SNAPSHOT (these values will also be prompted for)
  • Commit the modified POMs and push

(Taken from http://maven.apache.org/maven-release/maven-release-plugin/examples/prepare-release.html, see that for more info on the maven release plugin.)

In addition to the above, there will now be release.properties that contains all the information needed to perform the current release. If everything looks good, run mvn release:perform to build and upload your artifacts to Archiva. Note: This might take a while.

If you ran ``mvn release:prepareand then get cold feet, you can abort the release process by runningmvn release:clean```. This will remove the ```release.properties``` file and cause ```mvn release:prepare``` to again prompt you for the to-be-released version number the next time you run it.

You can’t perform that action at this time.