Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Ingestion Data Mapping Language

Build Status



IDML has been developed and maintained by DataSift (now a Meltwater company), where it's been used to deliver high throughput social media firehoses and to allow customers to onboard data easily.

Releasing to Sonatype

If you're a member of the io.idml organisation on sonatype you can perform a release with these commands:

  1. add default-key $KEYID to ~/.gnupg/gpg.conf to specify the key to release with
  2. ++ publishSigned
  3. sonatypePrepare
  4. sonatypeBundleUpload
  5. sonatypeRelease
  6. project tool then assembly to build the tool jar
  7. get the tool jar from ./tool/target/scala-2.12/idml-tool-$VERSION-assembly.jar
  8. create a release on github with the format $VERSION: Title of Release
  9. Upload the assembly jar to that release and write release notes

Special Thanks

This project has had many contributors before being open sourced, these include:

  • Andi Miller
  • Jon Davey
  • Stuart Dallas
  • Courtney Robinson
  • James Bloomer