Scala Common Enrich: add JavaScript scripting enrichment

[alexanderdean]

Title was: allow support for custom JS logic to write any field based on any other field

This came out of:

• https://groups.google.com/forum/#!topic/snowplow-user/ZL_EA-Pk4YE
• Allow certain params to be set via QS vars snowplow-javascript-tracker#42

The basic idea is: we add a point where JavaScript can be evaluated in our Scalding ETL to update any field in our CanonicalOutput based on the contents of any other field.

This should be the last thing done, when all other values are set.

This would allow some pretty custom stuff to be done without overloading the trackers - e.g. extracting specific values from the referer URI.

For other JS scripting ideas see also:

• Placeholder for kinesis-tee: filters/routes events using JavaScript #352

[jrobgood]

Another use case for this: we wrote a system tray app a few years back and wanted to log end-user behavior without the app communicating directly to WebTrends (our client's preferred platform.) The desire was to replay the log events from the app's calls to our servers and send the end user IP with the events.

Therefore, relaying/proxying events via the pixel request even down to the UA IP might be a nice piece of functionality to have here.

[alexanderdean]

@jrobgood not sure if I quite follow - but you can pass the end user's IP on the Tracker Protocol with the &ip= parameter. This was added to support proxying.

Actually, we don't have an equivalent for end user's useragent. We should add that - see #381.

[jrobgood]

Sorry, I did not see that documented in https://github.com/snowplow/snowplow/wiki/snowplow-tracker-protocol so I assumed it was not permitted. So to be clear: I can push the keyword 'ip' into the querystring for the pixel request, and the ETL job will overwrite the reported IP from the GET request in the Cloudfront log with what I pass in?

[alexanderdean]

@jrobgood - correct:

https://github.com/snowplow/snowplow/blob/master/3-enrich/hadoop-etl/src/main/scala/enrichments/EnrichmentManager.scala#L135

I think it's an undocumented feature currently - we'll get it added to the documentation.

[alexanderdean]

Updated design for this: we don't want to mutate the enriched event from JS, but rather to allow the addition of custom contexts generated from JavaScript.