Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scala Kinesis Enrich: fix error when fetching MaxMind file from s3:// URI #1645

Closed
alexanderdean opened this issue Apr 29, 2015 · 3 comments
Closed
Assignees

Comments

@alexanderdean
Copy link
Member

Exception in thread "main" java.net.MalformedURLException: unknown protocol: s3
    at java.net.URL.<init>(URL.java:596)
    at java.net.URL.<init>(URL.java:486)
    at java.net.URL.<init>(URL.java:435)
    at java.net.URI.toURL(URI.java:1096)
    at com.snowplowanalytics.snowplow.enrich.kinesis.KinesisEnrichApp$$anonfun$12.apply(KinesisEnrichApp.scala:152)
    at com.snowplowanalytics.snowplow.enrich.kinesis.KinesisEnrichApp$$anonfun$12.apply(KinesisEnrichApp.scala:149)
    at scala.collection.immutable.List.foreach(List.scala:318)
    at com.snowplowanalytics.snowplow.enrich.kinesis.KinesisEnrichApp$delayedInit$body.apply(KinesisEnrichApp.scala:149)
    at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
    at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
    at scala.App$$anonfun$main$1.apply(App.scala:71)
    at scala.App$$anonfun$main$1.apply(App.scala:71)
    at scala.collection.immutable.List.foreach(List.scala:318)
    at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32)
    at scala.App$class.main(App.scala:71)
    at com.snowplowanalytics.snowplow.enrich.kinesis.KinesisEnrichApp$.main(KinesisEnrichApp.scala:75)
    at com.snowplowanalytics.snowplow.enrich.kinesis.KinesisEnrichApp.main(KinesisEnrichApp.scala)
@alexanderdean alexanderdean added this to the Kinesis #6 milestone Apr 29, 2015
@fblundun
Copy link
Contributor

The batch pipeline has special logic for the case where the URI starts "s3", but the Kinesis pipeline just tries to treat it as a normal URL.

@alexanderdean
Copy link
Member Author

Sure thing - the Kinesis pipeline will need some different logic, because unlike Hadoop, it doesn't get s3:// protocol support "for free"...

@alexanderdean
Copy link
Member Author

Why is this not an issue with pipelines we have running existing Kinesis Enrich, which are downloading geoip fine? /cc @jbeemster

jbeemster added a commit that referenced this issue Jun 30, 2015
… URI (closes #1645)

Downloading from a signed url does not work, have to list the object
directly.
jbeemster added a commit that referenced this issue Jul 6, 2015
… URI (closes #1645)

Downloading from a signed url does not work, have to list the object
directly.
jbeemster added a commit that referenced this issue Jul 6, 2015
… URI (closes #1645)

Downloading from a signed url does not work, have to list the object
directly.
alexanderdean pushed a commit that referenced this issue Jan 14, 2016
… URI (closes #1645)

Downloading from a signed url does not work, have to list the object
directly.
oguzhanunlu pushed a commit to snowplow/stream-enrich that referenced this issue May 28, 2020
…snowplow#1645)

Downloading from a signed url does not work, have to list the object
directly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants