Swift driver: Stocator should be able to access Swift public containers without authentication

[gilv]

In particular it should support referrer access URLs
http://docs.openstack.org/developer/swift/misc.html

[jasoncl]

Can you give us an example where a swift public container is?

[gilv]

Done.

[gilv]

@djalova I am getting complaints that it's not working anymore. To reproduce it - you need a public container with some objects. Then use Spark with
val d = sc.binaryFiles('swift2d://dal.objectstorage.open.softlayer.com/v1/AUTH_/somedata/*', minPartitions=20)

Getting

Py4JJavaError: An error occurred while calling o31.binaryFiles.
: java.lang.NullPointerException
at com.ibm.stocator.fs.swift.auth.JossAccount.getAccessURL(JossAccount.java:116)
at com.ibm.stocator.fs.swift.SwiftAPIClient.getAccessURI(SwiftAPIClient.java:588)
at com.ibm.stocator.fs.ObjectStoreFileSystem.getUri(ObjectStoreFileSystem.java:104)
at org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:469)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(FileInputFormat.java:500)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(FileInputFormat.java:469)
at org.apache.spark.SparkContext$$anonfun$binaryFiles$1.apply(SparkContext.scala:898)
at org.apache.spark.SparkContext$$anonfun$binaryFiles$1.apply(SparkContext.scala:893)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.SparkContext.withScope(SparkContext.scala:687)
at org.apache.spark.SparkContext.binaryFiles(SparkContext.scala:893)
at org.apache.spark.api.java.JavaSparkContext.binaryFiles(JavaSparkContext.scala:256)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:507)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:237)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:280)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:214)
at java.lang.Thread.run(Thread.java:785)

It worked fine in the past, seems something is broken in the recent code.

@djalova can you please check it? Thanks a lot

[gilv]

Resolved.