Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"unexpected keyword argument" after loadImagesAsSeries on s3 #40

Closed
industrial-sloth opened this issue Oct 17, 2014 · 1 comment
Closed
Assignees

Comments

@industrial-sloth
Copy link
Contributor

Running on an ec2 cluster launched via the thunder-ec2 script...

series = tsc.loadImagesAsSeries("s3n://thunder.datasets/test/tif-stack-2/", inputformat='tif-stack')
series.first()

some output happens, then:

 org.apache.spark.api.python.PythonException: Traceback (most recent call last):
  File "/root/spark/python/pyspark/worker.py", line 79, in main
    serializer.dump_stream(func(split_index, iterator), outfile)
  File "/root/spark/python/pyspark/serializers.py", line 196, in dump_stream
    self.serializer.dump_stream(self._batched(iterator), stream)
  File "/root/spark/python/pyspark/serializers.py", line 127, in dump_stream
    for obj in iterator:
  File "/root/spark/python/pyspark/serializers.py", line 185, in _batched
    for item in iterator:
  File "/root/spark/python/pyspark/rdd.py", line 1148, in takeUpToNumLeft
    yield next(iterator)
  File "thunder/rdds/fileio/seriesloader.py", line 362, in readblockfromtif
  File "build/bdist.linux-x86_64/egg/thunder/rdds/fileio/readers.py", line 448, in close
    self._key.close(fast=True)
TypeError: close() got an unexpected keyword argument 'fast'
@industrial-sloth industrial-sloth self-assigned this Oct 17, 2014
@industrial-sloth
Copy link
Contributor Author

This is a boto versioning problem - the ec2 cluster is provisioned (apparently) with boto 2.8.0, where it appears the "fast" keyword in close() doesn't exist yet. I'll get rid of that keyword and anything else that crept in requiring newer versions of boto.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants