Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time


The Spectra S3 Python SDK is no longer supported. Please consider using the Spectra S3 Python3 SDK.

Python 2.7 Sunsetting

Python 2.7 has officially been sunsetted (End of Life/Support) as of January 1st, 2020. Please see our blog article about current and future support for the Python 2 SDK.

Spectra S3 Python SDK

Apache V2 License

An SDK conforming to the Spectra S3 specification for Python 2.7

Contact Us

Join us at our Google Groups forum to ask questions, or see frequently asked questions.


To install the ds3_python_sdk, either clone the latest code, or download a release bundle from Releases. Once the code has been download, cd into the bundle, and install it with sudo python install

Once completes the ds3_python_sdk should be installed and available to be imported into python scripts.

Upgrading from 1.x to 3.x

Please read our migration guide for information on upgrading any code written with the 1.x sdk to the 3.x sdk.


The documentation for the SDK can be found at


The SDK provides an interface for a user to add Spectra S3 functionality to their existing or new python application. In order to take advantage of the SDK you need to import the ds3 python package and module. The following is an example that creates a Spectra S3 client from environment variables, creates a bucket, and lists all the buckets that are visible to the user.

from ds3 import ds3

client = ds3.createClientFromEnv()


getServiceResponse = client.get_service(ds3.GetServiceRequest())

for bucket in getServiceResponse.result['BucketList']:
    print bucket['Name']


In the ds3_python_sdk there are two ways that you can create a Ds3Client instance: environment variables, or manually. ds3.createClientFromEnv will create a Ds3Client using the following environment variables:

  • DS3_ENDPOINT - The URL to the DS3 Endpoint
  • DS3_ACCESS_KEY - The DS3 access key
  • DS3_SECRET_KEY - The DS3 secret key
  • http_proxy - If set, the Ds3Client instance will proxy through this URL

The Ds3Client instance can also be created manually in code with:

from ds3 import ds3

client = ds3.Ds3Client("endpoint", ds3.Credentials("access_key", "secret_key"))

The proxy URL can be passed in as the named parameter proxy to Ds3Client().

Putting Data

To put data to a Spectra S3 appliance you have to do it inside of the context of what is called a Bulk Job. Bulk Jobs allow the Spectra S3 appliance to plan how data should land to cache, and subsequently get written/read to/from tape. The basic flow of every job is:

  • Generate the list of objects that will either be sent to or retrieved from Spectra S3
  • Send a bulk put/get to Spectra S3 to plan the job
  • The job will be split into multiple chunks. An application must then get the available list of chunks that can be processed
  • For each chunk that can be processed, sent the object (this step can be done in parallel)
  • Repeat getting the list of available chunks until all chunks have been processed

An example of the above using the Python SDK for putting data

An example of getting data with the Python SDK

An example of using getService and getBucket to list all accessible buckets and objects

An example of how give objects on the server a different name than what is on the filesystem, and how to delete objects by folder