Skip to content

Latest commit

 

History

History
80 lines (63 loc) · 4.86 KB

object-storage.md

File metadata and controls

80 lines (63 loc) · 4.86 KB
Raw
title
Object Storage Integration

As indicated earlier, one of the prerequisite dependencies needed before installing Score is to setup a cloud object storage (see Configuring Storage Providers). Score uses the storage as a location for uploading data to and downloading data from.

Once your object storage is setup, Score can be configured to connect to it using an appropriate profile in the score-server-[version]/conf/application.properties file, depending on your storage provider:

Profile Purpose
default Required if using AWS, Ceph, or Minio as your object storage.
azure Required if using Microsoft Azure as your object storage.

Default Profile Example

To connect Score to AWS, Ceph, or Minio storage, in the score-server-[version]/conf/application.properties file, make sure the default profile exists and configure these settings:

Setting Requirement Description
s3 -> endpoint Required URL of the storage service's API endpoint. Score will send requests to this URL when interacting with the service's API.
s3 -> accessKey Required Access key required to access the buckets in your object storage. You should have recorded this as part of your prequisite setup.
s3 -> secretKey Required Secret key required to access the buckets in your object storage. You should have recorded this as part of your prequisite setup.
s3 -> sigV4Enabled Required If your storage service uses the AWS S3 Signature Version 4 signing process for authentication, set this to true. Else set this to false.
bucket -> name.object Required ID of the bucket used to store object data for Score. You should have recorded this as part of your prequisite setup.
bucket -> name.state Required ID of the bucket used to store and maintain state information for Score. You should have recorded this as part of your prequisite setup.
upload -> partsize Required Size, expressing in bytes, of each part or chunk to upload at once to the object storage. You can use this parameter to adjust for your desired speed and performance.
upload -> retry.limit Required Number of times that Score will retry failed upload requests to the object storage before aborting.
upload -> connectionltimeout Required Number of milliseconds that Score will wait on a stale or idle connection to the object storage before timing out.
upload -> clean.cron Optional If clean.enabled = true, use this parameter to configure the schedule on which the cleanup cron job runs. The cleanup job cleans up old upload jobs, between Score and the object storage.
upload -> clean.enabled Optional Set to true if you want to run a cron job that cleans up old upload jobs between Score and the object storage. If true, the cron schedule can be set with the clean.cron parameter. Else set this value false if you do not want any cleanup.

For example:

s3:
  endpoint: "http://localhost:9000"
  accessKey: abc123
  secretKey: abc123
  sigV4Enabled: true

bucket:
  name.object: test_object_bucket
  name.state: test_state_bucket
  size.pool: 0
  size.key: 2

upload:
  partsize: 1048576
  retry.limit: 10
  connection.timeout: 60000
  clean.cron: “0 0 0 * * ?”
  clean.enabled: true

Azure Profile Example

To connect Score to Microsoft Azure storage, in the score-server-[version]/conf/application.properties file, make sure the azure profile exists and configure these settings:

Setting Requirement Description
azure -> endpointProtocol Required Indicates the communication protocol used by the Azure storage service's API endpoint. For example, https.
azure -> accountName Required Account name required to access your Azure objectd storage. You should have recorded this as part of your prequisite setup.
azure -> accountKey Required Account key required to access your Azure objectd storage. You should have recorded this as part of your prequisite setup.
bucket -> name.object Required ID of the bucket used to store object data for Score. You should have recorded this as part of your prequisite setup.
upload -> partsize Required Size, expressing in bytes, of each part or chunk to upload at once to the object storage. You can use this parameter to adjust for your desired speed and performance.

For example:

azure:
  endpointProtocol: https
  accountName: abc123
  accountKey: abc123

bucket:
  name.object: test_object_bucket

upload:
  partsize: 1048576