feat(dataset): importer

[reynld]

Allows users to import datasets

Accepted params

--clusterId Cluster to run import on
--machineType Machine to run import job on
--datasetId Dataset ID
--s3Url URL for s3 URLs
--httpUrl URL for s3 URLs
--s3AccessKey s3 Access Key
--s3SecretKey s3 Secret Key
--httpAuth HTTP Basic auth username:password

Examples

HTTPS

python3 ./gradient datasets import \
--clusterId clcjo828k \
--machineType c5.xlarge \
--datasetId dsr4bvtdcsefhgq \
--httpUrl google.com 

S3

python3 ./gradient datasets import \
--clusterId clcjo828k \
--machineType c5.xlarge \
--datasetId dsr4bvtdcsefhgq \
--s3Url s3://talmnist/tryagain2.csv \
--s3AccessKey ACCESS_KEY \
--s3SecretKey SECRET

QA TEST PLAN

• Set PAPERSPACE_CONFIG_HOST environment variable to point to staging
  export PAPERSPACE_CONFIG_HOST=https://staging-api.paperspace.io
• Create dataset
  Staging Google Saml Prod storage provider id: sp694axp9oevzyj
python3 ./gradient datasets create --name <NAME> --storageProviderId <STORAGE PROVIDER ID>
• Run import job using the examples above

[PSBOT]

🎉 This PR is included in version 1.3.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀