Conversation
| | `dataImport.config.s3AccessKeyId` | AWS S3 access key for reading the CSV data | `""` | | ||
| | `dataImport.config.s3AccessKeySecret` | AWS S3 secret key for reading the CSV data | `""` | | ||
| | `dataImport.config.s3Region` | AWS S3 region for accessing the bucket | `us-east-1` | | ||
| | `dataImport.config.csvSourceFile` | The source file for data import stored in S3 | `s3://new-bucket/data.csv` | |
There was a problem hiding this comment.
We should change the description to be generic here, not tied to S3
| | `dataImport.image.tag` | The Docker image tag for the custom data importer | `2.0.1` | | ||
| | `dataImport.image.pullPolicy` | The image pull policy | `Always` | | ||
| | `dataImport.config.expressUrl` | The URL for the express engine used in the import job | `https://express-engine-cluster-master:9200` | | ||
| | `dataImport.config.s3AccessKeyId` | AWS S3 access key for reading the CSV data | `""` | |
There was a problem hiding this comment.
can we move them under config.aws, later on when we add azure, and gcp we can add those sections here.
|
Whole version of helm chart also needs to be updated to 2.0.2. And the main readme needs to be updated with the release information. |
| apiVersion: batch/v1 | ||
| kind: Job | ||
| metadata: | ||
| name: "{{ .Release.Name }}-importer" |
There was a problem hiding this comment.
AWS jobs are immutable. we should add revision number to the JOB
There was a problem hiding this comment.
Please add revision number to the JOB
| imagePullSecrets: | ||
| {{- toYaml . | nindent 8 }} | ||
| {{- end }} | ||
| restartPolicy: Never |
There was a problem hiding this comment.
Please specify resources for the job
| --set dataImport.config.aws.s3AccessKeyId="NEW_AKIA_ACCESS_KEY" \ | ||
| --set dataImport.config.aws.s3AccessKeySecret="NEW_SECRET_ACCESS_KEY" \ | ||
| --set dataImport.config.aws.s3Region="us-west-2" \ | ||
| --set dataImport.config.csvSourceFile="s3://new-bucket/new-data.csv" |
There was a problem hiding this comment.
Please also add the nodeSelector here, because express service runs on arm64 so here we should mention the type of node needed for this importer image
2 participants
No description provided.