W-17037056 feat: add data update bulk/resume commands

[cristiand391]

What does this PR do?

• Adds data update bulk/resume commands
• Adds --column-delimiter flag to data import bulk
• move data import bulk logic into a generic bulkIngest func that will be used by all bulkIngest-related commands (data import/update bulk in this PR, data delete/upsert bulk will be migrated in a separate PR)

What issues does this PR fix or reference?

@W-17037056@

[mdonnalley]

just a few thoughts/suggestions - otherwise looks good 👍

[mdonnalley]

should we handle the possibility that .instanceUrl doesn't exist or undefined instead of using as string?

[cristiand391]

I can't find the slack thread but IIRC we said stuff like instanceUrl should always be in auth files but changing the types ins sfdx-core now would be a big change, I'll wrap it in ensureString 👍🏼 .

[mdonnalley]

is as DataImportBulkResult just there to remove the | undefined? If so, why not use optional chaining instead of using an assertion?

[cristiand391]

I think I started doing the assertion in other NUTs and just carried the assertion, I can remove it.

[mdonnalley]

QA

🟢 data import bulk still works as expected

❯ sf data import bulk --file ~/repos/salesforcecli/plugin-data/test/test-files/data-project/data/bulkUpsertLarge.csv --sobject account --wait 10
 ›   Warning: @salesforce/plugin-data is a linked ESM module and cannot be auto-transpiled. Existing compiled source will be
 ›    used instead.

 ───────────────── Importing data ─────────────────

 ✔ Creating ingest job 11.01s
 ✔ Processing the job 1m 22.73s
   ▸ Processed records: 76380
   ▸ Successful records: 76380
   ▸ Failed records: 0

 Status: JobComplete
 Job Id: 750Ov00000IpmhlIAB
 Elapsed Time: 1m 34.38s

🟡 sf data import bulk without setting --column-delimiter

It fails as expected but I'm wondering if there's a way to auto-detect the delimiter??

❯ sf data import bulk --file ~/repos/salesforcecli/plugin-data/test/test-files/data-project/data/bulkUpsertPipes.csv --sobject account --wait 10
 ›   Warning: @salesforce/plugin-data is a linked ESM module and cannot be auto-transpiled. Existing compiled source will be
 ›    used instead.

 ───────────────── Importing data ─────────────────

 ✔ Creating ingest job 1.57s
 ✘ Processing the job 11.74s
   ▸ Processed records: ✘
   ▸ Successful records: ✘
   ▸ Failed records: ✘

 Status: Failed
 Job Id: 750Ov00000IpeIoIAJ
 Elapsed Time: 13.41s

Error (JobFailedError): Job failed to be processed due to:

InvalidBatch : Field name not found : NAME|TYPE|PHONE|WEBSITE|ANNUALREVENUE

To review the details of this job, run this command:

sf org open --target-org test-p4bh29a7jcvc@example.com --path "/lightning/setup/AsyncApiJobStatus/page?address=%2F750Ov00000IpeIoIAJ"

🟢 data import bulk --column-delimiter works as expected

❯ sf data import bulk --file ~/repos/salesforcecli/plugin-data/test/test-files/data-project/data/bulkUpsertPipes.csv --sobject account --wait 10 --column-delimiter PIPE
 ›   Warning: @salesforce/plugin-data is a linked ESM module and cannot be auto-transpiled. Existing compiled source will be
 ›    used instead.

 ───────────────── Importing data ─────────────────

 ✔ Creating ingest job 1.39s
 ✔ Processing the job 11.66s
   ▸ Processed records: 10
   ▸ Successful records: 10
   ▸ Failed records: 0

 Status: JobComplete
 Job Id: 750Ov00000IpTdaIAF
 Elapsed Time: 13.11s

🟡 sf data update bulk with csv with no ID colum

As I was trying to figure out how to use the new command, I encountered this:

❯ sf data update bulk --file ~/repos/salesforcecli/plugin-data/test/test-files/data-project/data/bulkUpsert.csv --sobject account --wait 10
 ›   Warning: @salesforce/plugin-data is a linked ESM module and cannot be auto-transpiled. Existing compiled source will be
 ›    used instead.

 ───────────────── Updating data ─────────────────

 ✔ Creating ingest job 1.48s
 ✘ Processing the job 5.73s
   ▸ Processed records: 10
   ▸ Successful records: 0
   ▸ Failed records: 10

 Status: JobComplete
 Job Id: 750Ov00000IpoIAIAZ
 Elapsed Time: 7.42s

Error (FailedRecordDetailsError): Job finished being processed but failed to process 10 records.

To review the details of this job, run this command:

sf org open --target-org test-p4bh29a7jcvc@example.com --path "/lightning/setup/AsyncApiJobStatus/page?address=%2F750Ov00000IpoIAIAZ"

The sf org open works as expected but I'm not able to find any useful information about why it failed on that page. Are we able to surface a more descriptive error message to the user?

Looks like I needed to have an ID column in my csv - maybe that's something we can detect on our end if it's not something that the API tells us

🟢 sf data update bulk

❯ sf data update bulk --file ~/repos/salesforcecli/plugin-data/test/test-files/data-project/data/bulkUpsert.csv --sobject account --wait 10
 ›   Warning: @salesforce/plugin-data is a linked ESM module and cannot be auto-transpiled. Existing compiled source will be
 ›    used instead.

 ───────────────── Updating data ─────────────────

 ✔ Creating ingest job 1.40s
 ✔ Processing the job 5.79s
   ▸ Processed records: 10
   ▸ Successful records: 10
   ▸ Failed records: 0

 Status: JobComplete
 Job Id: 750Ec00000FTwWFIA1
 Elapsed Time: 7.23s

🟢 handles aborted job

~/repos/trailheadapps/dreamhouse-lwc on  main via ⬢ v20.15.0 took 8.4s
❯ sf data update bulk --file ~/repos/salesforcecli/plugin-data/test/test-files/data-project/data/bulkUpsertPipes.csv --sobject account --wait 10
 ›   Warning: @salesforce/plugin-data is a linked ESM module and cannot be auto-transpiled. Existing compiled source will be
 ›    used instead.

 ───────────────── Updating data ─────────────────

 ✔ Creating ingest job 1.77s
 ✘ Processing the job 2m 2.77s
   ▸ Processed records: ✘
   ▸ Successful records: ✘
   ▸ Failed records: ✘

 Status: Aborted
 Job Id: 750Ov00000IppR7IAJ
 Elapsed Time: 2m 5.28s

Error (JobAbortedError): Job has been aborted.

To review the details of this job, run this command:

sf org open --target-org test-p4bh29a7jcvc@example.com --path "/lightning/setup/AsyncApiJobStatus/page?address=%2F750Ov00000IppR7IAJ"

sf data update bulk --async and sf data update resume

~/repos/trailheadapps/dreamhouse-lwc on  main via ⬢ v20.15.0 took 9.6s
❯ sf data update bulk --file ~/repos/salesforcecli/plugin-data/test/test-files/data-project/data/bulkUpsert.csv --sobject account --async
 ›   Warning: @salesforce/plugin-data is a linked ESM module and cannot be auto-transpiled. Existing compiled source will be
 ›    used instead.

 ───────────── Updating data (async) ─────────────

 ✔ Creating ingest job 1.57s
 ◼ Processing the job

 Status: UploadComplete
 Job Id: 750Ec00000FU0MvIAL
 Elapsed Time: 1.61s

Run "sf data update resume --job-id 750Ec00000FU0MvIAL" to resume the operation.

~/repos/trailheadapps/dreamhouse-lwc on  main via ⬢ v20.15.0 took 3.5s
❯ sf data update resume --job-id 750Ec00000FU0MvIAL
 ›   Warning: @salesforce/plugin-data is a linked ESM module and cannot be auto-transpiled. Existing compiled source will be
 ›    used instead.

 ───────────────── Updating data ─────────────────

 ◯ Creating ingest job - Skipped
 ✔ Processing the job 931ms
   ▸ Processed records: 10
   ▸ Successful records: 10
   ▸ Failed records: 0

 Status: JobComplete
 Job Id: 750Ec00000FU0MvIAL
 Elapsed Time: 966ms