fix: don't delete existing content on the File Share and fix the destination folder

[lemeurherve]

This PR:

• fixes the File Share synchronization as currently this job is erasing the content uploaded to our update center prototype by our update-center2 test job.
• fixes the folder sync to ensure the content is in the correct destination folder, extracting the SAS token query string from the existing credentials to its own credentials so we can set the destination folder.
• logs each crawler name before executing them.

Note: while aws-cli creates the folder if needed, azcopy fails with a 404 error: I had to manually create the updates folder in the File Share.

Ref:

• [INFRA-3100] Migrate updates.jenkins.io to another Cloud helpdesk#2649

[daniel-beck]

Note that this should not delete existing output from the target directory even. https://updates.jenkins.io/updates/ notice anything about the file modification times? Some crawlers don't work, but the existing files will ensure continued operation of the plugins relying of them. They'll just slowly become outdated.

[dduportal]

Note that this should not delete existing output from the target directory even. https://updates.jenkins.io/updates/ notice anything about the file modification times? Some crawlers don't work, but the existing files will ensure continued operation of the plugins relying of them. They'll just slowly become outdated.

absolutely! the deletion only happened in the "new and not used" targets added in #128. The rule of thumb we have is "DO NOT CHANGE THE CURRENT updates.jenkins.io SYSTEM AND PROCESS" => so we don't plan to ever touch the rsync deployment step at all.

[dduportal]

Moving back to draft until @lemeurherve finished his tests

[lemeurherve]

Some crawlers don't work, but the existing files will ensure continued operation of the plugins relying of them. They'll just slowly become outdated.

Thanks for the info!

I imagine we'll have to ensure we copy them over the File Share and the R2 buckets too.

Should we open issues for each of them? (FlywayInstaller & PerlInstaller)

[lemeurherve]

Moving back to draft until @lemeurherve finished his tests

Test successful, ready for review.