feat(datasource/azure-pipelines-tasks): Adds support for the Azure DevOps Marketplace

[jessehouwing]

Changes

Adds support for Azure DevOps Marketplace
Fetches marketplace data from https://github.com/jessehouwing/renovate-azure-devops-marketplace which refreshes daily

Context

Renovate currently doesn't support the non-built-in tasks. My own security research shows how many older tasks have significant issues and conversations with admins shows it's very hard for admins to ensure the latest versions of tasks are used.

Documentation (please check one with an [x])

• I have updated the documentation, or
• No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

• Code inspection only, or
• Newly added/modified unit tests, or
• No unit tests but ran on a real repository, or
• Both unit tests + ran on a real repository

[CLAassistant]

All committers have signed the CLA.

[JamieMagee]

Overall this looks good. We might just want to migrate your script into this repo directly, but I don't see this as a blocker for now.

[jessehouwing]

Overall this looks good. We might just want to migrate your script into this repo directly, but I don't see this as a blocker for now.

The script caches all extension an task manifests to speed things up. Which was way more data than I thought, I'd be willing to transfer this repo to the renovate org after cleaning it up a bit more though. I have a few ideas to speed up the update process (which currently takes about 10 to 15 minutes for each run).

Without the cache the update takes more than 24h as-is and transfers about 650GB of data.

The cache stores the minimum files to be able to cleanly regenerate the renovate-data file, but still adds almost 100k files to the repo and slows things down considerably.

My recommendation: bring the repo into the renovate org, but keep it separate from the main renovate repo. Or I need to refactor it a bit to stick the cache into a separate repo and update that from the workflow.

To run the updater, you also need to configure an Azure DevOps Marketplace PAT with Marketplace (Read) permissions stored as a GitHub Actions Secret.

[JamieMagee]

Wow 👀 I didn't realise it was such a large amount of data.

That sounds like a good suggestion. We can create a followup issue to do that after this PR is merged.

[jessehouwing]

I hadn't anticipated that either when I got started :o).

[renovate-release]

🎉 This PR is included in version 34.92.0 🎉

The release is available on:

• GitHub release
• 34.92.0

Your semantic-release bot 📦🚀