Updating release-pipeline.yml to support building Docker images from main branch

[mattnowzari]

Closes #283

This PR aims to update our release-pipeline.yml file to allow building Docker images off of main branch. This is useful for generating SNAPSHOT images.

Checklists

Pre-Review Checklist

• This PR does NOT contain credentials of any kind, such as API keys or username/passwords (double check crawler.yml.example and elasticsearch.yml.example)
• This PR has a meaningful title
• This PR links to all relevant GitHub issues that it fixes or partially addresses
  • If there is no GitHub issue, please create it. Each PR should have a link to an issue
• this PR has a thorough description
• Added a label for each target release version (example: v0.1.0)

[jedrazb]

Was the main branch left out intentionally? If we build from main will it be clear in the docker img that this is a snapshot build not an official release?

[mattnowzari]

Currently Crawler's pipelines have a UI element that allow us to specify specific builds as snapshots - I've moved this PR back to draft so I can experiment with automatically having builds from main be snapshot builds without specifying with a UI prompt - hopefully it is possible! 😄

crawler/.buildkite/publish/publish-common.sh

Line 18 in 712026c

IS_SNAPSHOT=$(buildkite-agent meta-data get is_snapshot)

crawler/.buildkite/release-pipeline.yml

Lines 5 to 13 in 712026c

	- input: "Build information"
	fields:
	- select: "Build as a SNAPSHOT?"
	key: "is_snapshot"
	options:
	- label: "Yes"
	value: "true"
	- label: "No"
	value: "false"

[mattnowzari]

FYI for reviewer - this changed bc of work done here --> #248

Our builds just haven't failed bc we haven't actually made any builds since that PR 🤣

[seanstory]

I like it. It removes our ability to do on-demand SNAPSHOT builds, but I think that's probably ok, especially since you're planning to come back later and add nightlies. Nice work.