temporary fix for the ruler joining the ingester ring

[trevorwhitney]

What this PR does / why we need it:
Temporary fix until grafana/dskit#101 is merged.

Which issue(s) this PR fixes:
Fixes #4967

Checklist

• Documentation added
• Tests updated
• Add an entry in the CHANGELOG.md about the changes.

[slim-bean]

LGTM

[grafanabot]

The backport to release-2.4.1 failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-4968-to-release-2.4.1 origin/release-2.4.1
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x acbbe4ddfd3b352b93ad56a2f24254f566f4400b
# Push it to GitHub
git push --set-upstream origin backport-4968-to-release-2.4.1
git switch main
# Remove the local backport branch
git branch -D backport-4968-to-release-2.4.1

Then, create a pull request where the base branch is release-2.4.1 and the compare/head branch is backport-4968-to-release-2.4.1.