Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix handling large states in the Consul backend #28838

Merged
merged 2 commits into from Jul 30, 2021

Conversation

remilapeyre
Copy link
Contributor

The logic in e680211 to determine whether a given state is small enough to fit in a single KV entry in Consul is buggy: because we are using the Transaction API we are base64 encoding it so the payload sent is actually bigger than the raw state, and there is some JSON overhead. This fixes this issue by always trying to store the state in a single entry, and we split it if Consul tells us that is too large. Because we don't try to guess we don't fail anymore.

As far as I know no user as encountered this bug but this patch should be backported to the 0.14 branch as it was the first version to have this feature in the Consul backend.

…ckend

Storing a state just a bit smaller than the limit of the KV API fails
because we are using the Transaction API which has an overhead.
@42wim
Copy link

42wim commented Jul 2, 2021

Hashicorp peeps, can someone take a look at this?
I'm trying to use consul as a backend because support for chunks was added, but at the moment this isn't actually usable.

@jbardin jbardin self-assigned this Jul 27, 2021
@jbardin jbardin added the 1.0-backport If you add this label to a PR before merging, backport-assistant will open a new PR once merged label Jul 30, 2021
@jbardin jbardin merged commit 97a2694 into hashicorp:main Jul 30, 2021
@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 30, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
1.0-backport If you add this label to a PR before merging, backport-assistant will open a new PR once merged backend/consul
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants