adapter: reduce Lua list lock contention by bootjp · Pull Request #384 · bootjp/elastickv

bootjp · 2026-03-21T09:37:10Z

No description provided.

gemini-code-assist · 2026-03-21T09:37:27Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the performance and robustness of Redis list operations executed via Lua scripts. By adopting a lazy loading and delta-based commit approach, it drastically reduces the data processed and locked for list modifications, thereby mitigating contention. Furthermore, the transaction retry logic has been made more intelligent, distinguishing between different types of retryable errors and providing clearer error reporting by normalizing internal keys.

Highlights

Optimized Redis List Handling in Lua: Implemented a lazy loading and delta-based commit strategy for Redis lists within Lua scripts. This reduces the need to load and write entire lists, minimizing lock contention and improving performance for list operations.
Refined Transaction Retry Mechanism: Introduced distinct retry policies for write conflicts and transaction locks, allowing for more granular backoff strategies. Error messages for retryable transaction errors are now normalized to expose user-facing keys instead of internal key representations.
Enhanced List State Management: The luaListState struct was extended with fields to track partial list modifications (leftTrim, rightTrim, leftValues, rightValues) and a materialized flag, enabling efficient in-memory manipulation without full data retrieval.
New Test Coverage: Added new test cases to validate the behavior of RPOPLPUSH with the optimized list implementation, ensuring correct functionality and TTL preservation.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a significant optimization to reduce lock contention for Lua list operations by implementing a lazy-loading and delta-committing mechanism. Instead of materializing entire lists, operations like LPUSH/RPUSH and LPOP/RPOP now work on deltas, minimizing the transaction's read/write set. The transaction retry logic is also enhanced with different backoff policies for write conflicts versus transaction locks, and error messages are normalized to group retries for the same logical key. The changes are complex but well-implemented and include relevant tests. I've added a couple of suggestions for improving code clarity and robustness.

Copilot

Pull request overview

This PR aims to reduce contention in the Redis adapter’s Lua transaction path, primarily by avoiding full list materialization and by committing list changes as deltas where possible. It also refines retry behavior for retryable transaction errors by using different backoff policies and normalizing internal keys in error messages for clearer diagnostics.

Changes:

Add kv.ExtractTxnUserKey to recover logical user keys from !txn|...| internal keys.
Update Redis txn retry handling with per-error retry policies and error-key normalization (list keys, TTL keys, txn-internal keys).
Rework Lua list handling to lazily load list metadata, materialize only when required, and commit list edits via delta operations; add compatibility tests for RPOPLPUSH behavior and TTL preservation.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
`kv/txn_keys.go`	Adds helper to extract logical user key from txn-internal key namespaces.
`adapter/redis_retry.go`	Introduces retry policies and normalizes retryable txn error keys to reduce noisy internal key strings.
`adapter/redis_retry_test.go`	Adds unit tests for retry policy selection and error normalization on internal keys.
`adapter/redis_lua_context.go`	Implements lazy list state, list materialization on-demand, and delta-based list commits to reduce write amplification/lock contention.
`adapter/redis_lua_compat_test.go`	Adds integration coverage for `RPOPLPUSH` list semantics and TTL preservation in Lua flows.

Copilot · 2026-03-21T09:45:00Z

 }

-func (c *luaScriptContext) listCommitElems(key string) ([]*kv.Elem[kv.OP], error) {
+func (c *luaScriptContext) stringCommitElems(key string) ([]*kv.Elem[kv.OP], error) {
+	st, err := c.stringState([]byte(key))
+	if err != nil {


listCommitPlan returns preserveExisting: true for the non-materialized/delta path, which skips deleteLogicalKeyElems. If a Lua script calls deleteLogical (e.g., DEL key) and then recreates the same key as a list in the same transaction, listState will not load the persisted list meta (because c.deleted[key]=true makes keyType return redisTypeNone). The delta commit then writes only the new meta/items and leaves all previous list items (and potentially other-type internal keys) orphaned in storage. Suggest tracking an "ever deleted/reset" flag per key and forcing a full rewrite (preserveExisting=false) when a key was deleted/reset earlier in the script, even if later recreated as a list.

bootjp · 2026-03-21T09:45:49Z

@copilot open a new pull request to apply changes based on the comments in this thread

Copilot · 2026-03-21T09:45:57Z

@bootjp I've opened a new pull request, #385, to work on those changes. Once the pull request is ready, I'll request review from you.

Co-authored-by: bootjp <1306365+bootjp@users.noreply.github.com> Agent-Logs-Url: https://github.com/bootjp/elastickv/sessions/a35c08cd-cb6d-4ea5-9605-cd91074de396

…mbers) Co-authored-by: bootjp <1306365+bootjp@users.noreply.github.com> Agent-Logs-Url: https://github.com/bootjp/elastickv/sessions/c131ce44-56b0-443c-bf86-5921e0684a69

adapter: fix orphaned list items when DEL + recreate in same Lua script

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

adapter: reduce Lua list lock contention

a9cff05

bootjp requested a review from Copilot March 21, 2026 09:38

Copilot started reviewing on behalf of bootjp March 21, 2026 09:39 View session

gemini-code-assist Bot reviewed Mar 21, 2026

View reviewed changes

Comment thread adapter/redis_lua_context.go Outdated

Comment thread adapter/redis_retry.go Outdated

Merge branch 'main' into feature/reduce-lock

d0f85b2

Copilot AI reviewed Mar 21, 2026

View reviewed changes

Initial plan

dfcff5a

Copilot AI mentioned this pull request Mar 21, 2026

adapter: fix orphaned list items when DEL + recreate in same Lua script #385

Merged

Copilot AI and others added 3 commits March 21, 2026 09:54

adapter: fix DEL+recreate list leaving orphaned storage items

961206c

Co-authored-by: bootjp <1306365+bootjp@users.noreply.github.com> Agent-Logs-Url: https://github.com/bootjp/elastickv/sessions/a35c08cd-cb6d-4ea5-9605-cd91074de396

adapter,proxy: fix golangci-lint issues (gci formatting, mnd magic nu…

b4ab53e

…mbers) Co-authored-by: bootjp <1306365+bootjp@users.noreply.github.com> Agent-Logs-Url: https://github.com/bootjp/elastickv/sessions/c131ce44-56b0-443c-bf86-5921e0684a69

Merge pull request #385 from bootjp/copilot/sub-pr-384

939e785

adapter: fix orphaned list items when DEL + recreate in same Lua script

bootjp requested a review from Copilot March 21, 2026 10:09

Copilot started reviewing on behalf of bootjp March 21, 2026 10:10 View session

Copilot AI reviewed Mar 21, 2026

View reviewed changes

Comment thread adapter/redis_lua_context.go Outdated

bootjp and others added 2 commits March 21, 2026 19:13

adapter: use structured redis txn errors

f1d04ba

Update adapter/redis_lua_context.go

defb69a

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

bootjp requested a review from Copilot March 21, 2026 10:22

Copilot started reviewing on behalf of bootjp March 21, 2026 10:23 View session

Copilot AI reviewed Mar 21, 2026

View reviewed changes

Comment thread store/store.go Outdated

Comment thread kv/txn_errors.go Outdated

bootjp and others added 2 commits March 21, 2026 19:27

Update store/store.go

e5ebad4

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update kv/txn_errors.go

18547c5

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

bootjp enabled auto-merge March 21, 2026 10:28

bootjp merged commit bac2368 into main Mar 21, 2026
8 checks passed

bootjp deleted the feature/reduce-lock branch March 21, 2026 10:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

adapter: reduce Lua list lock contention#384

adapter: reduce Lua list lock contention#384
bootjp merged 10 commits into
mainfrom
feature/reduce-lock

bootjp commented Mar 21, 2026

Uh oh!

gemini-code-assist Bot commented Mar 21, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 21, 2026

Uh oh!

bootjp commented Mar 21, 2026

Uh oh!

Copilot AI commented Mar 21, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

bootjp commented Mar 21, 2026

Uh oh!

gemini-code-assist Bot commented Mar 21, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

bootjp commented Mar 21, 2026

Uh oh!

Copilot AI commented Mar 21, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants