Skip to content

CAMEL-24463: Improve KeyValueRepository infrastructure - #26086

Merged
gnodet merged 4 commits into
apache:mainfrom
gnodet:fix/kvr-improvements
Sep 3, 2026
Merged

CAMEL-24463: Improve KeyValueRepository infrastructure#26086
gnodet merged 4 commits into
apache:mainfrom
gnodet:fix/kvr-improvements

Conversation

@gnodet

@gnodet gnodet commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Changes

Comprehensive improvements to the KeyValueRepository infrastructure based on a full review of the SPI, all 11 backends, adapters, Cache EIP, and documentation.

Documentation

  • New keyValueRepository.adoc (1077 lines) — comprehensive backends documentation covering all 11 implementations with configuration examples (Java + YAML DSL), Maven dependencies, atomicity guarantees table, TTL behavior differences, and adapter usage for Idempotent Consumer / Aggregator patterns
  • Kafka Javadoc — TTL/topic growth guidance (log compaction recommendations) and non-atomic CAS limitation on putIfAbsent
  • Cassandra Javadoc — non-atomic put() (no native getAndSet equivalent)
  • Ehcache/JCache Javadoc — non-atomic put() (no native getAndPut with per-entry TTL wrapper)

Test Coverage

  • KeyValueRepositoryHelperTest — 22 new tests covering serialize/deserialize roundtrips (String, Integer, complex objects), ByteBuffer variant, offset/length variant, error cases (non-Serializable → RuntimeCamelException), and null handling. This helper had zero test coverage despite being shared across all persistent backends.
  • CacheProcessorTest — new test for cacheNull=true path (the trickier code path using contains() + get() instead of just get())

Atomicity Fixes

  • RedisKeyValueRepository.put() — replaced non-atomic get() + set() with Redisson's atomic getAndSet() (both TTL and non-TTL variants). Previous implementation could return stale previous values under concurrent writes.
  • JdbcKeyValueRepository — added atomic replace() and delete(key, expected) overrides using SQL WHERE ITEM_KEY = ? AND ITEM_VALUE = ? for server-side CAS. Includes customizable SQL strings and table-name substitution. Javadoc documents the serialized-bytes comparison semantics.

Code Deduplication

  • Extracted KeyValueTtlValue — the identical TtlValue inner class was duplicated in both EhcacheKeyValueRepository and JCacheKeyValueRepository. Extracted to a shared class in camel-support. Updated source and test files in both modules.

Build Verification

  • All existing tests pass: MemoryKeyValueRepositoryTest (27), KeyValueIdempotentRepositoryTest (14), KeyValueAggregationRepositoryTest (12), CacheProcessorTest (8 including new test)
  • New KeyValueRepositoryHelperTest — 22/22 ✅
  • All affected component modules compile: camel-ehcache, camel-jcache, camel-redis, camel-sql, camel-cassandraql, camel-kafka

- Add KeyValueRepositoryHelper unit tests (22 tests covering serialize/deserialize roundtrips, ByteBuffer, offset/length, error cases, null handling)
- Add CacheProcessor test for cacheNull=true path
- Fix RedisKeyValueRepository.put() to use atomic getAndSet() instead of non-atomic get+set
- Extract shared KeyValueTtlValue from duplicate TtlValue inner classes in Ehcache and JCache backends
- Fix import ordering in Ehcache/JCache source and test files
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

🧪 CI tested the following changed modules:

  • catalog/camel-catalog
  • components/camel-cassandraql
  • components/camel-ehcache
  • components/camel-jcache
  • components/camel-kafka
  • components/camel-redis
  • components/camel-sql
  • core/camel-core-engine
  • core/camel-core
  • core/camel-support

ℹ️ Dependent modules were not tested because the total number of affected modules exceeded the threshold (50). Use the test-dependents label to force testing all dependents.


🔬 Scalpel shadow comparison — Scalpel: 563 tested, 26 compile-only — current: 562 all tested

Maveniverse Scalpel detected 589 affected modules (current approach: 562).

⚠️ Modules only in Scalpel (27)
  • apache-camel
  • camel-allcomponents
  • camel-catalog
  • camel-catalog-console
  • camel-catalog-maven
  • camel-catalog-suggest
  • camel-componentdsl
  • camel-endpointdsl
  • camel-endpointdsl-support
  • camel-itest
  • camel-jbang-core
  • camel-jbang-it
  • camel-jbang-main
  • camel-jbang-plugin-edit
  • camel-jbang-plugin-generate
  • camel-jbang-plugin-kubernetes
  • camel-jbang-plugin-test
  • camel-kamelet-main
  • camel-launcher
  • camel-report-maven-plugin
  • camel-route-parser
  • camel-yaml-dsl
  • camel-yaml-dsl-deserializers
  • camel-yaml-dsl-maven-plugin
  • coverage
  • docs
  • dummy-component

Skip-tests mode would test 563 modules (10 direct + 553 downstream), skip tests for 26 (generated code, meta-modules)

Modules Scalpel would test (563)
  • archetypes
  • camel-a2a
  • camel-activemq
  • camel-activemq6
  • camel-ai-observability
  • camel-ai-observability-api
  • camel-ai-parent
  • camel-ai-resource
  • camel-ai-tool
  • camel-alibaba-common
  • camel-alibaba-eventbridge
  • camel-alibaba-fc
  • camel-alibaba-kms
  • camel-alibaba-mns
  • camel-alibaba-oss
  • camel-alibaba-ots
  • camel-alibaba-parent
  • camel-alibaba-sls
  • camel-alibaba-sms
  • camel-amqp
  • camel-api-component-maven-plugin
  • camel-arangodb
  • camel-archetype-api-component
  • camel-archetype-component
  • camel-archetype-dataformat
  • camel-archetype-java
  • camel-archetype-main
  • camel-as2
  • camel-as2-api
  • camel-as2-parent
  • camel-asn1
  • camel-asterisk
  • camel-atmosphere-websocket
  • camel-atom
  • camel-attachments
  • camel-avro
  • camel-avro-rpc
  • camel-avro-rpc-jetty
  • camel-avro-rpc-parent
  • camel-avro-rpc-spi
  • camel-aws-bedrock
  • camel-aws-cloudtrail
  • camel-aws-common
  • camel-aws-config
  • camel-aws-parameter-store
  • camel-aws-parent
  • camel-aws-secrets-manager
  • camel-aws-security-hub
  • camel-aws2-athena
  • camel-aws2-comprehend
  • camel-aws2-cw
  • camel-aws2-ddb
  • camel-aws2-ec2
  • camel-aws2-ecs
  • camel-aws2-eks
  • camel-aws2-eventbridge
  • camel-aws2-iam
  • camel-aws2-kinesis
  • camel-aws2-kms
  • camel-aws2-lambda
  • camel-aws2-mq
  • camel-aws2-msk
  • camel-aws2-polly
  • camel-aws2-redshift
  • camel-aws2-rekognition
  • camel-aws2-s3
  • camel-aws2-s3-vectors
  • camel-aws2-ses
  • camel-aws2-sns
  • camel-aws2-sqs
  • camel-aws2-step-functions
  • camel-aws2-sts
  • camel-aws2-textract
  • camel-aws2-timestream
  • camel-aws2-transcribe
  • camel-aws2-translate
  • camel-azure-common
  • camel-azure-cosmosdb
  • camel-azure-eventgrid
  • camel-azure-eventhubs
  • camel-azure-files
  • camel-azure-functions
  • camel-azure-key-vault
  • camel-azure-parent
  • camel-azure-schema-registry
  • camel-azure-servicebus
  • camel-azure-storage-blob
  • camel-azure-storage-datalake
  • camel-azure-storage-queue
  • camel-barcode
  • camel-base
  • camel-base-engine
  • camel-base64
  • camel-bean
  • camel-bean-validator
  • camel-beanio
  • camel-bindy
  • camel-bonita
  • camel-box
  • camel-box-api
  • camel-box-parent
  • camel-braintree
  • camel-browse
  • camel-caffeine
  • camel-camunda
  • camel-cassandraql
  • camel-catalog
  • camel-catalog-common
  • camel-cbor
  • camel-chatscript
  • camel-chunk
  • camel-cli-connector
  • camel-cli-debug
  • camel-clickhouse
  • camel-clickup
  • camel-cloudevents
  • camel-cluster
  • camel-cm-sms
  • camel-coap
  • camel-cometd
  • camel-console
  • camel-consul
  • camel-controlbus
  • camel-core
  • camel-core-all
  • camel-core-engine
  • camel-core-languages
  • camel-core-model
  • camel-core-processor
  • camel-core-reifier
  • camel-core-xml
  • camel-couchbase
  • camel-couchdb
  • camel-cron
  • camel-crypto
  • camel-crypto-pgp
  • camel-csv
  • camel-cxf-common
  • camel-cxf-parent
  • camel-cxf-rest
  • camel-cxf-soap
  • camel-cxf-spring-common
  • camel-cxf-spring-rest
  • camel-cxf-spring-soap
  • camel-cxf-spring-transport
  • camel-cxf-transport
  • camel-cyberark-vault
  • camel-dapr
  • camel-dataformat
  • camel-dataset
  • camel-datasonnet
  • camel-dataweave
  • camel-debezium-common
  • camel-debezium-common-parent
  • camel-debezium-db2
  • camel-debezium-maven-plugin
  • camel-debezium-mongodb
  • camel-debezium-mysql
  • camel-debezium-oracle
  • camel-debezium-parent
  • camel-debezium-postgres
  • camel-debezium-sqlserver
  • camel-debug
  • camel-dfdl
  • camel-dhis2
  • camel-dhis2-api
  • camel-dhis2-parent
  • camel-diagram
  • camel-direct
  • camel-disruptor
  • camel-djl
  • camel-dns
  • camel-docker
  • camel-docling
  • camel-drill
  • camel-dropbox
  • camel-dsl-modeline
  • camel-dsl-support
  • camel-duckdb
  • camel-dynamic-router
  • camel-ehcache
  • camel-elasticsearch
  • camel-elasticsearch-rest-client
  • camel-event
  • camel-exec
  • camel-fastjson
  • camel-fhir
  • camel-fhir-api
  • camel-fhir-parent
  • camel-file
  • camel-file-watch
  • camel-flatpack
  • camel-flink
  • camel-flowable
  • camel-fop
  • camel-fory
  • camel-freemarker
  • camel-ftp
  • camel-ftp-common
  • camel-geocoder
  • camel-git
  • camel-github2
  • camel-google-bigquery
  • camel-google-calendar
  • camel-google-common
  • camel-google-drive
  • camel-google-firestore
  • camel-google-functions
  • camel-google-mail
  • camel-google-parent
  • camel-google-pubsub
  • camel-google-secret-manager
  • camel-google-sheets
  • camel-google-speech-to-text
  • camel-google-storage
  • camel-google-text-to-speech
  • camel-google-vertexai
  • camel-google-vision
  • camel-graphql
  • camel-grok
  • camel-groovy
  • camel-grpc
  • camel-gson
  • camel-hashicorp-vault
  • camel-hazelcast
  • camel-health
  • camel-hivemq
  • camel-hl7
  • camel-http
  • camel-http-base
  • camel-http-common
  • camel-huawei-parent
  • camel-huaweicloud-common
  • camel-huaweicloud-dms
  • camel-huaweicloud-frs
  • camel-huaweicloud-functiongraph
  • camel-huaweicloud-iam
  • camel-huaweicloud-imagerecognition
  • camel-huaweicloud-obs
  • camel-huaweicloud-smn
  • camel-huggingface
  • camel-ibm-cos
  • camel-ibm-parent
  • camel-ibm-secrets-manager
  • camel-ibm-watson-discovery
  • camel-ibm-watson-language
  • camel-ibm-watson-speech-to-text
  • camel-ibm-watson-text-to-speech
  • camel-ibm-watsonx-ai
  • camel-ibm-watsonx-data
  • camel-ical
  • camel-iggy
  • camel-ignite
  • camel-infinispan
  • camel-infinispan-common
  • camel-infinispan-embedded
  • camel-infinispan-parent
  • camel-influxdb
  • camel-influxdb2
  • camel-iso8583
  • camel-jackson
  • camel-jackson-avro
  • camel-jackson-protobuf
  • camel-jackson3
  • camel-jackson3-avro
  • camel-jackson3-protobuf
  • camel-jackson3xml
  • camel-jacksonxml
  • camel-jactl
  • camel-jandex
  • camel-jasypt
  • camel-java-io
  • camel-java-joor-dsl
  • camel-javascript
  • camel-jaxb
  • camel-jbang-console
  • camel-jbang-mcp
  • camel-jbang-plugin-mcp
  • camel-jbang-plugin-route-parser
  • camel-jbang-plugin-tui
  • camel-jbang-plugin-validate
  • camel-jcache
  • camel-jcr
  • camel-jdbc
  • camel-jetty
  • camel-jetty-common
  • camel-jfr
  • camel-jgroups
  • camel-jgroups-raft
  • camel-jira
  • camel-jms
  • camel-jmx
  • camel-jolt
  • camel-jooq
  • camel-joor
  • camel-jpa
  • camel-jq
  • camel-jsch
  • camel-jslt
  • camel-json-validator
  • camel-jsonapi
  • camel-jsonata
  • camel-jsonb
  • camel-jsonpath
  • camel-jsoup
  • camel-jt400
  • camel-jta
  • camel-jte
  • camel-kafka
  • camel-kamelet
  • camel-kamelet-main-support
  • camel-keycloak
  • camel-knative
  • camel-knative-api
  • camel-knative-http
  • camel-knative-parent
  • camel-kserve
  • camel-kubernetes
  • camel-kudu
  • camel-langchain4j-agent
  • camel-langchain4j-agent-api
  • camel-langchain4j-chat
  • camel-langchain4j-core
  • camel-langchain4j-embeddings
  • camel-langchain4j-embeddingstore
  • camel-langchain4j-embeddingstore-api
  • camel-langchain4j-tokenizer
  • camel-langchain4j-web-search
  • camel-language
  • camel-launcher-container
  • camel-ldap
  • camel-ldif
  • camel-log
  • camel-lra
  • camel-lucene
  • camel-lumberjack
  • camel-lzf
  • camel-mail
  • camel-mail-microsoft-oauth
  • camel-main
  • camel-management
  • camel-mapstruct
  • camel-master
  • camel-maven-plugin
  • camel-mcp-server
  • camel-mcp-server-api
  • camel-mdc
  • camel-metrics
  • camel-micrometer
  • camel-micrometer-observability
  • camel-micrometer-prometheus
  • camel-microprofile-config
  • camel-microprofile-fault-tolerance
  • camel-microprofile-health
  • camel-microprofile-parent
  • camel-milo
  • camel-milvus
  • camel-mina
  • camel-mina-sftp
  • camel-minio
  • camel-mllp
  • camel-mock
  • camel-mongodb
  • camel-mongodb-gridfs
  • camel-mustache
  • camel-mvel
  • camel-mybatis
  • camel-nats
  • camel-neo4j
  • camel-netty
  • camel-netty-http
  • camel-oaipmh
  • camel-oauth
  • camel-observability-services
  • camel-observation
  • camel-ocsf
  • camel-ognl
  • camel-olingo2
  • camel-olingo2-api
  • camel-olingo2-parent
  • camel-olingo4
  • camel-olingo4-api
  • camel-olingo4-parent
  • camel-once
  • camel-openai
  • camel-openapi-java
  • camel-openapi-rest-dsl-generator
  • camel-openapi-validator
  • camel-opensearch
  • camel-openstack
  • camel-opentelemetry
  • camel-opentelemetry-metrics
  • camel-opentelemetry2
  • camel-optaplanner
  • camel-paho
  • camel-paho-mqtt5
  • camel-parquet-avro
  • camel-pdf
  • camel-pg-replication-slot
  • camel-pgevent
  • camel-pgvector
  • camel-pinecone
  • camel-platform-http
  • camel-platform-http-jolokia
  • camel-platform-http-main
  • camel-platform-http-vertx
  • camel-plc4x
  • camel-pqc
  • camel-printer
  • camel-protobuf
  • camel-pubnub
  • camel-pulsar
  • camel-python
  • camel-python3
  • camel-qdrant
  • camel-quartz
  • camel-quickfix
  • camel-quickjs
  • camel-reactive-streams
  • camel-reactor
  • camel-redis
  • camel-ref
  • camel-resilience4j
  • camel-resilience4j-micrometer
  • camel-resourceresolver-github
  • camel-rest
  • camel-rest-openapi
  • camel-rest-postman
  • camel-restdsl-openapi-plugin
  • camel-robotframework
  • camel-rocketmq
  • camel-rss
  • camel-rxjava
  • camel-saga
  • camel-salesforce
  • camel-salesforce-codegen
  • camel-salesforce-maven-plugin
  • camel-salesforce-parent
  • camel-sap-netweaver
  • camel-saxon
  • camel-scheduler
  • camel-schematron
  • camel-seda
  • camel-servicenow
  • camel-servicenow-maven-plugin
  • camel-servicenow-parent
  • camel-servlet
  • camel-shell
  • camel-shiro
  • camel-sjms
  • camel-sjms2
  • camel-slack
  • camel-smb
  • camel-smooks
  • camel-smpp
  • camel-snakeyaml
  • camel-snmp
  • camel-soap
  • camel-solr
  • camel-spiffe
  • camel-splunk-hec
  • camel-spring
  • camel-spring-ai-chat
  • camel-spring-ai-embeddings
  • camel-spring-ai-image
  • camel-spring-ai-parent
  • camel-spring-ai-vector-store
  • camel-spring-batch
  • camel-spring-cloud-config
  • camel-spring-jdbc
  • camel-spring-ldap
  • camel-spring-main
  • camel-spring-parent
  • camel-spring-rabbitmq
  • camel-spring-redis
  • camel-spring-security
  • camel-spring-ws
  • camel-spring-xml
  • camel-sql
  • camel-ssh
  • camel-state-store
  • camel-state-store-parent
  • camel-stax
  • camel-stitch
  • camel-stream
  • camel-streamcaching-test
  • camel-stringtemplate
  • camel-stripe
  • camel-stub
  • camel-support
  • camel-swift
  • camel-syslog
  • camel-tahu
  • camel-tarfile
  • camel-telegram
  • camel-telemetry
  • camel-telemetry-dev
  • camel-tensorflow-serving
  • camel-test-infra-all
  • camel-test-infra-artemis
  • camel-test-infra-cli
  • camel-test-infra-core
  • camel-test-infra-smb
  • camel-test-junit5
  • camel-test-junit6
  • camel-test-main-junit5
  • camel-test-main-junit6
  • camel-test-parent
  • camel-test-spring-junit5
  • camel-test-spring-junit6
  • camel-thrift
  • camel-thymeleaf
  • camel-tika
  • camel-timer
  • camel-tooling-maven
  • camel-toon
  • camel-tracing
  • camel-twilio
  • camel-twitter
  • camel-ubl
  • camel-undertow
  • camel-undertow-spring-security
  • camel-univocity-parsers
  • camel-validator
  • camel-velocity
  • camel-vertx
  • camel-vertx-common
  • camel-vertx-http
  • camel-vertx-parent
  • camel-vertx-websocket
  • camel-wal
  • camel-wasm
  • camel-weather
  • camel-weaviate
  • camel-web3j
  • camel-webhook
  • camel-whatsapp
  • camel-wordpress
  • camel-workday
  • camel-xchange
  • camel-xj
  • camel-xml-io
  • camel-xml-io-dsl
  • camel-xml-jaxb
  • camel-xml-jaxb-dsl
  • camel-xml-jaxb-dsl-test-definition
  • camel-xml-jaxb-dsl-test-spring
  • camel-xml-jaxp
  • camel-xmlsecurity
  • camel-xmpp
  • camel-xpath
  • camel-xslt
  • camel-xslt-saxon
  • camel-yaml-dsl-common
  • camel-yaml-dsl-validator
  • camel-yaml-dsl-validator-maven-plugin
  • camel-yaml-io
  • camel-zendesk
  • camel-zip-deflater
  • camel-zipfile
  • camel-zookeeper
  • camel-zookeeper-master
  • components
Modules with tests skipped (26)
  • apache-camel
  • camel-allcomponents
  • camel-catalog-console
  • camel-catalog-maven
  • camel-catalog-suggest
  • camel-componentdsl
  • camel-endpointdsl
  • camel-endpointdsl-support
  • camel-itest
  • camel-jbang-core
  • camel-jbang-it
  • camel-jbang-main
  • camel-jbang-plugin-edit
  • camel-jbang-plugin-generate
  • camel-jbang-plugin-kubernetes
  • camel-jbang-plugin-test
  • camel-kamelet-main
  • camel-launcher
  • camel-report-maven-plugin
  • camel-route-parser
  • camel-yaml-dsl
  • camel-yaml-dsl-deserializers
  • camel-yaml-dsl-maven-plugin
  • coverage
  • docs
  • dummy-component

ℹ️ Shadow mode — Scalpel observes but does not affect test execution. Learn more

⚠️ Some tests are disabled on GitHub Actions (@DisabledIfSystemProperty(named = "ci.env.name")) and require manual verification:

  • components/camel-kafka: 2 test(s) disabled on GitHub Actions
  • core/camel-core: 2 test(s) disabled on GitHub Actions
Build reactor — dependencies compiled but only changed modules were tested (10 modules)
  • Camel :: Cassandra CQL
  • Camel :: Catalog :: Camel Catalog
  • Camel :: Core
  • Camel :: Core Engine
  • Camel :: Ehcache
  • Camel :: JCache
  • Camel :: Kafka
  • Camel :: Redis
  • Camel :: SQL
  • Camel :: Support

⚙️ View full build and test results

…avadoc improvements

- Add comprehensive KeyValueRepository backends documentation page (keyValueRepository.adoc)
  covering all 11 backends with configuration examples, atomicity guarantees,
  TTL behavior, and adapter usage for Idempotent Consumer / Aggregator patterns
- Add atomic replace() and delete(key,expected) overrides to JdbcKeyValueRepository
  using SQL WHERE clause on ITEM_VALUE for server-side CAS
- Add Javadoc about non-atomic put() in Cassandra, Ehcache, and JCache backends
- Add Javadoc about TTL/topic growth and non-atomic CAS in KafkaKeyValueRepository
- Add 'Distributed Deployment' section to keyValueRepository.adoc with a table
  showing which backends are safe for clustered Camel deployments (shared state,
  distributed CAS, practical recommendations per pattern)
- Fix atomicity table: JDBC and Cassandra now correctly marked as atomic CAS (✓)
  reflecting the CAS overrides added in this PR and existing LWT support
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

✅ Generated files are up to date

An earlier CI run reported uncommitted generated changes; the latest run no longer does.

@gnodet gnodet added this to the 4.23.0 milestone Sep 3, 2026
@gnodet
gnodet merged commit 2d6568c into apache:main Sep 3, 2026
6 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants