feat: Added in-memory storage for testing purposes #59

Harshdev098 · 2025-10-12T08:40:04Z

Have added in_memory store for testing purpose.
We can edit config file to use specific store either postgresql or memory

ldk-reviews-bot · 2025-10-12T08:40:06Z

👋 Thanks for assigning @tankyleo as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

Harshdev098 · 2025-10-12T10:19:04Z

Hey @tnull @tankyleo Can you please review it

tnull

Thanks for looking into this!

Generally goes into the right direction, but we def. need to avoid re-allocating everything on every operation.

rust/Cargo.lock

rust/README.md

rust/impls/src/lib.rs

rust/impls/src/memory_store.rs

Harshdev098 · 2025-10-14T06:17:50Z

@tnull Have done the required changes

tnull

Looks much better, but I think we still need to handle global_version properly, even if we're currently not using it client-side.

tnull · 2025-10-14T07:58:30Z

rust/impls/src/in_memory_store.rs

+		}
+	}
+
+	async fn put(


Seems we'll also need to track a global_version and set it if it's set in the request?

@tnull I've added handling for global_version in put and delete:

In put:

If request.global_version is set, check it against the current global version

After successful transaction, increment the global version and store it.

In delete:

Increment the global version after a successful delete

ldk-reviews-bot · 2025-10-14T08:50:52Z

🔔 1st Reminder

Hey @tankyleo! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-10-16T08:51:33Z

🔔 2nd Reminder

Hey @tnull @tankyleo! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-10-16T16:44:43Z

🔔 1st Reminder

Hey @tnull @tankyleo! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

tnull

One comment, will take another look once @tankyleo also had a chance to do a review round here.

tnull · 2025-10-17T08:00:28Z

rust/impls/src/in_memory_store.rs

+			created_at: now,
+			last_updated_at: now,
+		};
+		guard.insert(global_key, record);


Shouldn't we only update the entry if it's already existing, so that we don't always override created_at? Maybe rather use guard.entry().or_insert_with(..) (here and elsewhere)?

Harshdev098 · 2025-10-17T09:15:56Z

@tankyleo Can you please review it!

ldk-reviews-bot · 2025-10-18T08:52:14Z

🔔 3rd Reminder

Hey @tankyleo! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-10-20T08:52:50Z

🔔 4th Reminder

Hey @tankyleo! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-10-22T08:53:27Z

🔔 5th Reminder

Hey @tankyleo! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

tankyleo

Sorry for the delay !

tankyleo · 2025-10-22T23:34:03Z

rust/impls/src/in_memory_store.rs

+		Self { store: Arc::new(Mutex::new(HashMap::new())) }
+	}
+
+	fn build_vss_record(&self, user_token: String, store_id: String, kv: KeyValue) -> VssDbRecord {


Let's make this into a helper function, as it does not need &self and does not return Self

tankyleo · 2025-10-22T23:34:28Z

rust/impls/src/in_memory_store.rs

+		}
+	}
+
+	fn build_key(user_token: &str, store_id: &str, key: &str) -> String {


Same here pull this out into a helper function in this file

tankyleo · 2025-10-22T23:42:32Z

rust/impls/src/in_memory_store.rs

+	match entry {
+		std::collections::hash_map::Entry::Occupied(mut occ) => {
+			let existing = occ.get_mut();
+			if existing.version >= record.version {


For already existing records, a conflict error should be returned anytime the versions don't match, unless record.version has been set to -1 (meaning non-conditional update).

tankyleo · 2025-10-22T23:49:11Z

rust/impls/src/in_memory_store.rs

+		if existing.version != record.version {
+			return Ok(());
+		}


If record.version is -1, this should be skipped and we should do a non-conditional delete.

tankyleo · 2025-10-22T23:59:11Z

rust/impls/src/in_memory_store.rs

+					version: record.version,
+				}),
+			})
+		} else if request.key == GLOBAL_VERSION_KEY {


Looks like by the time we are here, we know the GLOBAL_VERSION_KEY does not have a value, otherwise guard.get would have returned Some previously. We can just return the GetObjectResponse below directly with version: 0.

tankyleo · 2025-10-23T00:02:49Z

rust/impls/src/in_memory_store.rs

+		for kv in &request.transaction_items {
+			let key = Self::build_key(&user_token, &store_id, &kv.key);
+			if let Some(existing) = guard.get(&key) {
+				if existing.version >= kv.version {


Similar to the above comment, shouldn't this just make sure that the existing.version == kv.version, otherwise throw a conflict ? Unless the kv.version is -1.

tankyleo · 2025-10-23T00:04:29Z

rust/impls/src/in_memory_store.rs

+			let key = Self::build_key(&user_token, &store_id, &kv.key);
+			if let Some(existing) = guard.get(&key) {
+				if existing.version >= kv.version {
+					return Err(VssError::ConflictError(format!(


Let's pick a single spot to return ConflictError here similar to how things are done in postgres_store. As much as possible we should be checking for conflicts in a single spot.

rust/impls/src/in_memory_store.rs

tankyleo · 2025-10-23T00:08:20Z

rust/impls/src/in_memory_store.rs

+	use api::kv_store::INITIAL_RECORD_VERSION;
+	use bytes::Bytes;
+	use tokio::test;
+


Like we have in postgres_store, let's add a line similar to below here and make sure the tests pass

define_kv_store_tests!(InMemoryKvStoreTest, InMemoryBackendImpl, InMemoryBackendImpl::new());

tankyleo · 2025-10-23T00:16:35Z

rust/impls/src/migrations.rs

 pub(crate) const INIT_DB_CMD: &str = "CREATE DATABASE";
 #[cfg(test)]
-const DROP_DB_CMD: &str = "DROP DATABASE";
+pub(crate) const DROP_DB_CMD: &str = "DROP DATABASE";


Thanks ! let's pull the formatting changes in postgres_store + pub(crate) changes into a separate commit, and keep the in_memory_store changes to a single commit

Harshdev098 · 2025-10-26T01:53:09Z

@tankyleo Have done with the required changes! Can you please review it

tnull

When testing integration with LDK Node locally I found that the tests are currently failing. I now opened #62 to add LDK Node integration tests to our CI here. It would be great if that could land first, and we could also add a CI job for the in-memory store as part of this PR then, ensuring the implementation actually works as expected.

ldk-reviews-bot · 2025-10-28T01:53:31Z

🔔 1st Reminder

Hey @tankyleo! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot requested a review from tankyleo October 12, 2025 08:50

tnull self-requested a review October 13, 2025 07:10

tnull reviewed Oct 13, 2025

View reviewed changes

Harshdev098 force-pushed the memory_store branch 2 times, most recently from 4980a75 to 25d57e3 Compare October 14, 2025 06:16

Harshdev098 requested a review from tnull October 14, 2025 06:17

tnull reviewed Oct 14, 2025

View reviewed changes

Harshdev098 force-pushed the memory_store branch from 25d57e3 to 9012e95 Compare October 14, 2025 12:24

Harshdev098 requested a review from tnull October 14, 2025 16:44

tnull reviewed Oct 17, 2025

View reviewed changes

Harshdev098 force-pushed the memory_store branch from 9012e95 to 3b434d0 Compare October 17, 2025 09:14

tankyleo reviewed Oct 23, 2025

View reviewed changes

chore: formatted and pub(crate) visibility

ff3c7e5

Harshdev098 force-pushed the memory_store branch from 3b434d0 to e0c31bb Compare October 26, 2025 01:51

Harshdev098 requested a review from tankyleo October 26, 2025 01:53

feat: Added in-memory storage for testing purposes

5898609

Harshdev098 force-pushed the memory_store branch from 8003119 to 5898609 Compare October 26, 2025 02:18

tnull reviewed Oct 27, 2025

View reviewed changes

Uh oh!

feat: Added in-memory storage for testing purposes #59

Are you sure you want to change the base?

feat: Added in-memory storage for testing purposes #59

Uh oh!

Conversation

Harshdev098 commented Oct 12, 2025

Uh oh!

ldk-reviews-bot commented Oct 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Harshdev098 commented Oct 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tnull left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Harshdev098 commented Oct 14, 2025

Uh oh!

tnull left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ldk-reviews-bot commented Oct 14, 2025

Uh oh!

ldk-reviews-bot commented Oct 16, 2025

Uh oh!

ldk-reviews-bot commented Oct 16, 2025

Uh oh!

tnull left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Harshdev098 commented Oct 17, 2025

Uh oh!

ldk-reviews-bot commented Oct 18, 2025

Uh oh!

ldk-reviews-bot commented Oct 20, 2025

Uh oh!

ldk-reviews-bot commented Oct 22, 2025

Uh oh!

tankyleo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Harshdev098 commented Oct 26, 2025

Uh oh!

tnull left a comment

Choose a reason for hiding this comment

ldk-reviews-bot commented Oct 12, 2025 •

edited

Loading

Harshdev098 commented Oct 12, 2025 •

edited

Loading