channeldb: refactor `UpdateInvoice` to make it simpler to create SQL specific implementation #8100

bhandras · 2023-10-17T18:14:04Z

This PR is a collection of small changes refactoring the UpdateInvoice call and also InvoiceDB and InvoiceRegistry tests to be able to make InvoiceDB fully implementable both in k/v and SQL versions while testing with the same unit tests.

Part of: #6288

This change is

bhandras · 2023-10-31T17:31:12Z

I'm not 100% about this approach. Another would be to more explicitly flag changes in the update contexts as currently we pass around the changed invoice and assume that the underlying updater implementation will know what to do. For the key/value case it's simple as we just serialize and store.

Roasbeef · 2023-11-07T01:11:05Z

I'm not 100% about this approach

Just about to dive in my prior to this my mental model was something like:

Expose an InvoiceUpdate interface. This is an abstraction over a database transaction that can modify an invoice. This is returned from some other parent interface via a method like BeginTx() InvoiceUpdater (or something like that).
This has public methods like AddHtlc/RemoveHtlc/UpdateStatus, etc. So basically an extraction of all the current attributes that get modified directly within the callback today.
Underneath, we map these into descriptors for each action (newHtlcDesc, etc). These then map to the individual UPDATE SQL statements (skipping a few layers of abstractions) to finally be updated.
If we abstract the mapping of descriptor -> updater, where updater is a method/callback, then for the KV paradigm, we just use functional options to directly modify the invoice state. For the SQL paradigm, this is the set of update operations, with them being able to be aggregated for batching purposes.

Roasbeef

Great initial stab at this very important refactor!

Dropped some ideas of an alternative where we center the abstractions around encapsulated invoice updates, with the db transaction abstracted over using something similar to your current InvoiceUpdater. Lemmie know what you think, I can work on some PoC API code if we link this direction better. Goal here is to have most of the biz logic in the invoice registry, a middleware that takes the abstract updates to map into concrete db operations, then the lowest level that actually applies the updates.

For that middle layer, we'd have a unified interface that the invoice registry would use to queue up the updates. Then for KV land and SQL land we have distinct implementations. Then future backends implement the mapping from the middleware interface to how the operations would be expressed using that concrete backend.

Roasbeef · 2023-11-07T01:14:19Z

channeldb/invoices.go

+	return invoice, nil
+}
+
+func (d *DB) addHTLCsStoreUpdate(invoices, settleIndex, setIDIndex kvdb.RwBucket,


Cool, I think this is inline with what I had in my pending mental model. Can be abstracted one layer further as well, since rn it gives it raw access to the various buckets. I'm thinking we decouple the description of the update, from the application itself. Under that mental model, this function is the application, with the set of arguments making up part of the description. The gap here then is taking a method call and mapping it to some description, to be applied atomically at a later step.

One way to sort feel out these set of changes father would be to remove the Invoice argument from most/all of the update methods. Then the set of methods would be operating on an abstract InvoiceUpdate interface, which collects the updates, then applies them. For KV land application can just be mutate the invoice then sync to disk.

I was also thinking this would be one layer higher. So most of this biz logic is now in the invoice package, then the idealized channeldb package is just doing the update (tho it is the case that rn we have a lot of biz logic in chanenldb, and part of the original UpdateInvoice call back was meant to help us start to extract that logic.

Please note that the first few commits are just preliminary refactors in order to introduce the InvoiceUpdater and also to decouple in-memory invoice updates and DB updates. I should have flagged this in the PR description, but my general idea to do the least intrusive changes was to:

separate DB updates to their own functions

introduce a common interface and KV implementation for the updates

move all the update invoice logic to the invoices package

move all non-kv related tests to the invoices package too

This way we can implement an updater that's compatible with the existing implementation and works on SQL. The challenge here imho is the right level of abstraction as well as being careful with any changes to the Invoice or the in-memory update logic itself since our test coverage is somewhat limited. My choice here was to first make things work with a SQL updater in a naive way, and once we can deprecate the KV invoice DB do a deeper refactor to have a more SQL friendly implementation.

I'm looking into your idea of a more streamlined approach as I agree that having helpful types and more interfaces can generally help.

In the mean time I'm working on the separate branch in parallel to actually implement the missing bits for the SQL port, so this PR sometimes changes as I discover simpler or more elegant ways to separate things.

Roasbeef · 2023-11-07T01:16:20Z

channeldb/invoices.go

 	if err != nil {
 		return nil, err
 	}

 	return invoice, nil
 }

+func (d *DB) cancelInvoiceStoreUpdate(invoices kvdb.RwBucket, invoiceNum []byte,
+	invoice *invpkg.Invoice) error {
+


Not much distance between this and the old call site?

This is really just a boring preliminary step to separate in-memory vs storage stuff.

channeldb/invoices.go

Roasbeef · 2023-11-07T01:21:11Z

channeldb/invoices.go

 	}

 	switch update.UpdateType {
 	case invpkg.CancelHTLCsUpdate:
-		return d.cancelHTLCs(invoices, invoiceNum, &invoice, update)
+		return d.cancelHTLCs(invoices, invoiceNum, invoice, update)


I think the enum above is part of the way there, but rather then switch on the enum, we make a new InvoiceUpdater from the invoice itself (the KV impl maybe still copies like above), then we do ApplyUpdate on each of the abstract update descs, which are the items in the callback today), then finally commit. The internal impl then does a type switch or w/e on each of the updates to figure out how to apply it.

I choose to go with a similar although quite different approach. The update desc and in-memory update stays the same to avoid very intrusive code changes and instead the in-memory stuff is restructured a bit so that the DB stuff can be more fine-grained (which is ideal for the SQL stuff). PTAL all the commits, especially the one adding the InvoiceUpdater which will clear up why these changes are made.

Roasbeef · 2023-11-07T01:21:46Z

channeldb/invoices.go

+// invoice in the database. The methods of this interface are called when the
+// in-memory update of an invoice is complete, and the database needs to be
+// updated.
+type InvoiceUpdater interface {


great minds think alike ;)

Roasbeef · 2023-11-07T01:28:55Z

channeldb/invoices.go

+type InvoiceUpdater interface {
+	// StoreCancelHtlcsUpdate updates the invoice in the database after
+	// cancelling a set of HTLCs.
+	StoreCancelHtlcsUpdate(InvoiceUpdaterContext) error


What about a slightly diff model where:

We have two main methods ApplyUpdate, then Commit.

ApplyUpdate takes something similar to InvoiceUpdaterContext, but specific for each type of update. This is a new interface, InvoiceUpdate which is then destructed within the ApplyUpdate method to figure. out how to apply it.

Commit makes the db transaction, and tries to commit it all. If this fails, everything else above fails as well.

Eg we can take StoreAddHtlcsUpdate and make a new struct of NewHtlcUpdate that just has a field of the HTLCs to update, and w/e other information. This gets passed into ApplyUpdate that figures out how to stage it, if the state is invalid, etc. Ideally we can have the concrete implementation of the *Update structs handle the validation. Eg: you can't add more HTLCs if the state of the invoice is already cancelled, etc. Then we have all these smaller update modules, that can be tested in isolation.

The main InvoiceUpdate interface can be pretty slim. Mainly just to allow polymorphism to simplify the call site at the invoice registry: figure out all updates, make a new updater, apply then all, then commit. So maybe:

type InvoiceUpdate interface { InvoiceRef() invoice.InvoiceRef Type() invoice.UpdateType ValidateUpdate(oldInvoice *Invoice) error }

Thank you for the great suggestion!

IIUC what you describe is essentially the same model that's present in the PR + the existing in-memory update methods, with the added benefit of more streamlined approach through types and interfaces. So in this proposed approach the "ApplyUpdate" part is where we apply the update to the copied invoice in memory (with the existing legacy methods) and "Commit" is the InvoiceUpdater interface without the nice abstraction since we pass around the mutated+new state in the context.

I see a deeper refactor where the invoices.Invoice type could only be mutated through specific methods and the update itself would just call into those methods. That way the mutated state could just be queue'd up and committed to the database later. I'm open to such big change too, but its also not without risk as it implies changes to existing tests etc.

I think your suggestion is very similar to what @ziggie1984 expressed below, but I also think that doing a full-fledged "deep" refactor would be a lot more work that I'd like to first justify vs a simpler approach presented here. I think a more gradual multi PR refactor could be built as follow up once the e2e functionality is rock solid. The approach presented in this PR has a working SQL implementation too which we could use (once it's solidified) as a v1. wdyt?

Roasbeef · 2023-11-07T01:30:08Z

invoices/interface.go

+// invoice in the database. The methods of this interface are called when the
+// in-memory update of an invoice is complete, and the database needs to be
+// updated.
+type InvoiceUpdater interface {


+1 for having most of the logic here in the invoices package

Then we have another layer between this and the DB, that's responsible for mapping the invoice updates into concrete DB operations.

My main goal for this was to completely decouple invoice tests from the channeldb package such that we're easily able to test against other InvoiceDB implementations such as the SQL one.

bhandras · 2023-11-14T22:47:14Z

Please take reviews on-hold while I'm working on a larger refactor to accommodate suggestions by Laolu. Will move out of draft when ready for another look.

ziggie1984 · 2023-11-16T14:47:19Z

Really important and also difficult PR you are working on, respect 🤝 !

I like your approach and how you separated a lot of data from the db to the invoice package.

What caught my eye in this new approach is, that we have a lot of functions not really used in the UpdateInvoicer implementation of the KV world (Example: AddHtlc or ResolveHtlc) leaning on the Interface Segregation Principle I think we can avoid it tho I am not sure whether we are going to move some code back into the channeldb for the KV env.

I like the proposal of laolu abstracting everything even further.

I think this approach would keep most of your code but separate them even more, wdyt ?

For example:

In the InvoiceDB we add another interface as mentioned above:

type InvoiceDB interface {
...

	// DeleteCanceledInvoices removes all canceled invoices from the
	// database.
	DeleteCanceledInvoices(ctx context.Context) error

        // Interface used for updating the invoice in the db
	DBUpdater
}

This DBUpdater return your UpdateInvoicer interface:

type DBUpdater interface {
	BeginTx() InvoiceUpdater
}

type InvoiceUpdater interface {
	ApplyUpdate(UpdateInvoice) error
	Commit() error
}

The UpdateInvoice provides all the information which update we want to supply and what I think is very important to extract all the Validation logic which lives currently in more than one place just in one function ?

type UpdateInvoice interface {
	InvoiceRef() InvoiceRef
	Type() UpdateType
	ValidateUpdate(oldInvoice *Invoice) error
}

Then for a particular invoice update, let's say for the AddHTLC case currently living in notifyExitHopHtlcLocked the content of the callback would be straight forward:


	callback := func(inv *Invoice)  error {
               
                //Create the instance for adding an HTLC which implements the `UpdateInvoice` interface
                newHTLC := newUpdate( inv invoiceUpdateCtx, ADD type)
                 
               // Validate the new update
               err := newHTLC.ValidateUpdate(inv)

               //Get the UpdaterInvoicer 
               updater := i.db.BeginTx()                
               
              //Apply the update, maybe allow for more than 1 update
              err := updater.ApplyUpdate(newHTLC)
               
              err := updater.Commit()
    
	}

Then all the stuff how the Update is staged would be separate for both backends, so we would get rid of all the function currently not used by the KV-World ?

I have to point out your solution does also the job as well and I am not sure how much redundant code we would generate in the both ApplyUpdate functions of the KV and the SQL world. Moreover the above approach would move some code back into the channeldb package.

@Roasbeef you were talking about a middle-layer would be interesting how you would envision this interface to look like ?

bhandras

Thank you all for the reviews! 🙏 Let me just quickly also ask that if you would please ignore commenting on bfcddd9 as that commit just moves code around and I'd like to not add anything there to simplify the rebases a bit (as git is unable to properly track changes).

channeldb/invoices.go

bhandras · 2024-02-15T13:48:03Z

channeldb/invoices.go

@@ -2688,10 +2709,10 @@ func updateHtlc(resolveTime time.Time, htlc *invpkg.InvoiceHTLC,
 		return trySettle(false)

 	case invpkg.ContractOpen:


IIUC if the invoice is AMP then by accepting an HTLC we'd still have the state as ContractOpen. To validate I've commented this case and ran the unit tests and it failed exactly when we were trying to accept an AMP HTLC on a newly added invoice.

channeldb/invoices.go

bhandras · 2024-02-15T14:22:56Z

channeldb/invoices.go

+
+	// invoice is the invoice that we're updating. As a side effect of the
+	// update this invoice will be mutated.
+	invoice *invpkg.Invoice


Well we're actually changing the invoice, because the update to the fetched invoice is done in-memory too. The other option would be to do the update purely in the db on the copied invoice and then fetch it again?

bhandras · 2024-02-15T16:55:20Z

invoices/update_invoice.go

+		return invoice, nil
+	}
+
+	switch update.UpdateType {


I agree there's much room for more refactoring. Maybe we can do deeper cleanups in a follow up once the SQL version is in. WDYT?

invoices/update_invoice.go

bhandras · 2024-02-15T17:27:57Z

invoices/update_invoice.go

+	invoiceState ContractState, setID *[32]byte) (
+	bool, HtlcState, error) {
+
+	trySettle := func(persist bool) (bool, HtlcState, error) {


Since this commit is just moving code around, perhaps best to recreate these comments on other commits as every time I rebase I have to manually recreate this commit. Also maybe just to reduce the scope of the PR, perhaps we could do these refactors in a follow up. WDYT?

yyforyongyu · 2024-02-19T17:20:30Z

IMO this is good to go once the questions and the potential nil panic are resolved. It'd be great to have this merged so we can focus on its following PR. Again great work on the separation of handling ephemeral and permanent states of invoices👏

Just to be clear, the followings are non-blocking, as refactoring can be an endless endeavor,

keep unit tests small by leveraging mocks. The InvoiceUpdater has multiple implementations, each implementation has its own logic that should be tested separately. On the other hand, the invoices package should know nothing more than the interface to avoid leaking implementation details. On the other hand, we have itest to test the integrated behavior.
there's a circular import between channeldb and invoices, usually it says something about the design. And I'm curious about the alternative, that we put all implementaions of the InvoiceUpdater under invoices, giving us an MPV-ish design pattern. Asking this because soon I'll be working on the payments db, and wanna know if there're some serious pitfalls I'm not seeing by applying this alternative, as I don't wanna grow sqldb like how channeldb did.

channeldb/invoices.go

yyforyongyu · 2024-02-19T01:07:01Z

channeldb/invoices.go

+
+	ampState.InvoiceKeys[circuitKey] = struct{}{}
+
+	// Due to the way maps work, we need to read out the value, update it,


Think the comment is a bit outdated, also inaccurate since if we store a pointer then we won't have this issue. Maybe remove it?

yyforyongyu · 2024-02-19T01:21:31Z

channeldb/invoices.go

-			continue
+		// Verify that we don't get an action for htlcs that are not
+		// present on the invoice.
+		if !exists {


wanna note that this does change the behavior a bit - previously we'd cancel the intersection of the two sets then error out, now we'd error out before canceling the rest, I guess this would lead to some htlcs hanging?

Yeah I agree this looks suspicious but the overall behavior won't change because any error during the update will abort the transaction.

yyforyongyu · 2024-02-19T01:30:50Z

channeldb/invoices.go

-			htlc.State = htlcState
-			htlc.ResolveTime = resolveTime
+		newState := htlc.State
+		if persist && settled {


persist can be removed now since we are not mutating here?

Indeed, done.

yyforyongyu · 2024-02-19T01:46:17Z

channeldb/invoices.go

+// validate the state transition if we're cancelling the invoice.
+func getUpdatedInvoiceState(invoice *invpkg.Invoice, hash *lntypes.Hash,
+	update invpkg.InvoiceStateUpdateDesc) (*invpkg.ContractState,
+	*lntypes.Preimage, error) {


still getting through, but this method behaves more like a validator than a getter - we pass in an update, and return update.NewState and update.Preimage. I think at least we can stop returning preimage here and let the caller access it via update.Preimage. Or better, do something like canCancelSingleHtlc and only return an error.

Good idea! Done.

channeldb/invoices.go

yyforyongyu · 2024-02-19T03:09:13Z

channeldb/invoices.go

+	valueLen := copy(indexKey[:], k.invoiceNum)
+
+	if setID != nil {
+		valueLen += copy(indexKey[valueLen:], setID[:])


I don't think the copied date is used here so valueLen can be removed.

Let's keep it for readability.

by readability you mean to keep it so it's more like a pure code move than any functionality change?

yyforyongyu · 2024-02-19T03:40:53Z

invoices/update_invoice.go

+			AcceptTime:    updateTime,
+			State:         HtlcStateAccepted,
+			CustomRecords: htlcUpdate.CustomRecords,
+			AMP:           htlcUpdate.AMP.Copy(),


this may panic?

yyforyongyu · 2024-02-19T05:13:26Z

invoices/invoiceregistry_test.go

+// InvoiceDB instance. The purpose of this test is to be able to run all tests
+// with a custom DB instance, so that we can test the same logic with different
+// DB implementations.
+func TestInvoiceRegistry(t *testing.T) {


I'm not in favor of this testing setup - why are we testing the database implementation details in invoices, while each implementation should have their own dedicated unit tests? It's wonderful that we finally have the InvoiceUpdater interface, and our tests should stop at the interface, with each interface method being mocked using its defined rules, and provide more robust unit tests at the implementation side. Meanwhile it's more suitable to use integration tests to check different db behaviors.

bhandras · 2024-02-19T18:11:42Z

IMO this is good to go once the questions and the potential nil panic are resolved. It'd be great to have this merged so we can focus on its following PR. Again great work on the separation of handling ephemeral and permanent states of invoices👏

Just to be clear, the followings are non-blocking, as refactoring can be an endless endeavor,

Thanks for the review, will address comments asap!

keep unit tests small by leveraging mocks. The InvoiceUpdater has multiple implementations, each implementation has its own logic that should be tested separately. On the other hand, the invoices package should know nothing more than the interface to avoid leaking implementation details. On the other hand, we have itest to test the integrated behavior.

Something that might not be super obvious that there're plenty of tests that test the integration of the updater (invoices/invoices_test.go and invoices/invoiceregistry_test.go) so it's not only itests that are exercising the new boilerplate code that is added in the PR. I'm fairly confident that the existing test suite covers the update well, but also agree that we can always add more tests and that generally the refactor is kept minimal so future changes needed to make the code more testable as well.

there's a circular import between channeldb and invoices, usually it says something about the design. And I'm curious about the alternative, that we put all implementaions of the InvoiceUpdater under invoices, giving us an MPV-ish design pattern. Asking this because soon I'll be working on the payments db, and wanna know if there're some serious pitfalls I'm not seeing by applying this alternative, as I don't wanna grow sqldb like how channeldb did.

Fortunately invoices as a package does only import channeldb in tests. It's required to be able to later run the same set of test cases on all backends (bolt, etcd, sqlite and postgres). I'm not sure how to resolve this ugly import cycle without introducing one more top level package but I think it's ok if it is only for tests.

bhandras

Thanks for the super quick review 🏎️ @yyforyongyu 🙏

channeldb/invoices.go

bhandras · 2024-02-19T18:38:26Z

channeldb/invoices.go

+
+	ampState.InvoiceKeys[circuitKey] = struct{}{}
+
+	// Due to the way maps work, we need to read out the value, update it,


bhandras · 2024-02-19T18:43:10Z

channeldb/invoices.go

-			continue
+		// Verify that we don't get an action for htlcs that are not
+		// present on the invoice.
+		if !exists {


Yeah I agree this looks suspicious but the overall behavior won't change because any error during the update will abort the transaction.

bhandras · 2024-02-19T18:54:12Z

channeldb/invoices.go

@@ -2293,14 +2302,20 @@ func (d *DB) settleHodlInvoice(invoices, settleIndex kvdb.RwBucket,
 	// TODO(positiveblue): this logic can be further simplified.
 	var amtPaid lnwire.MilliSatoshi
 	for _, htlc := range invoice.Htlcs {
-		_, err := updateHtlc(
-			timestamp, htlc, invpkg.ContractSettled, nil,
+		if htlc.State == invpkg.HtlcStateSettled {


Sorry this was indeed wrong. Fixed.

channeldb/invoices.go

bhandras · 2024-02-19T19:16:22Z

channeldb/invoices.go

+	valueLen := copy(indexKey[:], k.invoiceNum)
+
+	if setID != nil {
+		valueLen += copy(indexKey[valueLen:], setID[:])


Let's keep it for readability.

bhandras · 2024-02-19T19:17:04Z

invoices/update_invoice.go

+			AcceptTime:    updateTime,
+			State:         HtlcStateAccepted,
+			CustomRecords: htlcUpdate.CustomRecords,
+			AMP:           htlcUpdate.AMP.Copy(),


Ok this is now fixed.

invoices/invoiceregistry_test.go

bhandras · 2024-02-19T19:27:22Z

invoices/invoiceregistry_test.go

+// InvoiceDB instance. The purpose of this test is to be able to run all tests
+// with a custom DB instance, so that we can test the same logic with different
+// DB implementations.
+func TestInvoiceRegistry(t *testing.T) {


I think it's great to be able to run the quick and simple to debug unit tests on all backends. This ha been essential for me when porting invoice db to SQL, otherwise i guess iteration speed would have been much slower.

I see your concern though, but I feel like given that the InvoiceDB interface itself is defined in the invoices package it's probably best to keep the tests here and implementation details elsewhere. Another option (for the future) might be to move the k/v implementation under invoices/kvdb and the sql implementation under invoices/sql.

This commit is a small refactor to move all actual DB updates after an invoice state is update to separate methods. This is a small preliminary change before we completely decouple DB updates from in-memory invocie update.

With this refactor updateHtlc is renamed to getUpdatedHtlcState and changed such that it won't change the HTLC's state and resolve time but instead returns whether the change is needed. This change is part of a multi-commit refactor to ensure that all changes to the invoice will be tracked individually.

This commit turns updateInvoiceState "const" by moving preimage update out of the function while also removing it to getUpdatedInvoiceState.

This change moves the HTLC state change out of the cancelSingleHtlc function. This is part of the larger refactor of collecting all changes to be later applied by the invoice updater.

With this commit updateInvoiceAmpState becomes getUpdatedInvoiceAmpState which will only return the new AMP state but that needs to be applied at the call site. This is a part of a larger refactor to gather all mutations of an invoice update to be later applied by the invoice updater.

This commit introduces the InvoiceUpdater interface which is meant to abstract and assist the in-memory invoice update procedure with the accompanying database updates. These abstract updater steps will enable further refactoring later while also ensuring that a full SQL implementation of the InvoiceDB interface will be possible.

With the introducation of the `InvoiceUpdater` interface we are now able to move the non-kv parts of `UpdateInvoice` completely under the invoices package. This is a preprequisite for being able to use the same code-base for the sql InvoiceDB implementation of UpdateInvoice.

This commit extracts the InvoiceDB construction from all invoice and registry tests such that we can later on run subtests with multiple backends without needing to use tags.

yyforyongyu

LGTM🌮 Thanks again for taking the effort!

PS, linter failed with no space left on device, seems like the CI ran out of resource?

yyforyongyu · 2024-02-19T23:13:59Z

channeldb/invoices.go

+	valueLen := copy(indexKey[:], k.invoiceNum)
+
+	if setID != nil {
+		valueLen += copy(indexKey[valueLen:], setID[:])


by readability you mean to keep it so it's more like a pure code move than any functionality change?

lightninglabs-deploy · 2024-02-26T23:39:01Z

@positiveblue: review reminder
@Roasbeef: review reminder
@ziggie1984: review reminder

Roasbeef · 2024-02-28T00:37:08Z

Linter looks good now after a kick!

Roasbeef

LGTM 👑

Reviewed 5 of 5 files at r5, 4 of 4 files at r6, 10 of 10 files at r7, all commit messages.
Reviewable status: all files reviewed, 83 unresolved discussions (waiting on @bhandras, @positiveblue, @yyforyongyu, and @ziggie1984)

bhandras force-pushed the updateinvoice-refactor branch 4 times, most recently from e3dd68a to 68d34a3 Compare October 20, 2023 11:10

bhandras force-pushed the updateinvoice-refactor branch from 68d34a3 to f91c3fd Compare October 26, 2023 12:42

bhandras changed the title ~~[wip] channeldb: refactor UpdateInvoice to make it simpler to create SQL specific implementation~~ channeldb: refactor UpdateInvoice to make it simpler to create SQL specific implementation Oct 26, 2023

bhandras added this to the v0.18.0 milestone Oct 26, 2023

bhandras marked this pull request as ready for review October 26, 2023 12:47

bhandras self-assigned this Oct 26, 2023

bhandras added database Related to the database/storage of LND refactoring labels Oct 26, 2023

saubyk requested a review from ziggie1984 October 26, 2023 18:45

bhandras force-pushed the updateinvoice-refactor branch 3 times, most recently from 5a99f61 to a535c6e Compare October 30, 2023 20:09

saubyk requested a review from Roasbeef October 31, 2023 17:17

bhandras force-pushed the updateinvoice-refactor branch 3 times, most recently from 25bd7c9 to 94775ea Compare November 6, 2023 20:05

Roasbeef requested changes Nov 7, 2023

View reviewed changes

bhandras force-pushed the updateinvoice-refactor branch from 913f01d to bd604c9 Compare November 14, 2023 22:45

bhandras marked this pull request as draft November 14, 2023 22:45

bhandras force-pushed the updateinvoice-refactor branch from bd604c9 to 5e16ec8 Compare November 15, 2023 22:44

bhandras force-pushed the updateinvoice-refactor branch 2 times, most recently from 259b2ae to fbd6170 Compare November 17, 2023 19:21

bhandras mentioned this pull request Nov 22, 2023

sqldb: InvoiceDB implementation #8052

Merged

bhandras commented Feb 15, 2024

View reviewed changes

bhandras requested a review from ziggie1984 February 15, 2024 17:32

yyforyongyu reviewed Feb 19, 2024

View reviewed changes

bhandras force-pushed the updateinvoice-refactor branch from 487ad5d to e3446dc Compare February 19, 2024 19:34

bhandras commented Feb 19, 2024

View reviewed changes

bhandras requested a review from yyforyongyu February 19, 2024 19:35

bhandras added 15 commits February 19, 2024 20:47

channeldb: extract store update methods when updaing an invoice

eb4198b

This commit is a small refactor to move all actual DB updates after an invoice state is update to separate methods. This is a small preliminary change before we completely decouple DB updates from in-memory invocie update.

channeldb: fetch the invoice before calling into updateInvoice

4bf6b52

channeldb: extract AMP state updates to updateInvoiceAmpState()

7298b2d

channeldb: do not change the update descriptor when cancelling htlcs

ef5a317

invoices: rename updateInvoiceState to getUpdatedInvoiceState

87044b8

This commit turns updateInvoiceState "const" by moving preimage update out of the function while also removing it to getUpdatedInvoiceState.

invoices: change cancelSingleHtlc to be purely used for validation

08df7f4

This change moves the HTLC state change out of the cancelSingleHtlc function. This is part of the larger refactor of collecting all changes to be later applied by the invoice updater.

invoices: move InvoiceDB tests unrelated to kvdb to invoices package

5e746b4

invoices: fix linter issues after the move

0e2b39e

invoices: parametrize invoice(registry) tests with InvoiceDB constructor

7d56d53

This commit extracts the InvoiceDB construction from all invoice and registry tests such that we can later on run subtests with multiple backends without needing to use tags.

mod: bump kvdb to v1.4.5

f5abc94

docs: update release notes for 0.18.0

bcc6a3f

bhandras force-pushed the updateinvoice-refactor branch from e3446dc to bcc6a3f Compare February 19, 2024 19:47

yyforyongyu approved these changes Feb 19, 2024

View reviewed changes

Roasbeef requested a review from yyforyongyu February 28, 2024 00:49

Roasbeef approved these changes Feb 28, 2024

View reviewed changes

Roasbeef merged commit 72764b1 into lightningnetwork:master Feb 28, 2024
24 of 25 checks passed

		@@ -2688,10 +2709,10 @@ func updateHtlc(resolveTime time.Time, htlc *invpkg.InvoiceHTLC,
		return trySettle(false)

		case invpkg.ContractOpen:


		ampState.InvoiceKeys[circuitKey] = struct{}{}

		// Due to the way maps work, we need to read out the value, update it,

channeldb: refactor UpdateInvoice to make it simpler to create SQL specific implementation #8100

channeldb: refactor UpdateInvoice to make it simpler to create SQL specific implementation #8100

Conversation

bhandras commented Oct 17, 2023 • edited by Roasbeef

bhandras commented Oct 31, 2023

Roasbeef commented Nov 7, 2023

Roasbeef left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bhandras Nov 7, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bhandras commented Nov 14, 2023

ziggie1984 commented Nov 16, 2023

bhandras left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yyforyongyu commented Feb 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bhandras commented Feb 19, 2024 • edited

bhandras left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yyforyongyu left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lightninglabs-deploy commented Feb 26, 2024

Roasbeef commented Feb 28, 2024

Roasbeef left a comment

Choose a reason for hiding this comment

channeldb: refactor `UpdateInvoice` to make it simpler to create SQL specific implementation #8100

channeldb: refactor `UpdateInvoice` to make it simpler to create SQL specific implementation #8100

bhandras commented Oct 17, 2023 •

edited by Roasbeef

bhandras Nov 7, 2023 •

edited

bhandras commented Feb 19, 2024 •

edited

yyforyongyu left a comment •

edited