invoices: refactor InvoiceDB to eliminate ScanInvoices
#8081
Conversation
bhandras
force-pushed
the
invoicedb-refactor
branch
from
113b633
to
7151759
Compare
bhandras
force-pushed
the
invoicedb-refactor
branch
from
7151759
to
8f051f3
Compare
|
|
||
| db, err := MakeTestInvoiceDB(t, OptionClock(testClock)) | ||
| require.NoError(t, err, "unable to make test db") | ||
|
|
There was a problem hiding this comment.
can we also add a case here to test the empty slice returned from FetchPendingInvoices?
There was a problem hiding this comment.
Yeah good idea! Done.
| invoiceIndexBucket, | ||
| ) | ||
| if invoiceIndex == nil { | ||
| return invpkg.ErrNoInvoicesCreated |
There was a problem hiding this comment.
Think we should unify the behaviors here - in FetchPendingInvoices we'd ignore this error, and we should. do the same here? In general I think DeleteCanceledInvoices should be a NOOP if there are no invoices in the db, instead of returning an error.
There was a problem hiding this comment.
This case is different though right? We do return nil above when we "open" the read-write bucket invoices however if that exist, we should assume that both the the invoice index and add index exists.
There was a problem hiding this comment.
could it be the case that this is just an empty bucket? like all the invoices are deleted?
There was a problem hiding this comment.
No, if the bucket is empty we'd still get back the handle.
There was a problem hiding this comment.
Cool - then I'm curious in what case will this ErrNoInvoicesCreated be returned?
There was a problem hiding this comment.
I think it's there to give us some reassurance that about expected existence of those buckets. Similar checks are done for other channeldb methods too.
There was a problem hiding this comment.
cool then I interpret it as a db corruption check.
| @@ -2761,6 +2761,102 @@ func delAMPSettleIndex(invoiceNum []byte, invoices, | |||
| return nil | |||
| } | |||
|
|
|||
| // DeleteCanceledInvoices deletes all canceled invoices from the database. | |||
| func (d *DB) DeleteCanceledInvoices(_ context.Context) error { | |||
| return kvdb.Update(d, func(tx kvdb.RwTx) error { | |||
There was a problem hiding this comment.
can we use batch here?
There was a problem hiding this comment.
I think it's probably not a good idea since we scan through all the invoices here and bolt's Batch method assumes that the closure can be executed multiple times since it'll only try to parallelize optimistically (given bolt has no idea about what we do here).
There was a problem hiding this comment.
I think it depends on whether there's an error and how we handle the error, from this bench test, if we distinguish the errors here, each closure is executed exactly once.
| @@ -3092,6 +3079,65 @@ func TestDeleteInvoices(t *testing.T) { | |||
| assertInvoiceCount(0) | |||
| } | |||
|
|
|||
| // TestDeleteCanceledInvoices tests that deleting canceled invoices with the | |||
| // specific DeleteCanceledInvoices method works correctly. | |||
| func TestDeleteCanceledInvoices(t *testing.T) { | |||
There was a problem hiding this comment.
I think it'd be great if we could add tests to cover more branches in this method, but guess it's too much work as most stuff here is not mocked.
There was a problem hiding this comment.
Yeah, maybe we should return to the test coverage question later on once we have both SQL and KV implementations run against this test suite since I'm afraid many of the branches we'd test will go away as we deprecate the KV implementation. Similarly TestDeleteInvoices is also very barebones, so perhaps a bigger overhaul is necessary.
In this commit we change how we select invoices to follow or delete when starting the InvoiceRegistry to instead of using the deperecated scan func from channeldb, use specific functions to gather pending and delete canceled invoices.
bhandras
force-pushed
the
invoicedb-refactor
branch
from
8f051f3
to
bedafca
Compare
This PR removes
ScanInvoicesfromInvoiceDB(and the channeldb implementation) and adds two specific methods that can be expressed in SQL more efficiently. Builds on top of #8066 and is a pre-requirement for #8052The downside of the refactor is that for k/v implementations we'll need to scan invoices twice, but since we're fully migrating to SQL this is just a temporary issue.
Part of #6288