New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add recursiveDelete() #1427
Conversation
f99da27
to
3d1fa79
Compare
Codecov Report
@@ Coverage Diff @@
## bc/rc-main #1427 +/- ##
==============================================
+ Coverage 98.20% 98.51% +0.31%
==============================================
Files 32 32
Lines 19502 19623 +121
Branches 1363 1296 -67
==============================================
+ Hits 19152 19332 +180
+ Misses 346 286 -60
- Partials 4 5 +1
Continue to review full report at Codecov.
|
dev/src/index.ts
Outdated
writer.flush().then(async () => { | ||
if (ref instanceof DocumentReference) { | ||
try { | ||
await ref.delete(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can't think of a more elegant way to differentiate between the final document reference delete failing vs. the BulkWriter deletes. Any suggestions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't looked at the rest of the PR, but if you want to work on it before I do - couldn't you just let the BulkWriter handle this delete as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still have a few more changes/cleanup I need to push upstream, so reviewing now might result in some wasted work.
Thanks for the suggestion! I originally figured a normal delete is less verbose than enqueuing another operation to BulkWriter and calling flush(), but using BulkWriter allows for any user callbacks to also run.
One more note: Can you ping the backend chat to determine whether we can retry deletes that fail with RST_STREAM? (I see this is already part of your TODOs, so please pretend that you never saw this note). |
dev/src/index.ts
Outdated
err.message = | ||
'Failed to fetch children documents. ' + | ||
'The provided reference was not deleted.'; | ||
deleteCompleted.reject(wrapError(err, stack)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is not deleting the provided reference in the case of a failed StructuredQuery the ideal, expected behavior? I need a quick sanity check. Here are the options:
- Do not delete the provided reference if StructuredQuery stream ever errors.
- Do not delete provided reference iff StructuredQuery stream errors before any deletes are enqueued.
- Delete the reference regardless of what happens.
I think we originally agreed on Option 2. It makes logical sense -- if Firestore can't fetch any of the descendants, that's a failed precondition, so Firestore shouldn't delete the provided reference. However, what's tripping me up is if the StructuredQuery stream errors out midway through the deletion process. In this case, an argument can be made both ways -- Firestore shouldn't delete the provided reference because it couldn't fetch the descendants (failed precondition argument) OR Firestore should delete the provided reference to be consistent in behavior (the provided reference is deleted even if BulkWriter fails to delete some references).
Trying to document all these edge cases led me to consider Option 3 -- if I'm a developer who simply wants the collection or document deleted, I don't care about the provided reference, and Firestore should just nuke whatever it can get a hold of.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The query can error at any point in time. I would suggest we make no guarantees whatsoever, which makes the overall behavior of the API much more predictable. If any operation fails, the database tree is in an undefined state, and the user needs to try again.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. Changed it to always delete the provided reference when possible, but kept the error message in for query stream failures. Not sure if we want the FAILED_PRECONDITION status code though.
); | ||
if (lastError instanceof BulkWriterError) { | ||
error.code = (lastError.code as number) as Status; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure why these lines aren't covered by Codecov, but I'm pretty sure they're being called. Maybe it's b/c I put it in a lambda function to access lastError
and errorCount
. Is this an anti-pattern?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Feedback for your implementation. I hope that I can get to the tests soon. They look a bit scary in that you wrote a lot of them :)
* | ||
* @private | ||
*/ | ||
private _bulkWriter: BulkWriter | undefined; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we move this to BulkWriter (kind of like a singleton pattern)? This could then be a shared instance that could be used by other operations in the future as well.
I also wonder whether the default ramp up makes sense here. It will slow down the delete (especially when compared to the CLI) and it also doesn't quite fit the use case for 5/5/5. We do not have to ramp up to give the database a chance to shard - the data is already sharded, we just want to delete it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we move this to BulkWriter (kind of like a singleton pattern)? This could then be a shared instance that could be used by other operations in the future as well.
Do you mean that calling firestore.bulkWriter()
would return the same instance each time, or having a Firestore-internal BulkWriter.bulkWriter()
getter that lazy-loads?
I also wonder whether the default ramp up makes sense here.
Yeah, that's something I was thinking about as well to implement as part of the RST_STREAM retries. Deletes aren't subject to 555 throttling, which creates two possibilities. 1) We can add an internal override on BulkWriter to turn off throttling when recursive delete is called. However, this would mean that writes being performed at the same time won't get throttled. 2) We can add logic for BulkWriter to automatically ignore throttling limits for deletes, but then we'd have to batch deletes separately from writes (or think of another solution that allows writes to still be throttled).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Firestore internal getter that lazy-loads. I want to keep it similar to what you have, but try to move these internals out of Firestore.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a static BulkWriter.bulkWriter()
method.
dev/src/index.ts
Outdated
err.message = | ||
'Failed to fetch children documents. ' + | ||
'The provided reference was not deleted.'; | ||
incrementErrorCount(err); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This shouldn't bump up the error count that is used for the error message, should it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. This made me realize that the # deletes failed
message won't be accurate if the stream fails since all the remaining docs not returned by the stream are unaccounted for. Should I special case the error message in the stream failure case to capture this fact?
dev/src/index.ts
Outdated
const writer = bulkWriter ?? this.getBulkWriter(); | ||
writer._verifyNotClosed(); | ||
const docStream = this.getAllDescendants(ref); | ||
const deleteCompletedDeferred = new Deferred<void>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this just be result
? While generic, it makes it clear that this is what the method returns.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Renamed to resultDeferred
. Since the return type is a void promise, I want to make it clear that there is no real "result" being returned.
dev/src/index.ts
Outdated
// TODO(chenbrian): Make this a private method after adding recursive delete. | ||
_getAllDescendants( | ||
ref: CollectionReference | DocumentReference | ||
private getAllDescendants<T = DocumentData>( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should still be underscore prefixed to prevent "usage by code completion".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Renamed. Aren't private methods hidden from code completion? How do they show up? Also, what's the difference between the _method()
and method_()
patterns in this file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"private" is only hidden from TS developers. JS developers will still see it in code completion.
Underscore suffix is the same as underscore prefix, but from about 10 years ago. I am surprised this client uses underscore suffix.
Co-authored-by: Sebastian Schmidt <mrschmidt@google.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Feedback for the tests. Generally looks great!
dev/test/index.ts
Outdated
try { | ||
expect(request.writes).to.deep.equal(expected.writes); | ||
} catch (e) { | ||
batchWriteError = e; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do the tests not fail if we simply let the exception throw here? Ideally, the implementation should be able to handle if the batchWrite
RPC throws an exception and propagate it as a failed BatchWrite.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a test that checks the error stack to make sure that the original error thrown in batchWrite
is surfaced. BulkWriter currently doesn't do any special handling for batchWrite
RPC failures, so recursive delete can't perform any handling logic for it. Do you think we should add logic to surface batchWrite
RPC failures differently?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, I think we can keep the logic as is. Individual RPC failures versus individual operations failures are not different from a user perspective. I was trying to see if we could simply bubble up the assertion failure as an individual RPC failure, which would surface these assertions in onWriteResult
or via failed promises rather than via batchWriteError
.
dev/test/index.ts
Outdated
firestore = await instantiateInstance([ | ||
['anna', 'bob', 'bob/children/charlie', 'bob/children/daniel'], | ||
]); | ||
await firestore.recursiveDelete(firestore.collection('collectionId')); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure I follow this test. The documents are not part of the collection, but they are still getting returned from the RunQuery
request if I am not mistaken.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The collectionId
is implicit in instantiateInstance()
, which follows the pattern that other test helpers use. So in this case, the documents under firestore.collection('collectionId')
are ['collectionId/anna', 'collectionId/bob', ...]
.
Added a comment to clarify.
dev/test/index.ts
Outdated
let callbackCount = 0; | ||
const bulkWriter = firestore.bulkWriter(); | ||
bulkWriter.onWriteResult(() => { | ||
callbackCount++; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we want to add a test that verifies that we return the expected list of DocumentReferences?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done.
dev/test/index.ts
Outdated
callbackCount++; | ||
}); | ||
await firestore.recursiveDelete(firestore.collection('foo'), bulkWriter); | ||
expect(callbackCount).to.equal(3); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We also don't seem to have tests that exercise the error handler.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added one.
dev/test/query.ts
Outdated
queryEqualsHelper(actual, '', ...protoComponents); | ||
} | ||
|
||
function queryEqualsHelper( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not a huge fan of Helpers
. Would it be possible to just make this queryEqualsWithParent
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A lot of tests use queryEquals()
without a parent
parameter, and I didn't want to pass in an empty string for parent
in all of them or change the spread operator to an array. Is it possible to combine these two signatures in an overload:
export function queryEquals(
actual: api.IRunQueryRequest | undefined,
parent: string,
...protoComponents: api.IStructuredQuery[]
): void;
export function queryEquals(
actual: api.IRunQueryRequest | undefined,
...protoComponents: api.IStructuredQuery[]
): void;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You could do:
export function queryEquals(
actual: api.IRunQueryRequest | undefined,
parentOrProtoComponent: string|api.IStructuredQuery|undefined,
...protoComponents: api.IStructuredQuery[]
): void;
But that is not all that pretty.
I was actually thinking you keep both queryEquals
and queryEqualsWithParent
, but merge queryEqualsWithParent
and queryEqualsHelper
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oops, that makes more sense. done.
types/firestore.d.ts
Outdated
* containing the number of failed deletes and the stack trace of the last | ||
* failed delete. The provided reference is deleted regardless of whether | ||
* all deletes succeeded, except when Firestore fails to fetch the provided | ||
* reference's descendants. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please update according to the suggestions above.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done.
types/firestore.d.ts
Outdated
* The promise is rejected if any of the deletes fail. | ||
*/ | ||
recursiveDelete( | ||
ref: CollectionReference | DocumentReference, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add <unknown>
, otherwise this typing won't allow converters.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the detailed review! Addressed comments for the logic, going to work on test comments next.
ref: CollectionReference<T> | DocumentReference<T>, | ||
bulkWriter?: BulkWriter | ||
): Promise<void> { | ||
const writer = bulkWriter ?? this.getBulkWriter(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
writer
has type BulkWriter
, whereas bulkWriter
always has type BulkWriter | undefined
, which makes using the variable later messier.
dev/src/index.ts
Outdated
): Promise<void> { | ||
const writer = bulkWriter ?? this.getBulkWriter(); | ||
writer._verifyNotClosed(); | ||
const docStream = this.getAllDescendants(ref); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, I'm not really sure where to move it. Are you suggesting we encapsulate all the behavior in a separate class?
dev/src/index.ts
Outdated
err.message = | ||
'Failed to fetch children documents. ' + | ||
'The provided reference was not deleted.'; | ||
incrementErrorCount(err); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. This made me realize that the # deletes failed
message won't be accurate if the stream fails since all the remaining docs not returned by the stream are unaccounted for. Should I special case the error message in the stream failure case to capture this fact?
dev/src/index.ts
Outdated
* The promise is rejected if any of the deletes fail. | ||
*/ | ||
recursiveDelete<T = DocumentData>( | ||
ref: CollectionReference<T> | DocumentReference<T>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had to change the signature to firestore.DocumentRef
and add some casting logic from the firestore typed classes to our internal classes. Can you double check to see if there's a cleaner way? Thanks!
* Firestore uses a BulkWriter instance with default settings to perform the | ||
* deletes. To customize throttling rates or add success/error callbacks, | ||
* pass in a custom BulkWriter instance. | ||
* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. I included the default error handler to make it easier to paste. Do you think including it would confuse developers?
* | ||
* @private | ||
*/ | ||
private _bulkWriter: BulkWriter | undefined; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we move this to BulkWriter (kind of like a singleton pattern)? This could then be a shared instance that could be used by other operations in the future as well.
Do you mean that calling firestore.bulkWriter()
would return the same instance each time, or having a Firestore-internal BulkWriter.bulkWriter()
getter that lazy-loads?
I also wonder whether the default ramp up makes sense here.
Yeah, that's something I was thinking about as well to implement as part of the RST_STREAM retries. Deletes aren't subject to 555 throttling, which creates two possibilities. 1) We can add an internal override on BulkWriter to turn off throttling when recursive delete is called. However, this would mean that writes being performed at the same time won't get throttled. 2) We can add logic for BulkWriter to automatically ignore throttling limits for deletes, but then we'd have to batch deletes separately from writes (or think of another solution that allows writes to still be throttled).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One more note: Can you ping the backend chat to determine whether we can retry deletes that fail with RST_STREAM? (I see this is already part of your TODOs, so please pretend that you never saw this note).
Checked in with with backend team, and it's fine to retry deletes since they're idempotent.
types/firestore.d.ts
Outdated
* containing the number of failed deletes and the stack trace of the last | ||
* failed delete. The provided reference is deleted regardless of whether | ||
* all deletes succeeded, except when Firestore fails to fetch the provided | ||
* reference's descendants. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done.
dev/test/index.ts
Outdated
let callbackCount = 0; | ||
const bulkWriter = firestore.bulkWriter(); | ||
bulkWriter.onWriteResult(() => { | ||
callbackCount++; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done.
dev/test/index.ts
Outdated
callbackCount++; | ||
}); | ||
await firestore.recursiveDelete(firestore.collection('foo'), bulkWriter); | ||
expect(callbackCount).to.equal(3); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added one.
dev/test/index.ts
Outdated
firestore = await instantiateInstance([ | ||
['anna', 'bob', 'bob/children/charlie', 'bob/children/daniel'], | ||
]); | ||
await firestore.recursiveDelete(firestore.collection('collectionId')); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The collectionId
is implicit in instantiateInstance()
, which follows the pattern that other test helpers use. So in this case, the documents under firestore.collection('collectionId')
are ['collectionId/anna', 'collectionId/bob', ...]
.
Added a comment to clarify.
dev/test/query.ts
Outdated
queryEqualsHelper(actual, '', ...protoComponents); | ||
} | ||
|
||
function queryEqualsHelper( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A lot of tests use queryEquals()
without a parent
parameter, and I didn't want to pass in an empty string for parent
in all of them or change the spread operator to an array. Is it possible to combine these two signatures in an overload:
export function queryEquals(
actual: api.IRunQueryRequest | undefined,
parent: string,
...protoComponents: api.IStructuredQuery[]
): void;
export function queryEquals(
actual: api.IRunQueryRequest | undefined,
...protoComponents: api.IStructuredQuery[]
): void;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking better and better. Some small suggestions remain.
dev/src/index.ts
Outdated
* bulkWriter | ||
* .onWriteError((error) => { | ||
* if ( | ||
* error.code === GrpcStatus.UNAVAILABLE && |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should drop this line. Deletes are almost always retyrable, and we should show this here and do a blind retry up to MAX_RETRY_ATTEMPTS times.
Regardless of what the developer does, we should apply a sensible backoff on top of these attempts. This means that we should backoff longer for RESOURCE_EXHAUSTED.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. Added the RESOURCE_EXHAUSTED note to the todo list at the top of the PR.
dev/src/index.ts
Outdated
* ) { | ||
* return true; | ||
* } else { | ||
* console.log('Failed write at document: ', error.documentRef); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should probably add a toString()
to make this output sensible.
"That sounds like a good idea, but it seems unrelated to this PR."
I came up with an answer for you to copy & paste.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I often rely on printing the javascript object for DocumentReference when debugging to view all the different attributes of the reference, so I'm afraid to replace that with a pretty-print.
Updated to use documentRef.path
which should be sufficient. Thoughts?
dev/src/index.ts
Outdated
|
||
docStream | ||
.on('error', err => { | ||
err.code = Status.FAILED_PRECONDITION; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please take a look at https://developers.google.com/maps-booking/reference/grpc-api/status_codes
FAILED_PRECONDITION means that a simple retry won't work here. The developer has to change something about the state of the system before retrying the operation. This is likely not the case here. I would guess that we will have an error code here already when we hit this code path, but if we don't, I would suggest using UNAVAILABLE as the return code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done.
let parentPath = ref._resourcePath; | ||
if (ref instanceof CollectionReference) { | ||
parentPath = parentPath.popLast(); | ||
} | ||
const collectionId = | ||
ref instanceof CollectionReference ? ref.id : ref.parent.id; | ||
ref instanceof CollectionReference |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CollectionReference means CollectionReference<DocumentData>
. If you change this to ref instanceof CollectionReference<unknown>
you should be able to remove the as DocumentReference
cast (which should also be as DocumentReference<unknown>
).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The TS compiler was not happy with that suggestion:
TS2348: Value of type 'typeof CollectionReference' is not callable. Did you mean to include 'new'
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the TS compiler is right, since the generics support of TypeScript is pretty much like Java and "erased at compile time". I was able to get the code to transpiler even without the ref as DocumentReference
though..
FWIW, an as DocumentReference<unknown>
would still be the correct way to type this if we were to keep the cast. Right now, you are dropping the converter in the cast, when really we don't know anything about it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added the <unknown>
type to the DocumentReference
cast.
dev/test/query.ts
Outdated
queryEqualsHelper(actual, '', ...protoComponents); | ||
} | ||
|
||
function queryEqualsHelper( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You could do:
export function queryEquals(
actual: api.IRunQueryRequest | undefined,
parentOrProtoComponent: string|api.IStructuredQuery|undefined,
...protoComponents: api.IStructuredQuery[]
): void;
But that is not all that pretty.
I was actually thinking you keep both queryEquals
and queryEqualsWithParent
, but merge queryEqualsWithParent
and queryEqualsHelper
.
}); | ||
await firestore.recursiveDelete(randomCol, bulkWriter); | ||
expect(callbackCount).to.equal(6); | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we also get a test that has "randomCollA/docA" and "randomCollB/docB" and only deletes randomCollA? There might be some nuances in how to delete parent collection (in other ports) and I want to make sure that all of our clients handle this case correctly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good point, done.
dev/test/index.ts
Outdated
try { | ||
expect(request.writes).to.deep.equal(expected.writes); | ||
} catch (e) { | ||
batchWriteError = e; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, I think we can keep the logic as is. Individual RPC failures versus individual operations failures are not different from a user perspective. I was trying to see if we could simply bubble up the assertion failure as an individual RPC failure, which would surface these assertions in onWriteResult
or via failed promises rather than via batchWriteError
.
dev/src/bulk-writer.ts
Outdated
* @private | ||
*/ | ||
static _getInstance(firestore: Firestore): BulkWriter { | ||
if (!this.instance || this.instance.firestore !== firestore) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't think about the fact that this would break if multiple Firestore instances are used together. I am sightly inclined to ask you to undo this change since it would simplify the instance creation and its lifetime management (at the cost of further increasing the size of Firestore).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can see arguments either way, but at this point, I'm more in favor of leaving it on the Firestore instance, since trying to track the different firestore instances to pass into BulkWriter is more confusing than just having the BulkWriter instance available on the Firestore instance.
Reverted back to before.
dev/system-test/firestore.ts
Outdated
|
||
expect(await countCollectionChildren(collA)).to.equal(0); | ||
expect(await countCollectionChildren(randomCol)).to.equal(3); | ||
expect(await countCollectionChildren(collB)).to.equal(1); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is now a similar test to what you have above (something akin to on nested collection
). I think the edge case is for root collections so something like randomCol.doc('bob')
and a recursive delete on "otherCollection" (which could really be any name). We can then ensure than randomCol
still has one document.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for clarifying what you meant. Added a test for that.
let parentPath = ref._resourcePath; | ||
if (ref instanceof CollectionReference) { | ||
parentPath = parentPath.popLast(); | ||
} | ||
const collectionId = | ||
ref instanceof CollectionReference ? ref.id : ref.parent.id; | ||
ref instanceof CollectionReference |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the TS compiler is right, since the generics support of TypeScript is pretty much like Java and "erased at compile time". I was able to get the code to transpiler even without the ref as DocumentReference
though..
FWIW, an as DocumentReference<unknown>
would still be the correct way to type this if we were to keep the cast. Right now, you are dropping the converter in the cast, when really we don't know anything about it.
* return false; | ||
* } | ||
* }); | ||
* await firestore.recursiveDelete(docRef, bulkWriter); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please update this snippet as well (drop the error code, change the logging).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for catching.
* Firestore uses a BulkWriter instance with default settings to perform the | ||
* deletes. To customize throttling rates or add success/error callbacks, | ||
* pass in a custom BulkWriter instance. | ||
* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good.
…e, comment updates
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the quick feedback cycles!
* return false; | ||
* } | ||
* }); | ||
* await firestore.recursiveDelete(docRef, bulkWriter); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for catching.
let parentPath = ref._resourcePath; | ||
if (ref instanceof CollectionReference) { | ||
parentPath = parentPath.popLast(); | ||
} | ||
const collectionId = | ||
ref instanceof CollectionReference ? ref.id : ref.parent.id; | ||
ref instanceof CollectionReference |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added the <unknown>
type to the DocumentReference
cast.
dev/src/bulk-writer.ts
Outdated
* @private | ||
*/ | ||
static _getInstance(firestore: Firestore): BulkWriter { | ||
if (!this.instance || this.instance.firestore !== firestore) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can see arguments either way, but at this point, I'm more in favor of leaving it on the Firestore instance, since trying to track the different firestore instances to pass into BulkWriter is more confusing than just having the BulkWriter instance available on the Firestore instance.
Reverted back to before.
dev/system-test/firestore.ts
Outdated
|
||
expect(await countCollectionChildren(collA)).to.equal(0); | ||
expect(await countCollectionChildren(randomCol)).to.equal(3); | ||
expect(await countCollectionChildren(collB)).to.equal(1); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for clarifying what you meant. Added a test for that.
First pass at adding recursive delete. Merging to a feature branch for iterative development.
Some things that I still have yet to add: