Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove files sub #344

Merged
merged 25 commits into from
Jun 29, 2023
Merged

Remove files sub #344

merged 25 commits into from
Jun 29, 2023

Conversation

LuKks
Copy link
Contributor

@LuKks LuKks commented Jun 16, 2023

Rebased from external Hans's branch to debug a bit

@LuKks LuKks marked this pull request as draft June 16, 2023 20:46
@LuKks LuKks marked this pull request as ready for review June 16, 2023 22:25
index.js Show resolved Hide resolved
index.js Outdated Show resolved Hide resolved
@LuKks
Copy link
Contributor Author

LuKks commented Jun 16, 2023

Does this also needs like in watch eg { gt: encode(folder + '/'), lt: encode(folder + '0') }?

node = await files.peek({ gt: folder + prev, lt: folder + '0' }, ENC)

Edit: Ah nop, it uses the encoding already (it's even in the line itself)

Then the watch method should do the same I think, passing in the encoding instead of doing it itself

index.js Outdated

if (recursive === false) return shallowReadStream(this.files, folder, false)
// '0' is binary +1 of /
return folder ? this.entries({ gt: folder + '/', lt: folder + '0' }) : this.entries()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something to consider, we are saying "if folder then use range, otherwise default" but we are not doing that in watch for example

Can't just assume to always use a range?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are normalizing all filenames

Or watch should do the same? (explicit condition to use range)

index.js Outdated
@@ -172,7 +163,8 @@ module.exports = class Hyperdrive extends ReadyResource {
await blobsCore.ready()

this.blobs = new Hyperblobs(blobsCore)
this.db.metadata.contentFeed = this.blobs.core.key
const db = this.db.tree || this.db
db.metadata.contentFeed = this.blobs.core.key
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here about .tree, I think it's ok

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think the batch needs a getter for the metadata or a static hyperbee method that returns the db from either a batch or bee

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setter*?

I think we're already in the path of making the Batch as similar as Hyperbee so a setter for the metadata makes sense. Same about adding getHeader method to Batch

So do I add that or we just leverage the getBee helper in Hyperdrive now?

index.js Outdated Show resolved Hide resolved
index.js Outdated Show resolved Hide resolved
index.js Outdated Show resolved Hide resolved
index.js Outdated Show resolved Hide resolved
index.js Outdated Show resolved Hide resolved
index.js Outdated
await this.files.flush()
return this.close()
flush () {
return this.db.flush()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sanity checking: we no longer need to close this upon flushing a batch? Not really understanding why it's no longer needed, as Hyperdrive.batch() does return a new Hyperdrive object

Also sanity checking: it's normal that batch() returns a Hyperdrive object on the same this.corestore instance, without creating a new session? That seems wrong (definitely if we close the passed in Hyperdrive on flushing, as I think we should)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noticed that we conditionally close the corestore in Hyperdrive.close(), not closing it if we're in a checkout or batch, so calling close with the code as-is seems fine, it won't close the corestore in a batch.

Do note that this means that, when we close a Hyperdrive, any existing checkouts will become unusable (because the underlying corestore is closed), but that might be expected.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're right! The reason I didn't close it is because in main branch I didn't immediately see this.files being closed but it's actually in the close method so my bad!

Also, it should close it because MirrorDrive expects it as well:
https://github.com/holepunchto/mirror-drive/blob/cbd6dea2f175522536cb767f38fc6d23c9712677/index.js#L87
I mean, it does flush but it doesn't close, I think Mathias saw that so flush must close

@mafintosh mafintosh merged commit e773adf into main Jun 29, 2023
3 checks passed
@mafintosh mafintosh deleted the remove-sub branch June 29, 2023 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants