Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: split db abstraction into new crate #8594

Merged
merged 19 commits into from
Jun 4, 2024
Merged

Conversation

onbjerg
Copy link
Member

@onbjerg onbjerg commented Jun 4, 2024

Splits the db abstraction into a new crate reth-db-api.

Todo:

  • Make it compile
  • Remove TABLE associated type in Table (otherwise this PR is not possible)
  • Move some codec impls to reth-db-api (e.g. impl_uints!(u8, u16, ...))
  • Fix features related to table handle caching
  • Fix features related to database metrics
  • Fix warnings/lints

Ref #8512

@onbjerg onbjerg added C-debt A section of code is hard to understand or change A-db Related to the database labels Jun 4, 2024
@onbjerg
Copy link
Member Author

onbjerg commented Jun 4, 2024

Removed dbi caching in 37fca74 since mdbx already does this and it would complicate doing this split if we wanted to keep it around (and complicate adding support for custom tables), see https://erthink.github.io/libmdbx/group__c__dbi.html#ga9bef4a9fdf27655e9343bbbf8b6fc5a1

The old database handle is returned if the database was already open.

@onbjerg onbjerg marked this pull request as ready for review June 4, 2024 18:11
crates/storage/db-api/src/lib.rs Show resolved Hide resolved
@@ -1,5 +1,4 @@
//! Storage sharded key

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why :(

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its a conversation starter

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👎

crates/storage/db/src/tables/mod.rs Outdated Show resolved Hide resolved
@onbjerg onbjerg requested a review from DaniPopes June 4, 2024 18:58
Comment on lines -139 to +133
impl $crate::table::Table for $name {
const TABLE: Tables = Tables::$name;
impl reth_db_api::table::Table for $name {
const NAME: &'static str = table_names::$name;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this also address @prestwich's limitation of not being able to define custom tables in ExEx?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not entirely, most of the code still assumes a fixed set of tables by using the generated Tables enum

Copy link
Collaborator

@prestwich prestwich Jun 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LOC was one of the primary issues and is now resolved, but i haven't identified others yet

https://github.com/paradigmxyz/reth/pull/8594/files#diff-ee4d680a970f12705c9230a199a58e271d1f250dba479dea525f10c53294f1dfR80

@onbjerg onbjerg enabled auto-merge June 4, 2024 21:15
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one suggestion when doing these kinds of refactors, I try to avoid most dep changes on the initial PR with re-exports and then update them once the pr is merged because doing it like this can result of a lot of conflicts and makes the initial pr harder to review

crates/storage/db/src/lib.rs Outdated Show resolved Hide resolved
@onbjerg onbjerg disabled auto-merge June 4, 2024 21:21
@onbjerg onbjerg enabled auto-merge June 4, 2024 21:33
@onbjerg onbjerg added this pull request to the merge queue Jun 4, 2024
Merged via the queue into main with commit 51a28f2 Jun 4, 2024
30 checks passed
@onbjerg onbjerg deleted the onbjerg/reth-db-api branch June 4, 2024 21:57
mw2000 pushed a commit to mw2000/reth that referenced this pull request Jun 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-db Related to the database C-debt A section of code is hard to understand or change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants