Skip to content

Commit

Permalink
add additional docs to etl Collector
Browse files Browse the repository at this point in the history
  • Loading branch information
joshieDo committed Feb 29, 2024
1 parent 83b5eae commit 0a88a3a
Showing 1 changed file with 5 additions and 2 deletions.
7 changes: 5 additions & 2 deletions crates/etl/src/lib.rs
Expand Up @@ -30,9 +30,12 @@ use tempfile::{NamedTempFile, TempDir};
/// An ETL (extract, transform, load) data collector.
///
/// Data is pushed (extract) to the collector which internally flushes the data in a sorted
/// (transform) manner to files of some specified capacity.
/// (transform) manner to files of some specified capacity. the data can later be iterated over
/// (load) in a sorted manner.
///
/// The data can later be iterated over (load) in a sorted manner.
/// Used mainly to insert data into `MDBX` in a sorted manner. This is important because performance
/// and storage space degrades greatly if the data is inserted unsorted (eg. tables with hashes as keys.) as
/// opposed to append & sorted insert. Some benchmarks can be found [here](https://github.com/paradigmxyz/reth/pull/1130#issuecomment-1418642755).
#[derive(Debug)]
pub struct Collector<K, V>
where
Expand Down

0 comments on commit 0a88a3a

Please sign in to comment.