Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: enable direct i/o on linux target #29

Merged
merged 2 commits into from
Jun 30, 2023
Merged

feat: enable direct i/o on linux target #29

merged 2 commits into from
Jun 30, 2023

Conversation

MrCroxx
Copy link
Owner

@MrCroxx MrCroxx commented Jun 30, 2023

Signed-off-by: MrCroxx mrcroxx@outlook.com

  • enable direct i/o on linux target
  • refine flusher and reclaimed

close #26

- enable direct i/o on linux target
- refine flusher and reclaimer

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
@MrCroxx MrCroxx added the feature New feature or request label Jun 30, 2023
@MrCroxx MrCroxx self-assigned this Jun 30, 2023
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
@codecov-commenter
Copy link

Codecov Report

Merging #29 (d8bb7ad) into main (64b72b4) will decrease coverage by 0.36%.
The diff coverage is 4.54%.

@@            Coverage Diff             @@
##             main      #29      +/-   ##
==========================================
- Coverage   52.37%   52.01%   -0.36%     
==========================================
  Files          45       45              
  Lines        5253     5306      +53     
==========================================
+ Hits         2751     2760       +9     
- Misses       2502     2546      +44     
Impacted Files Coverage Δ
foyer-intrusive/src/eviction/mod.rs 0.00% <0.00%> (ø)
foyer-storage-bench/src/main.rs 0.36% <0.00%> (-0.01%) ⬇️
foyer-storage/src/error.rs 0.00% <0.00%> (ø)
foyer-storage/src/flusher.rs 0.00% <0.00%> (ø)
foyer-storage/src/lib.rs 100.00% <ø> (ø)
foyer-storage/src/reclaimer.rs 0.00% <0.00%> (ø)
foyer-storage/src/region.rs 0.00% <0.00%> (ø)
foyer-storage/src/region_manager.rs 0.00% <0.00%> (ø)
foyer-storage/src/store.rs 0.00% <0.00%> (ø)
foyer-storage/src/device/fs.rs 77.77% <100.00%> (+4.44%) ⬆️

@MrCroxx
Copy link
Owner Author

MrCroxx commented Jun 30, 2023

Benchmark (on p44 pro)

foyer-storage-bench

args:

Args {
    dir: "/data/foyer",
    capacity: 10240,
    time: 60,
    concurrency: 16,
    pools: 16,
    report_interval: 2,
    iostat_dev: "",
    w_rate: 0.0,
    r_rate: 0.0,
    entry_size: 65536,
    lookup_range: 100000,
    region_size: 64,
    buffer_pool_size: 1024,
    flushers: 4,
    reclaimers: 4,
    align: 4096,
    io_size: 16384,
}
StoreConfig {
    eviction_config: LfuConfig {
        window_to_cache_size_ratio: 1,
        tiny_lru_capacity_ratio: 0.01,
    },
    device_config: FsDeviceConfig {
        dir: "/data/foyer",
        capacity: 10737418240,
        file_capacity: 67108864,
        align: 4096,
        io_size: 16384,
    },
    admission: AdmitAll(
        PhantomData<(u64, alloc::vec::Vec<u8>)>,
    ),
    reinsertion: ReinsertNone(
        PhantomData<(u64, alloc::vec::Vec<u8>)>,
    ),
    buffer_pool_size: 1073741824,
    flushers: 4,
    reclaimers: 4,
}

result:

Total:
disk total iops: 62274.0
disk total throughput: 938.5 MiB/s
disk read iops: 14686.8
disk read throughput: 194.9 MiB/s
disk write iops: 47587.3
disk write throughput: 743.6 MiB/s
insert iops: 11191.9/s
insert throughput: 699.5 MiB/s
insert lat p50: 165us
insert lat p90: 215us
insert lat p99: 86015us
get iops: 7330.6/s
get miss: 59.32%
get hit lat p50: 623us
get hit lat p90: 875us
get hit lat p99: 116223us
get miss lat p50: 0us
get miss lat p90: 0us
get miss lat p99: 0us

@MrCroxx MrCroxx merged commit 8388de6 into main Jun 30, 2023
4 checks passed
@MrCroxx MrCroxx deleted the xx/dio branch June 30, 2023 07:40
MrCroxx added a commit that referenced this pull request Apr 17, 2024
* feat: enable direct i/o on linux target

- enable direct i/o on linux target
- refine flusher and reclaimer

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* fix unit test

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat: direct i/o
2 participants