Introduce a CountedFileSystem for counting file operations #9283
Conversation
Added a CountedFileSystem that tracks a number of file operations (opens, closes, deletes, renames, flushes, syncs, fsyncs, reads, writes). This class was based on the ReportFileOpEnv from db_bench. This is a stepping stone PR to be able to change the SpecialEnv into a SpecialFileSystem, where several of the file varieties wish to do operation counting.
|
@mrambacher has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
|
@mrambacher has updated the pull request. You must reimport the pull request before landing. |
|
@mrambacher has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
1 similar comment
|
@mrambacher has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
|
@mrambacher has updated the pull request. You must reimport the pull request before landing. |
|
@mrambacher has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
utilities/counted_fs.h
Outdated
| } | ||
| }; | ||
|
|
||
| struct FileCounters { |
There was a problem hiding this comment.
I'm not a big fan of this name, as it sounds like it counts things associated with each file rather than across all files. I like FileOpCounters better than FileCounters though I don't have great ideas for the other one.
There was a problem hiding this comment.
Renamed to FileOpCounters
| std::atomic<int> opens; | ||
| std::atomic<int> deletes; | ||
| std::atomic<int> renames; | ||
| std::atomic<int> flushes; |
There was a problem hiding this comment.
Probably OK if this is only ever used in unit tests, but I would expect this to overflow if ever used in production. Likely also could in benchmarking.
There was a problem hiding this comment.
These were taken from ReportFileOpCounters in db_bench_tool.cc. If they should be uint64_t, I have no problems changing them.
There was a problem hiding this comment.
uint64_t is my preference here.
|
@mrambacher has updated the pull request. You must reimport the pull request before landing. |
|
@mrambacher has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@mrambacher has updated the pull request. You must reimport the pull request before landing. |
|
@mrambacher has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@mrambacher has updated the pull request. You must reimport the pull request before landing. |
|
@mrambacher has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
- Added NewDirectory to allow counting of Fsync from Directories - Added PrintCounters to dump the counters to a string. - Added ResetCounters to clear the counters - Remove skip_fsync
|
@mrambacher has updated the pull request. You must reimport the pull request before landing. |
|
@mrambacher has updated the pull request. You must reimport the pull request before landing. |
|
@mrambacher has updated the pull request. You must reimport the pull request before landing. |
|
@mrambacher has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@mrambacher has updated the pull request. You must reimport the pull request before landing. |
|
@mrambacher has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@mrambacher has updated the pull request. You must reimport the pull request before landing. |
|
@mrambacher has updated the pull request. You must reimport the pull request before landing. |
|
@mrambacher has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@mrambacher has updated the pull request. You must reimport the pull request before landing. |
|
@mrambacher has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
1 similar comment
|
@mrambacher has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
| std::atomic<int> opens; | ||
| std::atomic<int> deletes; | ||
| std::atomic<int> renames; | ||
| std::atomic<int> flushes; |
There was a problem hiding this comment.
uint64_t is my preference here.
|
@mrambacher has updated the pull request. You must reimport the pull request before landing. |
|
@mrambacher has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Introduce a CountedFileSystem for counting file operations (facebook#9283)
Added a CountedFileSystem that tracks a number of file operations (opens, closes, deletes, renames, flushes, syncs, fsyncs, reads, writes). This class was based on the ReportFileOpEnv from db_bench.
This is a stepping stone PR to be able to change the SpecialEnv into a SpecialFileSystem, where several of the file varieties wish to do operation counting.