os/filestore/FileJournal: set block size via config option #7628
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@liewegas |
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
liewegas
force-pushed
the
wip-journal-block-size
branch
from
30e8c0e
to
c8048ce
Compare
|
@wjwithagen thanks, fixed the assert placement. |
| assert((pos & (CEPH_MINIMUM_BLOCK_SIZE - 1)) == 0); | ||
| assert((bl.length() & (CEPH_DIRECTIO_ALIGNMENT - 1)) == 0); | ||
| assert((pos & (CEPH_DIRECTIO_ALIGNMENT - 1)) == 0); | ||
| assert(0 == "bl should be align"); |
There was a problem hiding this comment.
"aligned", while we're at it, and I think it's clearer to say "bl is not aligned"
Signed-off-by: Sage Weil <sage@redhat.com>
liewegas
force-pushed
the
wip-journal-block-size
branch
from
785137e
to
caa73ba
Compare
|
looks good to me. |
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Feb 19, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Feb 24, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Feb 26, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Mar 1, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Mar 2, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Mar 2, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Mar 7, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
liewegas
added a commit
that referenced
this pull request
Mar 7, 2016
os/filestore/FileJournal: set block size via config option Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Mar 16, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Mar 17, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Mar 18, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Mar 21, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Mar 21, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Mar 21, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Mar 25, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Mar 30, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Apr 4, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Apr 8, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Apr 15, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Apr 15, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Apr 20, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
wjwithagen
added a commit
to wjwithagen/ceph
that referenced
this pull request
Apr 21, 2016
We were setting the block_size as the MIN of the min size (4096) and the block size reported for the journal file/device. In reality, file systmes report all kinds of crazy block sizes. Usually it's the page size, but sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what we want. Using a size smaller than the file systems block size is not optimal for performance, but it doesn't affect how the IO happens--as long as it is larger than the hardware sector size, which is either 512 or 4096 bytes. And our min was hard-coded at 4096. So, instead, just set a config option to specify teh block size, and default that to 4096. The other uses of this constant we about *alignment* of memory buffers for the purposes of direct IO. Rename the constant but do not change the logic. That means we continue to use 4k alignment for direct io even if the device has 512 byte sectors, but that's fine--no reason to use a smaller alignment. Signed-off-by: Sage Weil <sage@redhat.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We were setting the block_size as the MIN of the min size (4096) and the
block size reported for the journal file/device. In reality, file systmes
report all kinds of crazy block sizes. Usually it's the page size, but
sometimes it is larger (e.g., 128KB for ZFS), and that's not actually what
we want. Using a size smaller than the file systems block size is not
optimal for performance, but it doesn't affect how the IO happens--as long
as it is larger than the hardware sector size, which is either 512 or
4096 bytes. And our min was hard-coded at 4096.
So, instead, just set a config option to specify teh block size, and
default that to 4096.
The other uses of this constant we about alignment of memory buffers
for the purposes of direct IO. Rename the option but do not change
the logic. That means we continue to use 4k alignment for direct io even
if the device has 512 byte sectors, but that's fine--no reason to use
a smaller alignment.
Signed-off-by: Sage Weil sage@redhat.com