Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creating a swapfile using dd is extremely slow, please use fallocate on Linux #127

Closed
Artoria2e5 opened this issue Nov 23, 2015 · 2 comments

Comments

@Artoria2e5
Copy link
Contributor

Artoria2e5 commented Nov 23, 2015

@butangmucat told me today that his Azure instance takes 10 mins to generate a swapfile on each boot. Most sane Linux admins, however, people prefer to use fallocate(1) to do the work of creating a non-sparse file, especially on IO-speed-challanged conditions like Azure. This also reduces disk wear (trivial to say, isn't it?).

The biggest difference of using dd and fallocate is that the former creates a file with some given contents, while the latter only allocates some space for a file. Since the space is allocated (therefore the file is not sparse), it's still effective for swaps.

https://github.com/Azure/WALinuxAgent/blob/2.0/waagent#L545 exhibits this behavior:

swap = Config.get("ResourceDisk.EnableSwap")
if swap == None or swap.lower().startswith("n"):
    return
sizeKB = int(Config.get("ResourceDisk.SwapSizeMB")) * 1024
if os.path.isfile(mountpoint + "/swapfile") and os.path.getsize(mountpoint + "/swapfile") != (sizeKB * 1024):
    os.remove(mountpoint + "/swapfile")
if not os.path.isfile(mountpoint + "/swapfile"):
    Run("dd if=/dev/zero of=" + mountpoint + "/swapfile bs=1024 count=" + str(sizeKB))
    Run("mkswap " + mountpoint + "/swapfile")
if not Run("swapon " + mountpoint + "/swapfile"):
    Log("Enabled " + str(sizeKB) + " KB of swap at " + mountpoint + "/swapfile")
else:
    Error("ActivateResourceDisk: Failed to activate swap at " + mountpoint + "/swapfile")

See also https://stackoverflow.com/questions/257844/quickly-create-a-large-file-on-a-linux-system.

Resolves #127.

@Artoria2e5
Copy link
Contributor Author

Oh……and you guys are using a super small block size. Why don't you scale it to units of at least 4M and fill up the nasty remainders using the old size?

Artoria2e5 added a commit to AOSC-Archive/WALinuxAgent that referenced this issue Nov 23, 2015
Artoria2e5 added a commit to AOSC-Archive/WALinuxAgent that referenced this issue Feb 25, 2016
This saves time and disk life. Fix Azure#127.

For BSD, a hopefully faster procedure with a bigger blocksize is used.
Artoria2e5 added a commit to AOSC-Archive/WALinuxAgent that referenced this issue Feb 25, 2016
This saves time and disk life. Fix Azure#127.

For BSD, a hopefully faster procedure with a bigger blocksize is used.
hglkrijger added a commit to hglkrijger/WALinuxAgent that referenced this issue Jun 1, 2016
hglkrijger added a commit to hglkrijger/WALinuxAgent that referenced this issue Jun 1, 2016
…lling back to dd; adds unit tests for basic functionality and shellutil.quote
hglkrijger added a commit to hglkrijger/WALinuxAgent that referenced this issue Jun 6, 2016
…lling back to dd; adds unit tests for basic functionality and shellutil.quote
hglkrijger added a commit to hglkrijger/WALinuxAgent that referenced this issue Jun 6, 2016
…lling back to dd; adds unit tests for basic functionality and shellutil.quote
hglkrijger added a commit to hglkrijger/WALinuxAgent that referenced this issue Jun 6, 2016
…lling back to dd; adds unit tests for basic functionality and shellutil.quote
hglkrijger added a commit to hglkrijger/WALinuxAgent that referenced this issue Jun 6, 2016
…lling back to dd; adds unit tests for basic functionality and shellutil.quote
Artoria2e5 added a commit to AOSC-Archive/WALinuxAgent that referenced this issue Jun 6, 2016
…lling back to dd; adds unit tests for basic functionality and shellutil.quote
Artoria2e5 added a commit to AOSC-Archive/WALinuxAgent that referenced this issue Jun 6, 2016
…lling back to dd; adds unit tests for basic functionality and shellutil.quote
hglkrijger pushed a commit that referenced this issue Jun 7, 2016
… back to dd; adds unit tests for basic functionality and shellutil.quote (#215)
@hglkrijger
Copy link
Member

fix merged, closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants