Skip to content
This repository has been archived by the owner. It is now read-only.

swapon fails with fallocate on xfs #5

Closed
dane-xa opened this issue Apr 4, 2013 · 24 comments

Comments

@dane-xa
Copy link

commented Apr 4, 2013

i'm using xfs on Ubuntu 10.04, 2.6.32-344-ec2. When the swapfile is created with fallocate, swapon fails and logs "swapfile has holes". Using dd works fine.

@sethvargo

This comment has been minimized.

Copy link
Contributor

commented Apr 4, 2013

What happens when you run fallocate on it's own? https://github.com/sethvargo-cookbooks/swap/blob/master/libraries/swapfile.rb#L16

Is fallocate supported on 10.04? It could be an ec2-specific thing too. Any more debugging output?

@dane-xa

This comment has been minimized.

Copy link
Author

commented Apr 6, 2013

On Thu, Apr 4, 2013 at 3:24 PM, Seth Vargo notifications@github.com wrote:

What happens when you run fallocate on it's own?
Is fallocate supported on 10.04? It could be an ec2-specific thing too.
Any more debugging output?

Assuming you've seen fallocate work properly on xfs while creating
swap space, my guess is that not all versions of xfs are equal.

Here's all the debugging output I can give:

size=$((1048576 * 100))

fallocate -l $size /mnt/testfallocate

chmod 0600 /mnt/testfallocate

mkswap /mnt/testfallocate

mkswap: /mnt/testfallocate: warning: don't erase bootbits sectors
on whole disk. Use -f to force.
Setting up swapspace version 1, size = 102396 KiB
no label, UUID=671ac461-e242-4564-8694-dad67c29b8d8

swapon -v /mnt/testfallocate

swapon on /mnt/testfallocate
swapon: /mnt/testfallocate: found swap signature: version 1, page-size
4, same byte order
swapon: /mnt/testfallocate: pagesize=4096, swapsize=104857600, devsize=104857600
swapon: /mnt/testfallocate: swapon failed: Invalid argument

tail /var/log/kern.log

Apr 5 17:46:08 prod1-db4b kernel: [4246771.122510] swapon: swapfile has holes

Dane

@sethvargo

This comment has been minimized.

Copy link
Contributor

commented Apr 7, 2013

@andrewgross can you take a look at this?

@sethvargo

This comment has been minimized.

Copy link
Contributor

commented Apr 7, 2013

@dane-xa, @andrewgross was the one who wrote the fallocate stuff, so I'll defer to him

@andrewgross

This comment has been minimized.

Copy link
Contributor

commented Apr 7, 2013

Checking it out now. My tests were done with Ubuntu 12.04 on EC2, I'll see if I can replicate this.

@sethvargo

This comment has been minimized.

Copy link
Contributor

commented Apr 7, 2013

Maybe we should rescue the fallocate call and use dd

This message was sent from my mobile device.

I apologize in advance for any typographical errors or autocorrections.

On Apr 7, 2013, at 1:51 PM, Andrew Gross notifications@github.com wrote:

Checking it out now. My tests were done with Ubuntu 12.04 on EC2, I'll see if I can replicate this.


Reply to this email directly or view it on GitHub.

@andrewgross

This comment has been minimized.

Copy link
Contributor

commented Apr 7, 2013

Not a bad plan, it could simplify a bunch of the "when to use fallocate" logic

@andrewgross

This comment has been minimized.

Copy link
Contributor

commented Apr 7, 2013

Also, I am able to reproduce this on 10.04, still investigating to see if there is a solution.

@andrewgross

This comment has been minimized.

Copy link
Contributor

commented Apr 7, 2013

Looks like an issue with older versions of XFS.

Ubuntu 12.04

  • ext4 - works
  • xfs - works

Ubuntu 10.04

  • ext4 - works
  • xfs - fails

In 10.04 the version of XFS Progs is 3.1.0ubuntu1
In 12.04 the version of XFS Progs is 3.1.7

I will see if I can track down anything from bug reports, but my guess is that it will be an issue that cannot be "fixed" without upgrading or degrading to DD.

The issue is coming from fallocate creating files that could behave as sparse files. When fallocate creates files it reserves the space. This is what gives it speed improvements over DD, since it is not actually writing the whole file, but marking the extents.

From the fallocate man page

fallocate is used to preallocate blocks to a file. For filesystems which support the fallocate system call, this is done quickly by allocating blocks and marking them as uninitialized, requiring no IO to the data blocks. This is much faster than creating a file by filling it with zeros.

It normally does not cause an issue with swapfiles since the filesystem is supposed to recognize this form of sparse file creation and emulate behavior that would make it seem like it is not a sparse file, however it seems that the older version of XFS is not behaving properly when called on to make swapfiles.

Check out some discussion on fallocate changes here. It mentions a few differences between how ext4 and xfs behave at the time with fallocate and swapfiles. Granted, this is from 2007, so there is no gaurantee this is accurate for the state of fallocate and xfs in Ubuntu 10.04

@andrewgross

This comment has been minimized.

Copy link
Contributor

commented Apr 8, 2013

@sethvargo any thoughts on a good way to wrap the logic to retry with dd? It looks like you should be able to catch the error from fallocate when running swapon and just restart the chain, starting with creating a new file. Unsure if the notifies/subscribes structure will behave without some testing.

@sethvargo

This comment has been minimized.

Copy link
Contributor

commented Apr 8, 2013

Well, we'd need to have some type of cleanup too. It wouldn't just be "retry" with dd. There's also no easy way to begin-rescue in Chef. Ideally we could use something like

bash 'create_swap_file' do
  code 'fallocate || dd'
end

But the fallocate will succeed, just fail to mount.

@andrewgross

This comment has been minimized.

Copy link
Contributor

commented Apr 8, 2013

It could be brought purely in to ruby code and manage the system calls with shell_out!

On the plus side we should not need to do any 'rolling back'. If we fail at swapon the first time, we can just restart with dd, none of the other commands will fail upon retry.

@sethvargo

This comment has been minimized.

Copy link
Contributor

commented Apr 8, 2013

Okay. Do you want to throw a PR together?

@andrewgross

This comment has been minimized.

Copy link
Contributor

commented Apr 8, 2013

Sure let me see if I can get a PoC working.

@andrewgross

This comment has been minimized.

Copy link
Contributor

commented Apr 9, 2013

I have a PoC working at https://github.com/andrewgross/swap/tree/feature/degrade-to-dd

I will need to polish it a little more to make sure it fits with HWRP standards.

@sethvargo

This comment has been minimized.

Copy link
Contributor

commented Apr 9, 2013

HWRP?

@andrewgross

This comment has been minimized.

Copy link
Contributor

commented Apr 9, 2013

Heavy Weight Resource Provider. Essentially just pure ruby not using the LWRP syntax.

@sethvargo

This comment has been minimized.

Copy link
Contributor

commented Apr 9, 2013

Oh dur. NVM. Heavy Weight Resource Provider.

Sorry, I was doing some stuff with Heavy Water cookbooks and all I could think was "Heavy Water ..."

@sethvargo

This comment has been minimized.

Copy link
Contributor

commented Apr 9, 2013

Okay, I left some comments inline, but overall it looks really code. When you PR, I can do a full code review on the diff.

@andrewgross

This comment has been minimized.

Copy link
Contributor

commented Apr 9, 2013

Ok, fixing the issues and will submit a PR soon.

@sethvargo

This comment has been minimized.

Copy link
Contributor

commented Apr 13, 2013

Fixed in #6

@sethvargo sethvargo closed this Apr 13, 2013

@diranged

This comment has been minimized.

Copy link

commented Nov 15, 2014

This issue seems to be happening again -- at least to an Ubuntu 12 and Ubuntu 14 system running a recent version of the xfsprogs (3.1.8). I'm going to, for now, fork the repo and disable the fallocate code entirely for our own purposes... but it seems like other people are seeing this too:

https://bugzilla.redhat.com/show_bug.cgi?id=1129205

@socketpair

This comment has been minimized.

Copy link

commented Apr 6, 2017

Ubuntu 10.10. The same.

@lock

This comment has been minimized.

Copy link

commented Jul 24, 2018

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked as resolved and limited conversation to collaborators Jul 24, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
5 participants
You can’t perform that action at this time.