Loop Device Mounting Behavior

[bgardiner]

I am trying to mount SquashFS images over goofys with catfs caching (i.e., --cache "--free:1%:/cache"). I experience no issues when catfs is disabled; however, enabling catfs causes some very weird behavior. The following sequence of events consistently happens when attempting to mount a SquashFS file:

1. The first attempt to mount fails with the following error message:

$ sudo mount /goofys/public_datasets/image/imagenet-2012.squash /image-net
mount: /dev/loop0: can't read superblock

And the kernel prints these error messages in /var/log/syslog:

Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.312455] blk_update_request: 6 callbacks suppressed
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.312458] blk_update_request: I/O error, dev loop0, sector 296022272
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.316717] blk_update_request: I/O error, dev loop0, sector 296022272
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.320672] buffer_io_error: 2 callbacks suppressed
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.320675] Buffer I/O error on dev loop0, logical block 148011136, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.325583] blk_update_request: I/O error, dev loop0, sector 296022274
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.329396] Buffer I/O error on dev loop0, logical block 148011137, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.334262] blk_update_request: I/O error, dev loop0, sector 296022276
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.338123] Buffer I/O error on dev loop0, logical block 148011138, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.343457] blk_update_request: I/O error, dev loop0, sector 296022278
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.347361] Buffer I/O error on dev loop0, logical block 148011139, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.352658] blk_update_request: I/O error, dev loop0, sector 0
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.356387] SQUASHFS error: squashfs_read_data failed to read block 0x0
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.356477] blk_update_request: I/O error, dev loop0, sector 296022272
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.360511] blk_update_request: I/O error, dev loop0, sector 296022272
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.360523] Buffer I/O error on dev loop0, logical block 148011136, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.360703] blk_update_request: I/O error, dev loop0, sector 296022274
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.360704] Buffer I/O error on dev loop0, logical block 148011137, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.360877] blk_update_request: I/O error, dev loop0, sector 296022276
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.360878] Buffer I/O error on dev loop0, logical block 148011138, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.361046] Buffer I/O error on dev loop0, logical block 148011139, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.362751] Buffer I/O error on dev loop0, logical block 148011136, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.363037] Buffer I/O error on dev loop0, logical block 148011137, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.403495] squashfs: SQUASHFS error: unable to read squashfs_super_block

I have posted the goofys output (with --debug_fuse enabled) for this operation here.

All subsequent attempts to mount similarly fail.

2. If I remount goofys (without clearing the cache directory), the SquashFS mount succeeds; however, the mount command does not return until the entire SquashFS has been downloaded to disk (which can take an extremely long time for large SquashFS images). I have posted the the first part of the goofys output (with --debug_fuse enabled) for this operation here. As you can see, the debug output shows continual ReadFile requests for 131072 bytes (the default SquashFS block size) for perpetuity until the file is completely downloaded. I don't understand why the file system ops would change so drastically in this case and why the entire file has to be downloaded before file I/O can proceed -- this mitigates the usefulness of goofys for this application.

For reference, I have posted the goofys output (with --debug_fuse enabled) for a successful mount (with catfs disabled) here -- you can see the very limited file I/O that normally occurs for a SquashFS mount and the mount command returns in less than a second. All subsequent file I/O into the SquashFS proceeds satisfactorily.

Any ideas on what causes this weird behavior with catfs?

[kahing]

catfs always caches full files, if something reads byte N all bytes from 0-N will be read. Using catfs for filesystem images is probably out of scope here.