…rt discard

On Tue, 12 Oct 2021, Christoph Hellwig wrote:

> On Mon, Oct 04, 2021 at 09:01:33AM -0400, Mikulas Patocka wrote:
> > Do you want this patch?
>
> Yes, this looks like what I want.  Minor nitpicks below:
>
> > +	.fallocate_flags = BLKDEV_FALLOC_FL_SUPPORTED,
>
> I'd probably call this fallocate_supported_flags.
>
> > +	.fallocate_flags = FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE | FALLOC_FL_ZERO_RANGE,
>
> Please avoid over 80 lines for a plain list of flags.

OK. Here I'm sending a new version of the patch.

BTW. for some filesystems (cifs, ext4, fuse, ...), the supported falloc
flags vary dynamically - for example, ext4 can do COLLAPSE_RANGE and
INSERT_RANGE operations only if the filesystem is not ext2 or ext3 and if
the file is not encrypted.

Should we add a new flag FALLOC_FL_RETURN_SUPORTED_FLAGS that will return
the supported flags instead of using a static field in the file_operations
structure?

Mikulas

From: Mikulas Patocka <mpatocka@redhat.com>

The loop driver checks for the fallocate method and if it is present, it
assumes that the filesystem can do FALLOC_FL_ZERO_RANGE and
FALLOC_FL_PUNCH_HOLE requests. However, some filesystems (such as fat, or
tmpfs) have the fallocate method, but lack the capability to do
FALLOC_FL_ZERO_RANGE and/or FALLOC_FL_PUNCH_HOLE.

This results in syslog warnings "blk_update_request: operation not
supported error, dev loop0, sector 0 op 0x9:(WRITE_ZEROES) flags 0x800800
phys_seg 0 prio class 0". The error can be reproduced with this command:
"truncate -s 1GiB /tmp/file; losetup /dev/loop0 /tmp/file; blkdiscard -z
/dev/loop0"

This patch introduces a field "fallocate_supported_flags" in struct
file_operations that specifies the flags that are supported by the
fallocate methods. The loopback driver will check this field to determine
if FALLOC_FL_PUNCH_HOLE or FALLOC_FL_ZERO_RANGE is supported

Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>

* for-5.16/block:
  blk-mq: inline hot part of __blk_mq_sched_restart
  block: inline hot paths of blk_account_io_*()

Extract a fast check out of __block_mq_sched_restart() and inline it for
performance reasons.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/894abaa0998e5999f2fe18f271e5efdfc2c32bd2.1633781740.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Extract hot paths of __blk_account_io_start() and
__blk_account_io_done() into inline functions, so we don't always pay
for function calls.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/b0662a636bd4cc7b4f84c9d0a41efa46a688ef13.1633781740.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>

* for-5.16/block-io_uring: (78 commits)
  nvme-multipath: enable polled I/O
  block: don't allow writing to the poll queue attribute
  block: switch polling to be bio based
  block: define 'struct bvec_iter' as packed
  block: use SLAB_TYPESAFE_BY_RCU for the bio slab
  block: rename REQ_HIPRI to REQ_POLLED
  io_uring: don't sleep when polling for I/O
  block: replace the spin argument to blk_iopoll with a flags argument
  blk-mq: remove blk_qc_t_valid
  blk-mq: remove blk_qc_t_to_tag and blk_qc_t_is_internal
  blk-mq: factor out a "classic" poll helper
  blk-mq: factor out a blk_qc_to_hctx helper
  io_uring: fix a layering violation in io_iopoll_req_issued
  iomap: don't try to poll multi-bio I/Os in __iomap_dio_rw
  block: don't try to poll multi-bio I/Os in __blkdev_direct_IO
  direct-io: remove blk_poll support
  io_uring: inform block layer of how many requests we are submitting
  io_uring: fix io_free_batch_list races
  io_uring: remove extra io_ring_exit_work wake up
  io_uring: optimise out req->opcode reloading
  ...

* for-5.16/drivers: (41 commits)
  brd: reduce the brd_devices_mutex scope
  swim3: add missing major.h include
  sx8: fix an error code in carm_init_one()
  pf: fix error codes in pf_init_unit()
  pcd: fix error codes in pcd_init_unit()
  xtensa/platforms/iss/simdisk: add error handling support for add_disk()
  block/ataflop: add error handling support for add_disk()
  block/ataflop: provide a helper for cleanup up an atari disk
  block/ataflop: add registration bool before calling del_gendisk()
  block/ataflop: use the blk_cleanup_disk() helper
  swim: add error handling support for add_disk()
  swim: add a floppy registration bool which triggers del_gendisk()
  swim: add helper for disk cleanup
  swim: simplify using blk_cleanup_disk() on swim_remove()
  amiflop: add error handling support for add_disk()
  floppy: add error handling support for add_disk()
  floppy: fix calling platform_device_unregister() on invalid drives
  floppy: use blk_cleanup_disk()
  floppy: fix add_disk() assumption on exit due to new developments
  block/swim3: add error handling support for add_disk()
  ...

* for-5.16/block: (45 commits)
  block: merge block_ioctl into blkdev_ioctl
  block: move the *blkdev_ioctl declarations out of blkdev.h
  block: unexport blkdev_ioctl
  blk-mq: cleanup blk_mq_submit_bio
  blk-mq: cleanup and rename __blk_mq_alloc_request
  block: pre-allocate requests if plug is started and is a batch
  block: bump max plugged deferred size from 16 to 32
  block: inherit request start time from bio for BLK_CGROUP
  block: move blk-throtl fast path inline
  blk-mq: Change shared sbitmap naming to shared tags
  blk-mq: Stop using pointers for blk_mq_tags bitmap tags
  blk-mq: Use shared tags for shared sbitmap support
  blk-mq: Refactor and rename blk_mq_free_map_and_{requests->rqs}()
  blk-mq: Add blk_mq_alloc_map_and_rqs()
  blk-mq: Add blk_mq_tag_update_sched_shared_sbitmap()
  blk-mq: Don't clear driver tags own mapping
  blk-mq: Pass driver tags to blk_mq_clear_rq_mapping()
  blk-mq-sched: Rename blk_mq_sched_free_{requests -> rqs}()
  blk-mq-sched: Rename blk_mq_sched_alloc_{tags -> map_and_rqs}()
  blk-mq: Invert check in blk_mq_update_nr_requests()
  ...

* for-5.16/cdrom:
  cdrom: docs: reformat table in Documentation/userspace-api/ioctl/cdrom.rst
  drivers/cdrom: improved ioctl for media change detection

Set the poll queue flag to enable polling, given that the multipath
node just dispatches the bios to a lower queue.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Tested-by: Mark Wunderlich <mark.wunderlich@intel.com>
Link: https://lore.kernel.org/r/20211012111226.760968-17-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

The poll attribute is a historic artefact from before when we had
explicit poll queues that require driver specific configuration.
Just print a warning when writing to the attribute.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Tested-by: Mark Wunderlich <mark.wunderlich@intel.com>
Link: https://lore.kernel.org/r/20211012111226.760968-16-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Replace the blk_poll interface that requires the caller to keep a queue
and cookie from the submissions with polling based on the bio.

Polling for the bio itself leads to a few advantages:

 - the cookie construction can made entirely private in blk-mq.c
 - the caller does not need to remember the request_queue and cookie
   separately and thus sidesteps their lifetime issues
 - keeping the device and the cookie inside the bio allows to trivially
   support polling BIOs remapping by stacking drivers
 - a lot of code to propagate the cookie back up the submission path can
   be removed entirely.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Tested-by: Mark Wunderlich <mark.wunderlich@intel.com>
Link: https://lore.kernel.org/r/20211012111226.760968-15-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

'struct bvec_iter' is embedded into 'struct bio', define it as packed
so that we can get one extra 4bytes for other uses without expanding
bio.

'struct bvec_iter' is often allocated on stack, so making it packed
doesn't affect performance. Also I have run io_uring on both
nvme/null_blk, and not observe performance effect in this way.

Suggested-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Tested-by: Mark Wunderlich <mark.wunderlich@intel.com>
Link: https://lore.kernel.org/r/20211012111226.760968-14-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

This flags ensures that the pages will not be reused for non-bio
allocations before the end of an RCU grace period.  With that we can
safely use a RCU lookup for bio polling as long as we are fine with
occasionally polling the wrong device.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Tested-by: Mark Wunderlich <mark.wunderlich@intel.com>
Link: https://lore.kernel.org/r/20211012111226.760968-13-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Unlike the RWF_HIPRI userspace ABI which is intentionally kept vague,
the bio flag is specific to the polling implementation, so rename and
document it properly.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
Tested-by: Mark Wunderlich <mark.wunderlich@intel.com>
Link: https://lore.kernel.org/r/20211012111226.760968-12-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

There is no point in sleeping for the expected I/O completion timeout
in the io_uring async polling model as we never poll for a specific
I/O.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Tested-by: Mark Wunderlich <mark.wunderlich@intel.com>
Link: https://lore.kernel.org/r/20211012111226.760968-11-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Switch the boolean spin argument to blk_poll to passing a set of flags
instead.  This will allow to control polling behavior in a more fine
grained way.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Tested-by: Mark Wunderlich <mark.wunderlich@intel.com>
Link: https://lore.kernel.org/r/20211012111226.760968-10-hch@lst.de
[axboe: adapt to changed io_uring iopoll]
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Move the trivial check into the only caller.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
Tested-by: Mark Wunderlich <mark.wunderlich@intel.com>
Link: https://lore.kernel.org/r/20211012111226.760968-9-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Merge both functions into their only caller to keep the blk-mq tag to
blk_qc_t mapping as private as possible in blk-mq.c.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Tested-by: Mark Wunderlich <mark.wunderlich@intel.com>
Link: https://lore.kernel.org/r/20211012111226.760968-8-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Factor the code to do the classic full metal polling out of blk_poll into
a separate blk_mq_poll_classic helper.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Tested-by: Mark Wunderlich <mark.wunderlich@intel.com>
Link: https://lore.kernel.org/r/20211012111226.760968-7-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Add a helper to get the hctx from a request_queue and cookie, and fold
the blk_qc_t_to_queue_num helper into it as no other callers are left.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Tested-by: Mark Wunderlich <mark.wunderlich@intel.com>
Link: https://lore.kernel.org/r/20211012111226.760968-6-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

syscall-level code can't just poke into the details of the poll cookie,
which is private information of the block layer.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/r/20211012111226.760968-5-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

If an iocb is split into multiple bios we can't poll for both.  So don't
bother to even try to poll in that case.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Tested-by: Mark Wunderlich <mark.wunderlich@intel.com>
Link: https://lore.kernel.org/r/20211012111226.760968-4-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

If an iocb is split into multiple bios we can't poll for both.  So don't
even bother to try to poll in that case.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/r/20211012111226.760968-3-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

The polling support in the legacy direct-io support is a little crufty.
It already doesn't support the asynchronous polling needed for io_uring
polling, and is hard to adopt to upcoming changes in the polling
interfaces.  Given that all the major file systems already use the iomap
direct I/O code, just drop the polling support.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Tested-by: Mark Wunderlich <mark.wunderlich@intel.com>
Link: https://lore.kernel.org/r/20211012111226.760968-2-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

The block layer can use this knowledge to make smarter decisions on
how to handle the request, if it knows that N more may be coming. Switch
to using blk_start_plug_nr_ios() to pass in that information.

Signed-off-by: Jens Axboe <axboe@kernel.dk>

* for-5.16/io_uring: (61 commits)
  io_uring: fix io_free_batch_list races
  io_uring: remove extra io_ring_exit_work wake up
  io_uring: optimise out req->opcode reloading
  io_uring: reshuffle io_submit_state bits
  io_uring: safer fallback_work free
  io_uring: optimise plugging
  io_uring: correct fill events helpers types
  io_uring: inline io_poll_complete
  io_uring: inline io_req_needs_clean()
  io_uring: remove struct io_completion
  io_uring: control ->async_data with a REQ_F flag
  io_uring: optimise io_free_batch_list()
  io_uring: mark cold functions
  io_uring: optimise ctx referencing by requests
  io_uring: merge CQ and poll waitqueues
  io_uring: don't wake sqpoll in io_cqring_ev_posted
  io_uring: optimise INIT_WQ_LIST
  io_uring: optimise request allocation
  io_uring: delay req queueing into compl-batch list
  io_uring: add more likely/unlikely() annotations
  ...

* for-5.16/block: (45 commits)
  block: merge block_ioctl into blkdev_ioctl
  block: move the *blkdev_ioctl declarations out of blkdev.h
  block: unexport blkdev_ioctl
  blk-mq: cleanup blk_mq_submit_bio
  blk-mq: cleanup and rename __blk_mq_alloc_request
  block: pre-allocate requests if plug is started and is a batch
  block: bump max plugged deferred size from 16 to 32
  block: inherit request start time from bio for BLK_CGROUP
  block: move blk-throtl fast path inline
  blk-mq: Change shared sbitmap naming to shared tags
  blk-mq: Stop using pointers for blk_mq_tags bitmap tags
  blk-mq: Use shared tags for shared sbitmap support
  blk-mq: Refactor and rename blk_mq_free_map_and_{requests->rqs}()
  blk-mq: Add blk_mq_alloc_map_and_rqs()
  blk-mq: Add blk_mq_tag_update_sched_shared_sbitmap()
  blk-mq: Don't clear driver tags own mapping
  blk-mq: Pass driver tags to blk_mq_clear_rq_mapping()
  blk-mq-sched: Rename blk_mq_sched_free_{requests -> rqs}()
  blk-mq-sched: Rename blk_mq_sched_alloc_{tags -> map_and_rqs}()
  blk-mq: Invert check in blk_mq_update_nr_requests()
  ...

[  158.514382] WARNING: CPU: 5 PID: 15251 at fs/io_uring.c:1141 io_free_batch_list+0x269/0x360
[  158.514426] RIP: 0010:io_free_batch_list+0x269/0x360
[  158.514437] Call Trace:
[  158.514440]  __io_submit_flush_completions+0xde/0x180
[  158.514444]  tctx_task_work+0x14a/0x220
[  158.514447]  task_work_run+0x64/0xa0
[  158.514448]  __do_sys_io_uring_enter+0x7c/0x970
[  158.514450]  __x64_sys_io_uring_enter+0x22/0x30
[  158.514451]  do_syscall_64+0x43/0x90
[  158.514453]  entry_SYSCALL_64_after_hwframe+0x44/0xae

We should not touch request internals including req->comp_list.next
after putting our ref if it's not final, e.g. we can start freeing
requests from the free cache.

Fixed: 62ca9cb ("io_uring: optimise io_free_batch_list()")
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/b1f4df38fbb8f111f52911a02fd418d0283a4e6f.1634047298.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Simplify the ioctl path and match the code structure on the compat side.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/r/20211012104450.659013-4-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

These are only used inside of block/.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/r/20211012104450.659013-3-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

With the raw driver gone, there is no modular user left.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/r/20211012104450.659013-2-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Move the blk_mq_alloc_data stack allocation only into the branch
that actually needs it, and use rq->mq_hctx instead of data.hctx
to refer to the hctx.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/r/20211012104045.658051-3-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

The newly added loop for the cached requests in __blk_mq_alloc_request
is a little too convoluted for my taste, so unwind it a bit.  Also
rename the function to __blk_mq_alloc_requests now that it can allocate
more than a single request.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/r/20211012104045.658051-2-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

…m.rst

Add extra space to the first column of ioctl values table, and also realign
the text in the second column, to fix a documentation build warning
introduced by: commit 67f1e02 ("drivers/cdrom: improved ioctl for
media change detection")

Suggested-by: Randy Dunlap <rdunlap@infradead.org>
Fixes: 67f1e02 ("drivers/cdrom: improved ioctl for media change detection")
Signed-off-by: Phillip Potter <phil@philpotter.co.uk>
Reviewed-by: Randy Dunlap <rdunlap@infradead.org>
Link: https://lore.kernel.org/r/20211007231720.31997-1-phil@philpotter.co.uk
Signed-off-by: Jens Axboe <axboe@kernel.dk>