Add support for RWF_NOWAIT into related engines #972

koct9i · 2020-04-27T09:46:34Z

No description provided.

sitsofe

Minor documentation comment otherwise LGTM!

sitsofe · 2020-05-24T19:39:33Z

HOWTO

@@ -2096,6 +2096,10 @@ with the caveat that when used on the command line, they must come after the
 	When hipri is set this determines the probability of a pvsync2 I/O being high
 	priority. The default is 100%.

+.. option:: nowait : [pvsync2] [libaio] [io_uring]
+
+	Set RWF_NOWAIT on I/O, do not wait if request cannot be executed immediately.


Could you mention what the default is? Is it also worth mentioning the user needs a 4.14 or later kernel?

Agree on those suggestions. Would also probably prefer a single commit, since that single commit could then have an explanation of what it is. The commit messages seem a bit messed up right now, and are either empty (io_uring), or some implementation detail (aio), or no explanation of the feature (sync).

So either make the first one have a full explanation, with the other referencing it, or combine them into one commit instead and include the whole thing. Either approach is fine with me.

sitsofe · 2020-05-29T06:21:21Z

HOWTO

+	Also process could wait for various locks in the kernel.
+
+	With this option I/O has flag RWF_NOWAIT and ends instanyly with EAGAIN
+	instead of waiting for anything. Or returns partial result.


"Or returns partial result." - I'm not sure this is clear. Are you saying using RWF_NOWAIT can return a partial result or are you saying it will return instantly without returning a partial result?

Maybe this:

This option sets the RWF_NOWAIT flag (supported from the 4.14 Linux kernel) and the call will return instantly with EAGAIN or a partial result rather than waiting.

yes, that's better.

sitsofe · 2020-05-29T06:25:31Z

HOWTO

+
+	By default if request cannot be executed immediately it is enqueued to
+	submit later or process blocks until required resouce becomes free.
+	Also process could wait for various locks in the kernel.


Maybe:

By default if request cannot be executed immediately (e.g. resource starvation, waiting on locks) it is queued and the initiating process will be blocked until required the required resource becomes free.

sitsofe · 2020-05-29T06:30:30Z

HOWTO

+	For cached I/O this usually means request returns only cached data and
+	never issues any disk requests. Direct I/O issues request only if cache
+	invalidation isn't required, file blocks are allocated and disk request
+	could be issued immediately.


For cached I/O, using this option usually means a request will only return cached data and

For Direct I/O, requests will only succeed if cache invalidation isn't required, file blocks are fully allocated and the disk request could be issued immediately.

(This text is very informative!)

sitsofe · 2020-05-29T06:31:58Z

HOWTO

+
+	Supported since Linux 4.14
+
+	Makes sense to use together with ignore_error=EAGAIN.


It is useful to also use ignore_error=EAGAIN when using this option.

sitsofe · 2020-05-31T08:22:39Z

HOWTO

@@ -2097,6 +2097,26 @@ with the caveat that when used on the command line, they must come after the
 	When hipri is set this determines the probability of a pvsync2 I/O being high
 	priority. The default is 100%.

+.. option:: nowait : [pvsync2] [libaio] [io_uring]
+
+	By default if request cannot be executed immediately (e.g. resource starvation,


default if request -> default if a request

sitsofe · 2020-05-31T08:23:01Z

HOWTO

+
+	By default if request cannot be executed immediately (e.g. resource starvation,
+	waiting on locks) it is queued and the initiating process will be blocked until
+	required the required resource becomes free.


required the required -> the required

sitsofe · 2020-05-31T08:23:46Z

HOWTO

+	They return EOPNOTSUP instead of EAGAIN.
+
+	For cached I/O, using this option usually means a request operates only with
+	cached data. Currently RWF_NOWAIT flag does not supported for cached write.


Currently RWF_NOWAIT -> Currently the RWF_NOWAIT

sitsofe · 2020-05-31T08:24:50Z

engines/io_uring.c

+		.lname	= "No Wait",
+		.type	= FIO_OPT_BOOL,
+		.off1	= offsetof(struct ioring_options, nowait),
+		.help	= "Use RWF_NOWAIT for read/writes",


read -> reads

sitsofe · 2020-05-31T08:25:27Z

engines/libaio.c

+		.lname	= "RWF_NOWAIT",
+		.type	= FIO_OPT_BOOL,
+		.off1	= offsetof(struct libaio_options, nowait),
+		.help	= "Set RWF_NOWAIT for read/write",


read -> reads

sitsofe · 2020-05-31T08:26:41Z

engines/io_uring.c

@@ -185,6 +186,15 @@ static struct fio_option options[] = {
 		.category = FIO_OPT_C_ENGINE,
 		.group	= FIO_OPT_G_IOURING,
 	},
+	{
+		.name	= "nowait",
+		.lname	= "No Wait",


Maybe use the same lname as below: RWF_NOWAIT (or the later ones the same as this)?

sitsofe · 2020-05-31T08:28:04Z

fio.1

+Set RWF_NOWAIT on I/O, indicating to the kernel to do not wait if request
+cannot be executed immediately. (supported from the 4.14 Linux kernel)
+
+It is useful to also use ignore_error=EAGAIN when using this option.


This should just match the same text as the HOWTO (but with different formatting if necessary)

engines/libaio.c

Add bool option "nowait" into engines which could support this feature. Add test for libaio >= 0.3.111 into configure script to be compatible with prior versions where field iocb.aio_rw_flags was declared as "__pad2". By default if a request cannot be executed immediately (e.g. resource starvation, waiting on locks) it is queued and the initiating process will be blocked until the required resource becomes free. This option sets the RWF_NOWAIT flag (supported from the 4.14 Linux kernel) and the call will return instantly with EAGAIN or a partial result rather than waiting. It is useful to also use ignore_error=EAGAIN when using this option. Note: glibc 2.27, 2.28 have a bug in syscall wrappers preadv2, pwritev2. They return EOPNOTSUP instead of EAGAIN. For cached I/O, using this option usually means a request operates only with cached data. Currently the RWF_NOWAIT flag does not supported for cached write. For direct I/O, requests will only succeed if cache invalidation isn't required, file blocks are fully allocated and the disk request could be issued immediately. Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru> Link: https://lwn.net/Articles/724631/

sitsofe

LGTM! @axboe what do you think?

sitsofe reviewed May 24, 2020

View reviewed changes

koct9i force-pushed the nowait branch from a096403 to 5872e8d Compare May 25, 2020 18:05

sitsofe reviewed May 29, 2020

View reviewed changes

koct9i force-pushed the nowait branch from 5872e8d to 7ef22a9 Compare May 30, 2020 07:44

sitsofe reviewed May 31, 2020

View reviewed changes

koct9i force-pushed the nowait branch from 7ef22a9 to 7d42e66 Compare May 31, 2020 10:49

sitsofe approved these changes May 31, 2020

View reviewed changes

axboe merged commit 24f67d1 into axboe:master Jun 10, 2020

koct9i deleted the nowait branch August 23, 2024 15:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for RWF_NOWAIT into related engines #972

Add support for RWF_NOWAIT into related engines #972

koct9i commented Apr 27, 2020

sitsofe left a comment

sitsofe May 24, 2020

axboe May 24, 2020

sitsofe May 29, 2020

sitsofe May 29, 2020

koct9i May 30, 2020

sitsofe May 29, 2020

sitsofe May 29, 2020

sitsofe May 29, 2020

sitsofe May 31, 2020

sitsofe May 31, 2020

sitsofe May 31, 2020

sitsofe May 31, 2020

sitsofe May 31, 2020

sitsofe May 31, 2020

sitsofe May 31, 2020

sitsofe left a comment


		Supported since Linux 4.14

		Makes sense to use together with ignore_error=EAGAIN.

Add support for RWF_NOWAIT into related engines #972

Add support for RWF_NOWAIT into related engines #972

Conversation

koct9i commented Apr 27, 2020

sitsofe left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sitsofe left a comment

Choose a reason for hiding this comment