qiurui
/
Centos-kernel-stream-9
mirror of https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

io_uring: ensure local task_work is run on wait timeout 

JIRA: https://issues.redhat.com/browse/RHEL-21391

commit 6ff1407e24e6fdfa4a16ba9ba551e3d253a26391
Author: Jens Axboe <axboe@kernel.dk>
Date:   Thu Jan 4 12:21:08 2024 -0700

    io_uring: ensure local task_work is run on wait timeout
    
    A previous commit added an earlier break condition here, which is fine if
    we're using non-local task_work as it'll be run on return to userspace.
    However, if DEFER_TASKRUN is used, then we could be leaving local
    task_work that is ready to process in the ctx list until next time that
    we enter the kernel to wait for events.
    
    Move the break condition to _after_ we have run task_work.
    
    Cc: stable@vger.kernel.org
    Fixes: 846072f16eed ("io_uring: mimimise io_cqring_wait_schedule")
    Signed-off-by: Jens Axboe <axboe@kernel.dk>

Signed-off-by: Jeff Moyer <jmoyer@redhat.com>
This commit is contained in:
Jeff Moyer 2024-01-04 12:21:08 -07:00
parent 4745d2c8b7
commit 273593f53e
1 changed files with 12 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
io_uring/io_uring.c
View File

@ -2626,8 +2626,6 @@ static int io_cqring_wait(struct io_ring_ctx *ctx, int min_events,
__set_current_state(TASK_RUNNING);
atomic_set(&ctx->cq_wait_nr, 0);
if (ret < 0)
break;
/*
* Run task_work after scheduling and before io_should_wake().
* If we got woken because of task_work being processed, run it
@ -2637,6 +2635,18 @@ static int io_cqring_wait(struct io_ring_ctx *ctx, int min_events,
if (!llist_empty(&ctx->work_llist))
io_run_local_work(ctx);
/*
* Non-local task_work will be run on exit to userspace, but
* if we're using DEFER_TASKRUN, then we could have waited
* with a timeout for a number of requests. If the timeout
* hits, we could have some requests ready to process. Ensure
* this break is _after_ we have run task_work, to avoid
* deferring running potentially pending requests until the
* next time we wait for events.
*/
if (ret < 0)
break;
check_cq = READ_ONCE(ctx->check_cq);
if (unlikely(check_cq)) {
/* let the caller flush overflows, retry */