From 773faedcd8a6048417593f27af344e3264116d98 Mon Sep 17 00:00:00 2001 From: Ian Kent Date: Fri, 6 Jun 2025 10:39:28 +0800 Subject: [PATCH] fs/file.c: add fast path in find_next_fd() JIRA: https://issues.redhat.com/browse/RHEL-38570 Upstream status: Linus commit 0c40bf47cf2d9e1413b1e62826c89c2341e66e40 Author: Yu Ma Date: Wed Jul 17 10:50:18 2024 -0400 fs/file.c: add fast path in find_next_fd() Skip 2-levels searching via find_next_zero_bit() when there is free slot in the word contains next_fd, as: (1) next_fd indicates the lower bound for the first free fd. (2) There is fast path inside of find_next_zero_bit() when size<=64 to speed up searching. (3) After fdt is expanded (the bitmap size doubled for each time of expansion), it would never be shrunk. The search size increases but there are few open fds available here. This fast path is proposed by Mateusz Guzik , and agreed by Jan Kara , which is more generic and scalable than previous versions. And on top of patch 1 and 2, it improves pts/blogbench-1.1.0 read by 8% and write by 4% on Intel ICX 160 cores configuration with v6.10-rc7. Reviewed-by: Jan Kara Reviewed-by: Tim Chen Signed-off-by: Yu Ma Link: https://lore.kernel.org/r/20240717145018.3972922-4-yu.ma@intel.com Signed-off-by: Christian Brauner Signed-off-by: Al Viro Signed-off-by: Ian Kent --- fs/file.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/fs/file.c b/fs/file.c index 8ea03d1f36d8..452cdb93a4c6 100644 --- a/fs/file.c +++ b/fs/file.c @@ -457,6 +457,15 @@ static unsigned int find_next_fd(struct fdtable *fdt, unsigned int start) unsigned int maxfd = fdt->max_fds; unsigned int maxbit = maxfd / BITS_PER_LONG; unsigned int bitbit = start / BITS_PER_LONG; + unsigned int bit; + + /* + * Try to avoid looking at the second level bitmap + */ + bit = find_next_zero_bit(&fdt->open_fds[bitbit], BITS_PER_LONG, + start & (BITS_PER_LONG - 1)); + if (bit < BITS_PER_LONG) + return bit + bitbit * BITS_PER_LONG; bitbit = find_next_zero_bit(fdt->full_fds_bits, maxbit, bitbit) * BITS_PER_LONG; if (bitbit > maxfd)