Go to file
Jerome Marchand 2463e2b3c5 tracing: Ensure visibility when inserting an element into tracing_map
JIRA: https://issues.redhat.com/browse/RHEL-30459
CVE: CVE-2024-26645

commit aef1cb00856ccfd614467cfb50b791278992e177
Author: Petr Pavlu <petr.pavlu@suse.com>
Date:   Mon Jan 22 16:09:28 2024 +0100

    tracing: Ensure visibility when inserting an element into tracing_map

    [ Upstream commit 2b44760609e9eaafc9d234a6883d042fc21132a7 ]

    Running the following two commands in parallel on a multi-processor
    AArch64 machine can sporadically produce an unexpected warning about
    duplicate histogram entries:

     $ while true; do
         echo hist:key=id.syscall:val=hitcount > \
           /sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/trigger
         cat /sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/hist
         sleep 0.001
       done
     $ stress-ng --sysbadaddr $(nproc)

    The warning looks as follows:

    [ 2911.172474] ------------[ cut here ]------------
    [ 2911.173111] Duplicates detected: 1
    [ 2911.173574] WARNING: CPU: 2 PID: 12247 at kernel/trace/tracing_map.c:983 tracing_map_sort_entries+0x3e0/0x408
    [ 2911.174702] Modules linked in: iscsi_ibft(E) iscsi_boot_sysfs(E) rfkill(E) af_packet(E) nls_iso8859_1(E) nls_cp437(E) vfat(E) fat(E) ena(E) tiny_power_button(E) qemu_fw_cfg(E) button(E) fuse(E) efi_pstore(E) ip_tables(E) x_tables(E) xfs(E) libcrc32c(E) aes_ce_blk(E) aes_ce_cipher(E) crct10dif_ce(E) polyval_ce(E) polyval_generic(E) ghash_ce(E) gf128mul(E) sm4_ce_gcm(E) sm4_ce_ccm(E) sm4_ce(E) sm4_ce_cipher(E) sm4(E) sm3_ce(E) sm3(E) sha3_ce(E) sha512_ce(E) sha512_arm64(E) sha2_ce(E) sha256_arm64(E) nvme(E) sha1_ce(E) nvme_core(E) nvme_auth(E) t10_pi(E) sg(E) scsi_mod(E) scsi_common(E) efivarfs(E)
    [ 2911.174738] Unloaded tainted modules: cppc_cpufreq(E):1
    [ 2911.180985] CPU: 2 PID: 12247 Comm: cat Kdump: loaded Tainted: G            E      6.7.0-default #2 1b58bbb22c97e4399dc09f92d309344f69c44a01
    [ 2911.182398] Hardware name: Amazon EC2 c7g.8xlarge/, BIOS 1.0 11/1/2018
    [ 2911.183208] pstate: 61400005 (nZCv daif +PAN -UAO -TCO +DIT -SSBS BTYPE=--)
    [ 2911.184038] pc : tracing_map_sort_entries+0x3e0/0x408
    [ 2911.184667] lr : tracing_map_sort_entries+0x3e0/0x408
    [ 2911.185310] sp : ffff8000a1513900
    [ 2911.185750] x29: ffff8000a1513900 x28: ffff0003f272fe80 x27: 0000000000000001
    [ 2911.186600] x26: ffff0003f272fe80 x25: 0000000000000030 x24: 0000000000000008
    [ 2911.187458] x23: ffff0003c5788000 x22: ffff0003c16710c8 x21: ffff80008017f180
    [ 2911.188310] x20: ffff80008017f000 x19: ffff80008017f180 x18: ffffffffffffffff
    [ 2911.189160] x17: 0000000000000000 x16: 0000000000000000 x15: ffff8000a15134b8
    [ 2911.190015] x14: 0000000000000000 x13: 205d373432323154 x12: 5b5d313131333731
    [ 2911.190844] x11: 00000000fffeffff x10: 00000000fffeffff x9 : ffffd1b78274a13c
    [ 2911.191716] x8 : 000000000017ffe8 x7 : c0000000fffeffff x6 : 000000000057ffa8
    [ 2911.192554] x5 : ffff0012f6c24ec0 x4 : 0000000000000000 x3 : ffff2e5b72b5d000
    [ 2911.193404] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff0003ff254480
    [ 2911.194259] Call trace:
    [ 2911.194626]  tracing_map_sort_entries+0x3e0/0x408
    [ 2911.195220]  hist_show+0x124/0x800
    [ 2911.195692]  seq_read_iter+0x1d4/0x4e8
    [ 2911.196193]  seq_read+0xe8/0x138
    [ 2911.196638]  vfs_read+0xc8/0x300
    [ 2911.197078]  ksys_read+0x70/0x108
    [ 2911.197534]  __arm64_sys_read+0x24/0x38
    [ 2911.198046]  invoke_syscall+0x78/0x108
    [ 2911.198553]  el0_svc_common.constprop.0+0xd0/0xf8
    [ 2911.199157]  do_el0_svc+0x28/0x40
    [ 2911.199613]  el0_svc+0x40/0x178
    [ 2911.200048]  el0t_64_sync_handler+0x13c/0x158
    [ 2911.200621]  el0t_64_sync+0x1a8/0x1b0
    [ 2911.201115] ---[ end trace 0000000000000000 ]---

    The problem appears to be caused by CPU reordering of writes issued from
    __tracing_map_insert().

    The check for the presence of an element with a given key in this
    function is:

     val = READ_ONCE(entry->val);
     if (val && keys_match(key, val->key, map->key_size)) ...

    The write of a new entry is:

     elt = get_free_elt(map);
     memcpy(elt->key, key, map->key_size);
     entry->val = elt;

    The "memcpy(elt->key, key, map->key_size);" and "entry->val = elt;"
    stores may become visible in the reversed order on another CPU. This
    second CPU might then incorrectly determine that a new key doesn't match
    an already present val->key and subsequently insert a new element,
    resulting in a duplicate.

    Fix the problem by adding a write barrier between
    "memcpy(elt->key, key, map->key_size);" and "entry->val = elt;", and for
    good measure, also use WRITE_ONCE(entry->val, elt) for publishing the
    element. The sequence pairs with the mentioned "READ_ONCE(entry->val);"
    and the "val->key" check which has an address dependency.

    The barrier is placed on a path executed when adding an element for
    a new key. Subsequent updates targeting the same key remain unaffected.

    From the user's perspective, the issue was introduced by commit
    c193707dde ("tracing: Remove code which merges duplicates"), which
    followed commit cbf4100efb ("tracing: Add support to detect and avoid
    duplicates"). The previous code operated differently; it inherently
    expected potential races which result in duplicates but merged them
    later when they occurred.

    Link: https://lore.kernel.org/linux-trace-kernel/20240122150928.27725-1-petr.pavlu@suse.com

    Fixes: c193707dde ("tracing: Remove code which merges duplicates")
    Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
    Acked-by: Tom Zanussi <tom.zanussi@linux.intel.com>
    Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
    Signed-off-by: Sasha Levin <sashal@kernel.org>

Signed-off-by: Jerome Marchand <jmarchan@redhat.com>
2024-07-08 11:27:06 +02:00
Documentation Merge: locking/atomic: Make test_and_*_bit() ordered on failure 2024-07-03 13:13:50 +00:00
LICENSES
arch Merge: Rebase mmc driver to v6.6 2024-07-03 13:12:02 +00:00
block Merge: block: fix deadlock between bd_link_disk_holder and partition scan 2024-07-01 12:44:49 +00:00
certs
crypto crypto: pcrypt - Fix hungtask for PADATA_RESET 2024-05-29 13:20:49 +08:00
drivers Merge: bonding: fix incorrect software timestamping report 2024-07-03 13:14:17 +00:00
fs Merge: Update afs and rxrpc 2024-07-02 13:29:48 +00:00
include Merge: locking/atomic: Make test_and_*_bit() ordered on failure 2024-07-03 13:13:50 +00:00
init modules: wait do_free_init correctly 2024-06-17 14:17:30 -04:00
io_uring Merge: io_uring: drop any code related to SCM_RIGHTS 2024-07-02 13:27:03 +00:00
ipc Merge: ipc/msg: mitigate the lock contention in ipc/msg 2024-05-16 13:29:40 +00:00
kernel tracing: Ensure visibility when inserting an element into tracing_map 2024-07-08 11:27:06 +02:00
lib Merge: CNB95: convert tunnel metadata flags 2024-07-01 12:48:47 +00:00
mm Merge: CVE-2024-36028: mm/hugetlb: fix DEBUG_LOCKS_WARN_ON(1) when dissolve_free_hugetlb_folio() 2024-07-03 13:11:05 +00:00
net Merge: CVE-2024-36978: net: sched: sch_multiq: fix possible OOB write in multiq_tune() 2024-07-03 13:13:17 +00:00
redhat Merge: redhat/configs: increase CONFIG_DEFAULT_MMAP_MIN_ADDR from 32K to 64K for aarch64 2024-07-03 13:14:02 +00:00
samples samples/bpf: Use %lu format specifier for unsigned long values 2024-06-25 11:07:33 +02:00
scripts Merge: bpf 6.8 rebase 2024-07-01 12:44:09 +00:00
security Revert BPF token-related functionality 2024-06-25 11:07:29 +02:00
sound Merge DRM changes from upstream v6.7..v6.8 2024-06-11 06:44:18 +10:00
tools Merge: CNB95: netdev-genl: Introduce queue and NAPI support in netdev-genl 2024-07-02 13:28:26 +00:00
usr kexec.h: add linux/kexec.h to UAPI compile-test coverage 2024-05-15 13:58:51 +08:00
virt KVM: Drop unused @may_block param from gfn_to_pfn_cache_invalidate_start() 2024-05-13 23:58:51 -04:00
.clang-format printk: Prepare for SRCU console list protection 2024-05-09 11:25:16 -04:00
.cocciconfig
.get_maintainer.ignore
.gitattributes
.gitignore redhat: rhdocs: delete .get_maintainer.conf 2024-06-06 09:36:57 -04:00
.gitlab-ci.yml
.mailmap
COPYING
CREDITS MAINTAINERS: Drop Gustavo Pimentel as PCI DWC Maintainer 2024-05-13 15:56:48 -06:00
Kbuild
Kconfig
Kconfig.redhat
MAINTAINERS Merge: Rebase mmc driver to v6.6 2024-07-03 13:12:02 +00:00
Makefile Merge: redhat: include resolve_btfids in kernel-devel 2024-06-19 18:28:12 +00:00
Makefile.rhelver [redhat] kernel-5.14.0-474.el9 2024-07-02 13:32:32 +00:00
README
makefile

README

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.