mm/hugetlb: fix huge_pmd_unshare() vs GUP-fast race

JIRA: https://issues.redhat.com/browse/RHEL-101247
CVE: CVE-2025-38085
Conflicts:
  * minor difference due to RHEL9 missing the port of upstream commit
    59d9094df3d7 ("mm: hugetlb: independent PMD page table shared count")

commit 1013af4f585fccc4d3e5c5824d174de2257f7d6d
Author: Jann Horn <jannh@google.com>
Date:   Tue May 27 23:23:54 2025 +0200

    mm/hugetlb: fix huge_pmd_unshare() vs GUP-fast race

    huge_pmd_unshare() drops a reference on a page table that may have
    previously been shared across processes, potentially turning it into a
    normal page table used in another process in which unrelated VMAs can
    afterwards be installed.

    If this happens in the middle of a concurrent gup_fast(), gup_fast() could
    end up walking the page tables of another process.  While I don't see any
    way in which that immediately leads to kernel memory corruption, it is
    really weird and unexpected.

    Fix it with an explicit broadcast IPI through tlb_remove_table_sync_one(),
    just like we do in khugepaged when removing page tables for a THP
    collapse.

    Link: https://lkml.kernel.org/r/20250528-hugetlb-fixes-splitrace-v2-2-1329349bad1a@google.com
    Link: https://lkml.kernel.org/r/20250527-hugetlb-fixes-splitrace-v1-2-f4136f5ec58a@google.com
    Fixes: 39dde65c99 ("[PATCH] shared page table for hugetlb page")
    Signed-off-by: Jann Horn <jannh@google.com>
    Reviewed-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
    Cc: Liam Howlett <liam.howlett@oracle.com>
    Cc: Muchun Song <muchun.song@linux.dev>
    Cc: Oscar Salvador <osalvador@suse.de>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Signed-off-by: Rafael Aquini <raquini@redhat.com>
This commit is contained in:
Rafael Aquini 2025-07-14 08:30:45 -04:00
parent 320a3c85e9
commit 12a6db384d
1 changed files with 7 additions and 0 deletions

View File

@ -7200,6 +7200,13 @@ int huge_pmd_unshare(struct mm_struct *mm, struct vm_area_struct *vma,
return 0;
pud_clear(pud);
/*
* Once our caller drops the rmap lock, some other process might be
* using this page table as a normal, non-hugetlb page table.
* Wait for pending gup_fast() in other threads to finish before letting
* that happen.
*/
tlb_remove_table_sync_one();
put_page(virt_to_page(ptep));
mm_dec_nr_pmds(mm);
return 1;