mm/migrate: fix wrongly apply write bit after mkdirty on sparc64

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2168392

This patch is a backport of the following upstream commit:
commit 96a9c28
Author: Peter Xu <peterx@redhat.com>
Date:   Thu Feb 16 10:30:59 2023 -0500

    mm/migrate: fix wrongly apply write bit after mkdirty on sparc64

    Nick Bowler reported another sparc64 breakage after the young/dirty
    persistent work for page migration (per "Link:" below).  That's after a
    similar report [2].

    It turns out page migration was overlooked, and it wasn't failing before
    because page migration was not enabled in the initial report test
    environment.

    David proposed another way [2] to fix this from sparc64 side, but that
    patch didn't land somehow.  Neither did I check whether there's any other
    arch that has similar issues.

    Let's fix it for now as simple as moving the write bit handling to be
    after dirty, like what we did before.

    Note: this is based on mm-unstable, because the breakage was since 6.1 and
    we're at a very late stage of 6.2 (-rc8), so I assume for this specific
    case we should target this at 6.3.

    [1] https://lore.kernel.org/all/20221021160603.GA23307@u164.east.ru/
    [2] https://lore.kernel.org/all/20221212130213.136267-1-david@redhat.com/

    Link: https://lkml.kernel.org/r/20230216153059.256739-1-peterx@redhat.com
    Fixes: 2e34687 ("mm: remember young/dirty bit for page migrations")
    Link: https://lore.kernel.org/all/CADyTPExpEqaJiMGoV+Z6xVgL50ZoMJg49B10LcZ=8eg19u34BA@mail.gmail.com/
    Signed-off-by: Peter Xu <peterx@redhat.com>
    Reported-by: Nick Bowler <nbowler@draconx.ca>
    Acked-by: David Hildenbrand <david@redhat.com>
    Tested-by: Nick Bowler <nbowler@draconx.ca>
    Cc: <regressions@lists.linux.dev>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Signed-off-by: Rafael Aquini <aquini@redhat.com>

Author: aquini

mm/huge_memory.c

@@ -3292,15 +3292,17 @@ void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new)
 	pmde = mk_huge_pmd(new, READ_ONCE(vma->vm_page_prot));
 	if (pmd_swp_soft_dirty(*pvmw->pmd))
 		pmde = pmd_mksoft_dirty(pmde);
-	if (is_writable_migration_entry(entry))
-		pmde = maybe_pmd_mkwrite(pmde, vma);
 	if (pmd_swp_uffd_wp(*pvmw->pmd))
 		pmde = pmd_wrprotect(pmd_mkuffd_wp(pmde));
 	if (!is_migration_entry_young(entry))
 		pmde = pmd_mkold(pmde);
 	/* NOTE: this may contain setting soft-dirty on some archs */
 	if (PageDirty(new) && is_migration_entry_dirty(entry))
 		pmde = pmd_mkdirty(pmde);
+	if (is_writable_migration_entry(entry))
+		pmde = maybe_pmd_mkwrite(pmde, vma);
+	else
+		pmde = pmd_wrprotect(pmde);
 
 	if (PageAnon(new)) {
 		rmap_t rmap_flags = RMAP_COMPOUND;

mm/migrate.c

@@ -214,6 +214,8 @@ static bool remove_migration_pte(struct folio *folio,
 			pte = maybe_mkwrite(pte, vma);
 		else if (pte_swp_uffd_wp(*pvmw.pte))
 			pte = pte_mkuffd_wp(pte);
+		else
+			pte = pte_wrprotect(pte);
 
 		if (folio_test_anon(folio) && !is_readable_migration_entry(entry))
 			rmap_flags |= RMAP_EXCLUSIVE;