Skip to content

fix: Improvements for useExtracted#2152

Merged
amannn merged 113 commits intomainfrom
canary
Dec 3, 2025
Merged

fix: Improvements for useExtracted#2152
amannn merged 113 commits intomainfrom
canary

Conversation

@amannn
Copy link
Owner

@amannn amannn commented Dec 2, 2025

Fixes #2141
Fixes #2144
Fixes #2142


Note

Improves extractor transforms for useExtracted/getExtracted (alias-safe, multi-hook), debounces catalog saves, and appends trailing newlines to JSON catalogs; updates tests and snapshots.

  • Extractor (SWC plugin):
    • Map multiple hook locals and transform useExtracted/getExtracted imports to aliased useTranslations$1/getTranslations$1, avoiding collisions and supporting aliases/multiple instances.
    • Update call/await handling to use the new local names and preserve namespace detection.
  • Catalog saving:
    • Enhance SaveScheduler to coalesce pending saves, debounce via nextSaveTask, resolve/reject only the relevant pending promises, and schedule follow-up saves when needed.
  • JSON formatter:
    • Serialize with a trailing newline (.json catalogs end with \n).
  • Tests:
    • Add fixtures for multiple hooks and existing aliased hooks; update snapshots to reflect aliased imports and newline-terminated JSON outputs.

Written by Cursor Bugbot for commit 839c94e. This will update automatically on new commits. Configure here.

# Conflicts:
#	packages/next-intl/.size-limit.ts
#	packages/next-intl/__mocks__/react.tsx
#	packages/next-intl/src/navigation/shared/createSharedNavigationFns.tsx
#	packages/next-intl/src/react-server/index.test.tsx
#	packages/next-intl/src/server/react-server/RequestLocale.tsx
#	packages/next-intl/src/server/react-server/getConfig.tsx
In Next.js 15.3, [Turbopack config has become
stable](https://nextjs.org/blog/next-15-3#turbopack-configuration-in-nextconfigts-stable).
With this fix, the new option is used in order to avoid a deprecation
warning.
# Conflicts:
#	packages/next-intl/src/plugin/getNextConfig.tsx
…on APIs (#1922)

With #959, the middleware
already handled decoding of non-ASCII characters.

This allows you to define localized
[`pathnames`](https://next-intl.dev/docs/routing#pathnames) like so:

```tsx
import {defineRouting} from 'next-intl/routing';
 
export const routing = defineRouting({
  locales: ['en', 'ja'],
  defaultLocale: 'en',
  pathnames: {
    '/about': {
      'de': '/über-uns'
  }
}
```

Since Next.js automatically encodes incoming pathnames, this supports
incoming requests both for decoded pathnames (e.g. `/de/über-uns`), as
well as encoded ones (e.g. `/de/%C3%BCber-uns`).

One piece has been missing though: Pathnames returned from [navigation
APIs](https://next-intl.dev/docs/routing/navigation) should be turned
into an encoded form.

Now, `next-intl` handles this as well:

```tsx
import {Link, getPathname} from '@/i18n/navigation';

// href="/de/%C3%BCber-uns"
<Link href="/about" locale="de" />

// pathname = "/de/%C3%BCber-uns"
const pathname = getPathname({href: '/about', locale: 'de'});
```

This change brings the navigation APIs in line with [Google's
recommendation to encode non-ASCII
pathnames](https://developers.google.com/search/docs/crawling-indexing/url-structure).
cursoragent and others added 25 commits November 18, 2025 12:12
Removes tests for t.rich, t.markup, object syntax, and getExtracted as they are no longer relevant.

Co-authored-by: jan <jan@amann.work>
Move most `MessageExtractor.test.tsx` tests to Rust fixtures to separate
simple input/output transforms from integration and error-handling
tests.

---
<a
href="https://cursor.com/background-agent?bcId=bc-7b2f8500-1a72-41a6-8f3c-4795a9b7ae1d"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-cursor-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-cursor-light.svg"><img alt="Open in
Cursor"
src="https://cursor.com/open-in-cursor.svg"></picture></a>&nbsp;<a
href="https://cursor.com/agents?id=bc-7b2f8500-1a72-41a6-8f3c-4795a9b7ae1d"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-web-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-web-light.svg"><img alt="Open in Web"
src="https://cursor.com/open-in-web.svg"></picture></a>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
…to fix/useextracted-sourcemap

# Conflicts:
#	packages/next-intl/src/extractor/extractor/MessageExtractor.test.tsx
…urce maps (#2094)

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: DongYun Kang <kdy.1997.dev@gmail.com>
# Conflicts:
#	packages/next-intl/package.json
#	packages/next-intl/src/extractor/extractor/MessageExtractor.tsx
#	packages/swc-plugin-extractor/package.json
…lead to reset of messages with `useExtracted` (#2131)

<details>
<summary>Problem doc</summary>

# wiped out messages

## user comments

### colin 1

I might have run into another issue, but I’m not fully confident because
it’s hard to reproduce.

A few times, all translations in a locale were replaced with empty
strings. I first saw it while testing the canary build when pt.po
translations suddenly got wiped, but I couldn’t reproduce it and assumed
I had deleted that file by mistake and it had to be rebuilt.

However, now it happened twice on my branch using 4.5.2. I’ve probably
run the build 20+ times and only saw this happen 3 times, so I don’t
have a reliable repro. Just mentioning it in case it helps you spot
something. This happened twice with my pt.po file and once with es.po,
never with the source en.po file.

### colin 2

This just happened again when I was setting up this repo on a new
MacBook. It happened when I ran pnpm build from the root directory
(which uses Turborepo).

I have three translation files there: en.po, es.po, pt.po. Only es.po
got wiped out.

This happened only the first time I ran pnpm build. I couldn't reproduce
it again. One thing I realized is that this always happened when I was
building multiple apps at the same time (using pnpm build with
Turborepo). I don't know if it's just a coincidence since that's how I
often run builds, though.

### julius 1

I was migrating a project from useTranslations to useExtracted. json
files set up for useTranslations, and po files for useExtractions.

My process was following: Migrate one component or page to useExtracted,
and wait for the other locale to refresh - once that was done I stopped
the dev server (when editing target files with Poedit, something crashes
in the dev server, which results in the target files only partially
updating, or not at all until the dev server is restarted), and added
the translations to the target file.
Every time when clicking "Needs work" for some translation in Poedit and
starting the dev server, for a split second the nextjs browser tab
returned with "fuzzy comments not suppported" before emptying all values
in my "de.po" file.

Next.js: 16.0.3
next-intl: 4.5.5
MacOS Tahoe
Macbook Air M3 | Beefy hackintosh
Locales: "en.po" (source) and "de.po"

### next.config.ts

```ts
import type {NextConfig} from 'next';
import createNextIntlPlugin from 'next-intl/plugin';

const withNextIntl = createNextIntlPlugin({
  experimental: {
    srcPath: './src',
    extract: {
      sourceLocale: 'en'
    },
    messages: {
      path: './messages',
      format: 'po',
      locales: 'infer'
    }
  }
});

const nextConfig: NextConfig = {
  reactCompiler: true,
  experimental: {
    turbopackFileSystemCacheForDev: true
  }
};

export default withNextIntl(nextConfig);
```

request.ts

```ts
export const locales = ['de', 'en', 'fr'] as const;
export const defaultLocale: Locale = 'de';

export default getRequestConfig(async () => {
  const store = await cookies();
  const candidate = store.get('locale')?.value;
  const locale = hasLocale(locales, candidate) ? candidate : defaultLocale;

  const jsonMessages = await import(`../../messages/${locale}.json`);
  const poMessages = await import(`../../messages/${locale}.po`);

  return {
    locale,
    messages: {
      ...jsonMessages.default,
      ...poMessages.default
    }
  };
});
```

## issue description

Translation files (.po) are intermittently getting wiped out—all
translations replaced with empty strings. This affects non-source locale
files (pt.po, es.po, de.po) but never the source locale (en.po). It can
affect a single locale, while other target locales are not affected.

Observed triggers:

- Building multiple apps simultaneously with Turborepo
- Dev server crashes/restarts during file editing

## my notes

fuzzy flag is handled in separate pr and shouldnt make difference. colin
reports that it happened during builds.

## ideas

- Could it be that we reset target translations since all the ones from
the source are considered changed? Maybe the save date of their source
file randomly changes
- Check how often we read a catalog during build? Could there be a
concurrency issue somewhere? Or is there a chance that we write before
init is done?
- Multiple compiler instances in parallel builds: Turborepo creates
separate processes, each with its own singleton compiler instance.
Multiple CatalogManager instances writing to the same files
simultaneously without coordination
- Race condition in `saveLocale`: `translations.get(message.id) || ''`
(line 358) writes empty strings if `translationsByTargetLocale` hasn't
loaded yet or is empty
- Race condition in `performInitialScan`: Calls `loadMessages()` then
`save()`, but if `loadTargetMessages()` hasn't completed, `saveLocale`
could write empty translations
- Empty Map initialization window: In `loadTargetMessages` (line 170),
an empty Map is set before loading messages. If `save()` runs between
these steps, it writes empty strings
- No file locking: Multiple processes can write simultaneously, last
write wins, potentially overwriting valid translations with empty ones
- Timestamp check race: `lastWriteByLocale` check (line 341-353) can
fail with concurrent writes - process A reads file, process B writes,
process A writes over it
- `onLocalesChange` callback race: File watcher could trigger while save
is in progress, causing `loadLocaleMessages` to run concurrently with
`saveLocale`

## todo

- "when editing target files with Poedit, something crashes in the dev
server, which results in the target files only partially updating, or
not at all until the dev server is restarted" > test this
- investigate if turbopackFileSystemCacheForDev makes a difference
- add debug logging, ask users to use it, report back when it happens
- check how often file updates happen (during build?)


</details>
…Extracted` (#2134)

Move `setNestedProperty` to a central `utils.tsx` file and add a
`localeCompare` helper with a hardcoded "en" locale.

The `localeCompare` helper ensures consistent sorting behavior across
all environments by explicitly setting the locale to "en".

---
<a
href="https://cursor.com/background-agent?bcId=bc-fa0d4eda-5084-43d7-9de6-f85da87e45e3"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-cursor-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-cursor-light.svg"><img alt="Open in
Cursor"
src="https://cursor.com/open-in-cursor.svg"></picture></a>&nbsp;<a
href="https://cursor.com/agents?id=bc-fa0d4eda-5084-43d7-9de6-f85da87e45e3"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-web-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-web-light.svg"><img alt="Open in Web"
src="https://cursor.com/open-in-web.svg"></picture></a>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
…x encoding (#2128)

Support parsing, storing, and preserving PO flags and other arbitrary
message properties in the extractor.

This change aligns the extractor with the updated `po-parser`
capabilities, allowing external tools to add flags to messages which are
then preserved across compilation cycles. It also makes the
`ExtractedMessage` type more extensible for future properties.

---
<a
href="https://cursor.com/background-agent?bcId=bc-332ff50f-2647-4d07-b6a9-b0ae43c01439"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-cursor-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-cursor-light.svg"><img alt="Open in
Cursor"
src="https://cursor.com/open-in-cursor.svg"></picture></a>&nbsp;<a
href="https://cursor.com/agents?id=bc-332ff50f-2647-4d07-b6a9-b0ae43c01439"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://cursor.com/open-in-web-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://cursor.com/open-in-web-light.svg"><img alt="Open in Web"
src="https://cursor.com/open-in-web.svg"></picture></a>

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Preserves .po flags and arbitrary message metadata across compilation
by merging catalogs and refining save/load logic; upgrades po-parser to
^1.0.2.
> 
> - **Extractor/Catalog**:
> - Preserve arbitrary message properties (e.g., PO flags) by storing
full `ExtractedMessage` in `translationsByTargetLocale` and merging
disk/source data in `reloadLocaleCatalog`.
> - Add `loadCatalogsPromise` and ensure catalogs load before source
scan to avoid races; refine `onLocalesChange` chaining.
> - Update save flow to write source and targets in parallel, poll
timestamps to detect external edits, and merge previous entries when
emitting locale catalogs.
> - Ignore `references` in equality checks to prevent false positives;
rename `haveMessagesChanged` to `haveMessagesChangedForFile`.
> - Extend `ExtractedMessage` with index signature to allow extra
metadata.
> - Change `SaveScheduler` typing and `save`/`saveImpl` to return
`void`.
> - **Tests**:
> - Replace brittle waits with `sleep`/`waitForWriteFileCalls(atLeast)`;
broaden snapshots.
> - Add coverage for preserving/removing flags, metadata retention,
reference updates across catalogs, and race conditions when adding
locales or reloading catalogs.
>   - Reorganize `srcPath` filtering tests without behavioral change.
> - **Dependencies**:
>   - Bump `po-parser` from `^0.1.2` to `^1.0.2` (lockfile updated).
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
4a1733a. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
@vercel
Copy link

vercel bot commented Dec 2, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
next-intl-docs Ready Ready Preview Comment Dec 2, 2025 5:02pm
next-intl-example-app-router Ready Ready Preview Comment Dec 2, 2025 5:02pm
next-intl-example-app-router-without-i18n-routing Ready Ready Preview Comment Dec 2, 2025 5:02pm

amannn and others added 3 commits December 2, 2025 17:08
…2149)

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
…2150)

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
@amannn amannn merged commit 9b7c9fe into main Dec 3, 2025
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants