Problem
The supported-languages tables on the search and filtered-stream operator pages list deprecated ISO 639-1 codes that the lang: operator rejects:
| Language |
Docs say |
API accepts |
| Indonesian |
in |
id |
| Hebrew |
iw |
he |
A request such as GET /2/tweets/search/all?query=lang:in fails validation with:
Invalid language specifier 'in'. Language specifier must be one of [..., id, ..., he, ...]
Query validation has required the canonical BCP-47 codes since 2020; the docs were never updated. This was reported by an enterprise customer (Emplifi) who was blocked trying to filter Indonesian posts.
Affected pages
Requested change
- In the supported-languages tables, change Indonesian from
in to id and Hebrew from iw to he on both pages (and localized copies).
- Add a note that the
lang response field still returns the legacy codes ("lang": "in" for Indonesian, "lang": "iw" for Hebrew) while the lang: operator requires the canonical codes (id, he). Filtering with lang:id matches all Indonesian posts, including those returned with "lang": "in" — no data is missed. Documenting this explicitly will prevent confusion, since the mismatch is intentional and stable (changing the response field would break existing consumers).
Background
Indonesian's ISO 639-1 code changed from in to id in 1989 (likewise Hebrew iw → he). The language classifier and response rendering use Java's legacy Locale.getLanguage() codes, while search query validation requires canonical BCP-47 tags. Both codes are indexed, so matching is unaffected — only the operator input and the docs are strict/stale.
Problem
The supported-languages tables on the search and filtered-stream operator pages list deprecated ISO 639-1 codes that the
lang:operator rejects:inidiwheA request such as
GET /2/tweets/search/all?query=lang:infails validation with:Query validation has required the canonical BCP-47 codes since 2020; the docs were never updated. This was reported by an enterprise customer (Emplifi) who was blocked trying to filter Indonesian posts.
Affected pages
x-api/posts/search/integrate/operators.mdx(lines ~150, ~153) — https://docs.x.com/x-api/posts/search/integrate/operators#supported-languagesx-api/posts/filtered-stream/integrate/operators.mdx(lines ~204, ~207)ja/,ko/,es/versions of both pagesRequested change
intoidand Hebrew fromiwtoheon both pages (and localized copies).langresponse field still returns the legacy codes ("lang": "in"for Indonesian,"lang": "iw"for Hebrew) while thelang:operator requires the canonical codes (id,he). Filtering withlang:idmatches all Indonesian posts, including those returned with"lang": "in"— no data is missed. Documenting this explicitly will prevent confusion, since the mismatch is intentional and stable (changing the response field would break existing consumers).Background
Indonesian's ISO 639-1 code changed from
intoidin 1989 (likewise Hebrewiw→he). The language classifier and response rendering use Java's legacyLocale.getLanguage()codes, while search query validation requires canonical BCP-47 tags. Both codes are indexed, so matching is unaffected — only the operator input and the docs are strict/stale.