ESQL: TO_ASCII function

[pfcoperez]

Adds TO_ASCII string function to ESQL functions.

It escapes:

• Unicode characters when they don't have direct correspondence with ASCII characters.
• Java escape sequences such as \n, \t ...

Closes: #137282

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[github-actions]

🔍 Preview links for changed docs

• docs/reference/query-languages/esql/kibana/docs/functions/to_ascii.md
• docs/reference/query-languages/esql/functions-operators/string-functions.md

[github-actions]

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

• Check out the cumulative docs guidelines
• Reach out in the #docs Slack channel

[nik9000]

I looked at the docs for Postgresql and a few others and it looks like the function called ASCII returns an integer that is the ascii code for the first character.

I'm fine having a function that escapes non-ascii characters though. Is there a better name for it?

[pfcoperez]

I looked at the docs for Postgresql and a few others and it looks like the function called ASCII returns an integer that is the ascii code for the first character.

Thank you @nik9000 !

I'm fine having a function that escapes non-ascii characters though. Is there a better name for it?

ESCAPE sounds simple enough but it assumes the target alphabet is ASCII when you could be escaping things like just quotes in JSON, etc.

Following the convention of other functions in the catalog, like TO_BASE64, what do you think about TO_ASCII ?

[nik9000]

I believe format needs Locale.ROOT on the front of it. Without it the build complains.

[nik9000]

TO_ASCII feels fine with me.

[nik9000]

Could you name the capability fn_to_ascii instead please?

[nik9000]

Because this is going to be displayed in the docs it might be nicer to format it like:

ROW s = [
  "Hello world!\n\t",
  "世界!",
  "🌍",
  "Café naïve résumé",
  "こんにちは",
  " 🎉",
  "中文测试",
  "αβγδε",
  "日本語テスト"
  "🚀🔥💧🪨"
]
| EVAL s = TO_ASCII(s)

Otherwise it gets really wide to fit into the screen.

[nik9000]

It'd be nice to also have a csv-spec test for this that uses an index. Something like:

FROM airports
| WHERE abbrev = "BTS"
| EVAL ascii_name = TO_ASCII(name)
| KEEP name, ascii_name

Maybe even:

FROM airports
| WHERE TO_ASCII(abbrev) LIKE "%Mu\u(whatever ñ is)oz%"
| KEEP name

[mouhc1ne]

The common logic in these 11 test cases could be delegated to a private method, that only needs to be passed the input, the output and the test title. Would be nicer to read.

Also there is a way to add random tests for string functions, we should probably add those here as they test for more than just input/output. An example can be found in AbstractUrlEncodeDecodeTestCase.java#L78.

[mouhc1ne]

Is 9.2.0 still the correct version?