Skip to content

fix: parse identifiers with leading underscores - #371

Merged
santiagocardo merged 1 commit into
developfrom
fix/leading-underscore-identifiers-v2
Sep 2, 2026
Merged

fix: parse identifiers with leading underscores#371
santiagocardo merged 1 commit into
developfrom
fix/leading-underscore-identifiers-v2

Conversation

@santiagocardo

@santiagocardo santiagocardo commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Problem

Some payloads may have keys that start with an underscore, such as _vnd. Expressions traversing them stop parsing at the underscore:

Expression.evaluate_as_string!("@event.message._vnd.v1.chat", context)
# => "%{\"_vnd\" => %{...}}._vnd.v1.chat"
#    (the inspected message map, then the rest as literal text)

The atom combinator in Expression.Parser requires the first character of an identifier to be a letter or digit, so ._vnd fails the attribute parse, and the expression is truncated after event.message.

Fix

Allow leading underscores in the atom grammar while still requiring at least one letter or digit, so a bare _ keeps failing to parse.

atom =
  ascii_string([?_], min: 0)
  |> ascii_string([?a..?z, ?A..?Z, ?0..?9], min: 1)
  |> ascii_string([?a..?z, ?A..?Z, ?0..?9, ?_, ?-], min: 0)

Behavior changes

Input Before After
@event.message._vnd.v1.chat truncated after event.message, tail rendered as literal text resolves the full chain
@_missing (not in context) rendered _missing — the @ was silently swallowed by an empty expression match rendered @_missing, the documented round-trip for unresolved variables
@(_) literal, no expression parsed unchanged

The @(_) row matters: keeping the min: 1 letter-or-digit requirement is what stops a bare underscore becoming an identifier, so the existing guard test stays green and Elixir's "underscore means unused" convention doesn't bleed into map keys.

This supersedes #369, which was closed unmerged while still a draft.

Journey expressions could not traverse webhook vendor keys that start
with an underscore: @event.message._vnd.v1.chat rendered the inspected
message map followed by ._vnd.v1.chat as literal text, because the atom
combinator required identifiers to start with a letter or digit.

The _vnd namespace is added by the platform's own webhook enrichment, so
flow authors cannot rename it, and the only workaround was bracket
syntax (event.message["_vnd"]["v1"]["chat"]).

An atom still needs at least one letter or digit, so a bare _ keeps
failing to parse and @(_) remains literal text.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@santiagocardo santiagocardo self-assigned this Sep 1, 2026
@santiagocardo
santiagocardo requested a review from smn September 1, 2026 22:59
@santiagocardo
santiagocardo merged commit 1e1d5e6 into develop Sep 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants