Summary
Two generated ES functional test scripts run to completion but produce incorrect assertions, printing FAIL: expected <field> = <value> against live Elasticsearch output.
Observed in Buildkite ES functional matrix
FAIL: expected queries = [object Object]
(from test/functional/es/esql/30_queries.sh)
FAIL: expected acknowledged = true
(from test/functional/es/ingest/10_basic.sh)
Analysis
esql/30_queries.sh — expected queries = [object Object] is a tell-tale sign the codegen serialised a nested JS object into the comparison literal with default String(value) instead of JSON.stringify. The match expression in the source YAML likely has a structured value (array/object) that the generator should either serialise with jq or assert field-by-field.
ingest/10_basic.sh — expected acknowledged = true. Either (a) the CLI's --format=json output for this endpoint doesn't include acknowledged at the top level (e.g. wrapped in a data: envelope), or (b) the assertion is reading the wrong path. Worth diffing actual CLI output vs. what the generator assumes.
Suggested investigation
- Run both scripts against a local ES
9.1.0 with set -x and capture the actual CLI output.
- Compare against the
match: / is_true: block in the upstream YAML to determine whether the issue is codegen (wrong path), CLI (output shape mismatch), or both.
- If it's codegen, add a mapper test covering object-valued
match steps.
Acceptance criteria
Summary
Two generated ES functional test scripts run to completion but produce incorrect assertions, printing
FAIL: expected <field> = <value>against live Elasticsearch output.Observed in Buildkite ES functional matrix
Analysis
esql/30_queries.sh—expected queries = [object Object]is a tell-tale sign the codegen serialised a nested JS object into the comparison literal with defaultString(value)instead ofJSON.stringify. The match expression in the source YAML likely has a structured value (array/object) that the generator should either serialise withjqor assert field-by-field.ingest/10_basic.sh—expected acknowledged = true. Either (a) the CLI's--format=jsonoutput for this endpoint doesn't includeacknowledgedat the top level (e.g. wrapped in adata:envelope), or (b) the assertion is reading the wrong path. Worth diffing actual CLI output vs. what the generator assumes.Suggested investigation
9.1.0withset -xand capture the actual CLI output.match:/is_true:block in the upstream YAML to determine whether the issue is codegen (wrong path), CLI (output shape mismatch), or both.matchsteps.Acceptance criteria