Skip to content

rabbitmq: resolve internal_interface through Ansible - #2578

Merged
berendt merged 2 commits into
mainfrom
stack/4-template-internal-interface
Aug 7, 2026
Merged

rabbitmq: resolve internal_interface through Ansible#2578
berendt merged 2 commits into
mainfrom
stack/4-template-internal-interface

Conversation

@ideaship

@ideaship ideaship commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Fixes the reported bug. Two commits: the reusable helper, then the switch-over.

The bug

get_rabbitmq_node_addresses() read internal_interface with ansible-inventory --host, which returns variables as defined, so a Jinja-valued internal_interface came back as the raw {{ ... }}. It then resolved that by hand: a regex captured the contents of the first {{ ... }} and the dotted path was walked through the ansible facts.

That covers exactly one shape — a single expression whose dotted path is rooted in facts, i.e. the testbed's internal_interface: "{{ ansible_local.testbed_network_devices.management }}" — and nothing else:

  • An internal_interface pointing at an inventory variable fails, because the walk only looks in the facts: Could not resolve template '{{ some_var }}' from facts for <host>. This is what the reporter hit.
  • A mixed literal and template such as vlan{{ vlan_id }} is passed through verbatim, because re.match requires {{ at offset 0; the lookup then asks for a fact named ansible_vlan{{ vlan_id }}.
  • The regex is unanchored, so {{ base }}.100 matches only the leading expression and silently discards the .100 tail, resolving to the wrong interface — a wrong answer rather than an error.
  • Filters, hostvars lookups and default() are not evaluated at all.

The fix

Delegate the lookup to Ansible, which is what osism/defaults all/README.md ("Consuming these values from code") prescribes for external consumers of these variables. The supported set becomes "whatever Jinja2 supports" instead of a list of anticipated shapes. The resolver goes away, along with the subsequent walk from interface name to ansible_<name> to ipv4.address, since the expression covers all of it.

resolve_in_host_context() has Ansible template the expression and copy the result into a file, run with -c local so the module executes on the controller: no connection is made, a node that is down still resolves, and the value is read back byte-exact rather than parsed out of human-readable output.

The package runs under more than one ansible-core — 2.19.11 in its own image, 2.18.x in the osism-ansible, kolla-ansible and ceph-ansible images — so the lookup avoids every surface where those differ:

  • Facts travel as extra vars, not a fact-cache plugin. The redis plugin lives in community.general, which is not installed here, and a jsonfile cache is honoured by an ad-hoc lookup on 2.18 but not on 2.19.
  • The value comes from a file, not a callback. Callback wording and stream differ between the two, and --tree is deprecated for removal in 2.23.
  • Success is decided by the return code, never by matching an error string: the undefined-variable sentinel is VARIABLE IS NOT DEFINED! on 2.18 and <<error1-'x' is undefined>> on 2.19.

The resolved value is validated as an IPv4 address before use, so a non-address string cannot be passed on as one.

Coverage

The xfail(strict=True) case added in the previous PR passes now, so its marker goes — with strict set, leaving it would fail the suite on the unexpected pass. That transition is the demonstration: the shapes that already worked are still covered by the same tests, and the one that did not now passes against real Ansible.

The two remaining unit tests that described only the deleted resolver's internals (a facts-walk yielding a non-string, and hitting a non-dict) go with the code they covered — Ansible has no such failure modes, so there is nothing to express at the integration level.

Verification. Against ansible-core 2.18.9 and 2.19.11: the reported shape, a dotted interface name, a dashed one, the fact-derived shape the old resolver supported, and a missing fact.

One behaviour change for review: a missing internal_interface is now reported through Ansible's undefined-variable message rather than a dedicated one.

osism status rabbitmq shares the helper and is fixed with it.

Fixes:

🤖 Generated with Claude Code

@berendt
berendt force-pushed the stack/4-template-internal-interface branch from 72a4cdb to 2941d94 Compare August 7, 2026 05:33
@ideaship
ideaship force-pushed the stack/4-template-internal-interface branch from 2941d94 to f1875b4 Compare August 7, 2026 05:59
@ideaship
ideaship force-pushed the stack/4-template-internal-interface branch from f1875b4 to 5b3a4a1 Compare August 7, 2026 06:54
@berendt
berendt force-pushed the stack/4-template-internal-interface branch from 5b3a4a1 to 6db241e Compare August 7, 2026 07:12
Base automatically changed from stack/3-interface-resolution-tests to main August 7, 2026 08:59
@berendt
berendt force-pushed the stack/4-template-internal-interface branch from 6db241e to 45650ab Compare August 7, 2026 08:59
@ideaship
ideaship marked this pull request as ready for review August 7, 2026 10:11

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 security issue, and 1 other issue

Security issues:

  • Detected subprocess function 'run' without a static string. If this data can be controlled by a malicious actor, it may be an instance of command injection. Audit the use of this call to ensure it is not controllable by an external resource. You may consider using 'shlex.escape()'. (link)

Fixed security issues:

  • Command injection from untrusted input passed to OS command execution (link)
Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="osism/utils/inventory.py" line_range="172" />
<code_context>
+            )
+
+        try:
+            with open(value_path) as fp:
+                return fp.read()
+        except OSError as exc:
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Open the result file with an explicit encoding to avoid locale-dependent behavior.

Using the platform default encoding is brittle for non-ASCII content and non‑UTF‑8 locales. Please specify `encoding="utf-8"` here and ensure the producing side also writes UTF‑8 so the encoding is consistent end to end.

Suggested implementation:

```python
        try:
            with open(value_path, encoding="utf-8") as fp:
                return fp.read()
        except OSError as exc:
            raise HostContextResolutionError("ansible wrote no value") from exc

```

To fully implement your suggestion end-to-end, you should also verify that the Ansible task or plugin producing `value_path` writes the file using UTF-8 encoding (e.g., ensure templates or modules use UTF-8 and that any explicit file writes specify `encoding: utf-8` or equivalent). Those changes will be in the Ansible side rather than in `osism/utils/inventory.py`.
</issue_to_address>

### Comment 2
<location path="osism/utils/inventory.py" line_range="150-156" />
<code_context>
            result = subprocess.run(
                command,
                capture_output=True,
                text=True,
                timeout=timeout,
                env=env,
            )
</code_context>
<issue_to_address>
**security (python.lang.security.audit.dangerous-subprocess-use-audit):** Detected subprocess function 'run' without a static string. If this data can be controlled by a malicious actor, it may be an instance of command injection. Audit the use of this call to ensure it is not controllable by an external resource. You may consider using 'shlex.escape()'.

*Source: opengrep*
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread osism/utils/inventory.py
)

try:
with open(value_path) as fp:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk): Open the result file with an explicit encoding to avoid locale-dependent behavior.

Using the platform default encoding is brittle for non-ASCII content and non‑UTF‑8 locales. Please specify encoding="utf-8" here and ensure the producing side also writes UTF‑8 so the encoding is consistent end to end.

Suggested implementation:

        try:
            with open(value_path, encoding="utf-8") as fp:
                return fp.read()
        except OSError as exc:
            raise HostContextResolutionError("ansible wrote no value") from exc

To fully implement your suggestion end-to-end, you should also verify that the Ansible task or plugin producing value_path writes the file using UTF-8 encoding (e.g., ensure templates or modules use UTF-8 and that any explicit file writes specify encoding: utf-8 or equivalent). Those changes will be in the Ansible side rather than in osism/utils/inventory.py.

Comment thread osism/utils/inventory.py
Add resolve_in_host_context(), which templates a Jinja2 expression in a
host's Ansible variable context and returns the result. Nothing calls it
yet; the callers follow.

"ansible-inventory --host" returns variables as defined, so anything
Jinja-valued comes back as the raw "{{ ... }}". osism/defaults
all/README.md ("Consuming these values from code") tells external tooling
how such a value has to be resolved instead: through templating in the
host context, never by re-implementing the expression in the consumer.
This helper is that operation, in one place, so callers do not each grow
their own approximation of Jinja.

It runs "ansible <host> -m copy" with the expression as the content and a
file in a temporary directory as the destination, then reads the file:

- "-c local" keeps the module on the controller. No connection is made to
  the host, so a node that is down or unreachable still resolves, and
  ANSIBLE_GATHERING=explicit states that no fact gathering happens.
- The value is read back from a file rather than parsed out of Ansible's
  output. The package runs under more than one ansible-core (2.19.11 in
  its own image, 2.18.x in the osism-ansible, kolla-ansible and
  ceph-ansible images), and their callbacks differ in wording and in
  which stream they use. "--tree" would give structured output but is
  deprecated for removal in ansible-core 2.23.
- Module arguments are passed as JSON, so a value containing spaces
  survives.
- Facts are supplied as extra vars rather than through a fact-cache
  plugin. The redis cache plugin lives in community.general, which is not
  installed here, and a jsonfile cache is honoured by an ad-hoc lookup on
  2.18 but not on 2.19. Extra vars behave identically on both, and the
  precedence is right for facts, which already outrank host_vars.
- Failure is decided by the return code alone. The undefined-variable
  sentinel differs between the two versions ("VARIABLE IS NOT DEFINED!"
  against "<<error1-'x' is undefined>>"), so matching on it would work on
  one and not the other. On failure the error text Ansible produced is
  passed through unparsed, because it names the variable that could not be
  resolved.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Roger Luethi <luethi@osism.tech>
get_rabbitmq_node_addresses() read internal_interface with
"ansible-inventory --host", which returns variables as defined, so a
Jinja-valued internal_interface came back as the raw "{{ ... }}". It then
resolved that by hand: a regex captured the contents of the first
"{{ ... }}" and the dotted path was walked through the ansible facts.

That covers exactly one shape -- a single expression whose dotted path is
rooted in facts, i.e. the testbed's

  internal_interface: "{{ ansible_local.testbed_network_devices.management }}"

and nothing else:

- An internal_interface pointing at an inventory variable fails, because
  the walk only ever looks in the facts:
  "Could not resolve template '{{ some_var }}' from facts for <host>".
- A mixed literal and template such as "vlan{{ vlan_id }}" is passed
  through verbatim, because re.match requires "{{" at offset 0, and the
  lookup then asks for a fact named ansible_vlan{{ vlan_id }}.
- The regex is unanchored, so "{{ base }}.100" matches only the leading
  expression and silently discards the ".100" tail, resolving to the
  wrong interface -- a wrong answer rather than an error.
- Filters, hostvars lookups and defaults are not evaluated at all.

Use resolve_in_host_context() instead, so Ansible does the templating in
the host's own variable context. This is what osism/defaults
all/README.md ("Consuming these values from code") prescribes for
external consumers, and it makes the supported set "whatever Jinja2
supports" rather than a list of anticipated shapes. The whole resolver
goes away, along with the subsequent walk from interface name to
ansible_<name> to ipv4.address, since the expression covers all of it.

The resolved value is validated as an IPv4 address before use. The return
code already reports a templating failure, but a value that comes back
looking nothing like an address must not be passed on as one either.

The integration case for an internal_interface that points at an inventory
variable was marked xfail(strict) when it was added, because the resolver
could not resolve it. It passes now, so the marker goes: with strict set,
leaving it would fail the suite on the unexpected pass. That is the
demonstration this change needed -- the shapes that already worked are
still covered by the same tests, and the one that did not now passes
against real Ansible rather than against a mock.

The tests for the deleted resolver go with it: Jinja2 traversal, the
non-string and non-dict cases, the interface-name to fact-key mapping and
the ipv4 extraction all tested behaviour that is now Ansible's. What
replaces them asserts the contract that remains -- that the expression and
the cached facts are handed over unchanged, that a resolution failure
surfaces Ansible's own message, and that a non-address result is refused.

Verified against ansible-core 2.18.9 and 2.19.11 with the reported
variable shape ("{{ vlan_var }}" where vlan_var is itself
"vlan{{ id }}"), a dotted interface name, a dashed one, and the
fact-derived shape the old resolver supported, plus a missing fact.

One behaviour change worth noting for review: a missing internal_interface
is now reported through Ansible's undefined-variable message rather than
a dedicated one.

osism status rabbitmq shares the helper and is fixed with it.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Roger Luethi <luethi@osism.tech>
@ideaship
ideaship force-pushed the stack/4-template-internal-interface branch from 45650ab to 386e976 Compare August 7, 2026 10:27
@ideaship

ideaship commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

@sourcery-ai dismiss

@ideaship
ideaship requested a review from berendt August 7, 2026 11:00
@berendt
berendt merged commit 3d9c4e5 into main Aug 7, 2026
3 checks passed
@berendt
berendt deleted the stack/4-template-internal-interface branch August 7, 2026 11:24
@github-project-automation github-project-automation Bot moved this from New to Done in Human Board Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants