fix(e2e): unconditionally skip Telegram reachability probe for fake-token tests by hunglp6d · Pull Request #4555 · NVIDIA/NemoClaw

hunglp6d · 2026-05-30T01:22:58Z

Summary

Three nightly E2E jobs (messaging-providers-e2e, channels-stop-start-e2e,
token-rotation-e2e) failed in run 26668952264
because commit b13e4f4 introduced checkTelegramReachability() which validates
the Telegram bot token via HTTP before onboarding.

The E2E scripts use fake tokens. On GitHub Actions runners
api.telegram.org is reachable, so the existing conditional guard
(if ! curl … api.telegram.org) never fires. The new probe sends the fake
token, receives HTTP 401, and silently drops Telegram from onboard —
breaking every assertion that expects Telegram to be configured.

Root cause

File	Issue
`src/lib/onboard/telegram-reachability.ts` (line 65-69)	HTTP 401/404 → `{skipped: true}` removes Telegram from `found` channels
`src/lib/onboard.ts` (`setupMessagingChannels`)	Filters Telegram out of `found` when `reachability.skipped` is true
`test/e2e/test-messaging-providers.sh` (line 567-572)	Guard checks host reachability, not token validity
`test/e2e/test-channels-stop-start.sh` (line 422-427)	Same guard pattern
`test/e2e/test-token-rotation.sh`	No guard at all

Fix

Export NEMOCLAW_SKIP_TELEGRAM_REACHABILITY=1 unconditionally in all three
test scripts so the probe is always bypassed when running with fake tokens.
This is the correct layer: the workflow YAML already passes fake tokens as
env vars, and the scripts know they are running with fake credentials.

Changes

test/e2e/test-messaging-providers.sh — replace conditional
curl-reachability check with unconditional export
test/e2e/test-channels-stop-start.sh — same
test/e2e/test-token-rotation.sh — add unconditional export before
first token assignment

Validation

Note: The automation token lacks actions:write / workflows scope,
so a validation workflow_dispatch could not be triggered automatically.
A maintainer should trigger:
gh workflow run nightly-e2e.yaml --repo NVIDIA/NemoClaw \
  --ref fix/nightly-e2e-telegram-probe-fake-token-a5e7e63 \
  -f jobs="messaging-providers-e2e,channels-stop-start-e2e,token-rotation-e2e"

Test plan

Trigger nightly-e2e.yaml on this branch with the three affected jobs
Verify messaging-providers-e2e passes (Telegram channel configured)
Verify channels-stop-start-e2e passes (Telegram stop/start works)
Verify token-rotation-e2e passes (Telegram token rotation detected)
Confirm no regressions in unrelated nightly jobs

Signed-off-by: Hung Le hple@nvidia.com

Summary by CodeRabbit

Tests
- E2E test setups now skip the Telegram reachability probe only when configured tokens appear to be fake, avoiding network-dependent failures for fake-token runs.
- Removed reliance on an external host-reachability check, making test behavior deterministic for fake-token scenarios.

…oken tests Commit b13e4f4 introduced `checkTelegramReachability()` which sends an HTTP request to api.telegram.org to validate the bot token. On HTTP 401/404 it marks the channel as "skipped", silently dropping Telegram from onboard. The three E2E scripts that use fake Telegram tokens (test-messaging-providers, test-channels-stop-start, test-token-rotation) only set NEMOCLAW_SKIP_TELEGRAM_REACHABILITY when api.telegram.org is *unreachable* from the host. On GitHub Actions runners the host IS reachable, so the env var is never set, the probe fires with the fake token, gets 401, and onboard drops Telegram — breaking every assertion that expects Telegram to be configured. Fix: export NEMOCLAW_SKIP_TELEGRAM_REACHABILITY=1 unconditionally in all three scripts so the probe is always bypassed when running with fake tokens. Signed-off-by: Hung Le <hple@nvidia.com>

copy-pr-bot · 2026-05-30T01:23:02Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

coderabbitai · 2026-05-30T01:23:13Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6a68a769-73a2-4be2-8be1-1a82471c18a6

📥 Commits

Reviewing files that changed from the base of the PR and between dd8b33f and b88c011.

📒 Files selected for processing (3)

test/e2e/test-channels-stop-start.sh
test/e2e/test-messaging-providers.sh
test/e2e/test-token-rotation.sh

📝 Walkthrough

Walkthrough

Three E2E test scripts add a fake-token detector and set NEMOCLAW_SKIP_TELEGRAM_REACHABILITY=1 when tokens match the fake pattern, removing prior curl-based api.telegram.org reachability checks and restricting the skip behavior to fake-token test runs.

Changes

E2E Telegram fake-token gating

Layer / File(s)	Summary
Add helper and Phase 0 gating `test/e2e/test-token-rotation.sh`	Adds `is_fake_telegram_token` and, after sandbox/gateway pre-clean (Phase 0), conditionally exports `NEMOCLAW_SKIP_TELEGRAM_REACHABILITY=1` when configured Telegram tokens look fake and the flag is unset.
install_for_active_agent fake-token gating `test/e2e/test-channels-stop-start.sh`	Adds `is_fake_telegram_token` and changes `install_for_active_agent` to set `NEMOCLAW_SKIP_TELEGRAM_REACHABILITY=1` when the Telegram bot token is fake, removing the prior `curl https://api.telegram.org` reachability probe.
Phase 1 restrict skip to fake-token path `test/e2e/test-messaging-providers.sh`	Replaces logic that could set the skip flag based on host `curl` reachability with a condition that sets it only for the fake-token E2E path (unset skip flag, empty `TELEGRAM_BOT_TOKEN_REAL`, and `TELEGRAM_TOKEN` contains `fake`).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

NVIDIA/NemoClaw#4307: Related changes around using NEMOCLAW_SKIP_TELEGRAM_REACHABILITY=1 to skip Telegram reachability during onboarding.
NVIDIA/NemoClaw#4554: Modifies E2E Telegram handling and the conditions under which the skip flag is set.

Suggested labels

fix, nightly-e2e

Suggested reviewers

cv
ericksoa

Poem

🐰 I found a token that shouted "fake!",
So I hopped and said, "Let's skip the flaky wake."
A tiny helper, a flag set with care,
Now E2E runs skip the probe and breathe easy air. ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: unconditionally skipping Telegram reachability probe for fake-token E2E tests across three test scripts.
Linked Issues check	✅ Passed	All coding objectives from issue `#4556` are met: the fix exports NEMOCLAW_SKIP_TELEGRAM_REACHABILITY=1 in all three affected test scripts (test-messaging-providers.sh, test-channels-stop-start.sh, test-token-rotation.sh) to prevent the probe from sending fake tokens to api.telegram.org.
Out of Scope Changes check	✅ Passed	All changes are in-scope: modifications are limited to the three E2E test scripts specified in issue `#4556`, adding fake-token detection helpers and conditional skip logic without unrelated alterations.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch fix/nightly-e2e-telegram-probe-fake-token-a5e7e63

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

…ram-probe-fake-token-a5e7e63 # Conflicts: # test/e2e/test-messaging-providers.sh

github-actions · 2026-05-30T03:19:15Z

E2E Advisor Recommendation

Required E2E: None
Optional E2E: messaging-providers-e2e, channels-stop-start-e2e, token-rotation-e2e

Dispatch hint: messaging-providers-e2e,channels-stop-start-e2e,token-rotation-e2e

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

None. No merge-blocking E2E is required because this PR changes only E2E test scripts/harness behavior and cannot affect NemoClaw runtime or real user flows. The affected suites are useful optional validation of the test changes themselves.

Optional E2E

messaging-providers-e2e (high): Optional validation that the modified fake-token Telegram reachability skip still lets the messaging provider/placeholder/L7 proxy suite run as intended.
channels-stop-start-e2e (high): Optional validation for the edited channel lifecycle E2E harness, especially fake-token onboarding before stop/start/rebuild assertions.
token-rotation-e2e (medium): Optional validation that token-rotation E2E still handles fake Telegram tokens without requiring live Telegram reachability.

New E2E recommendations

messaging-onboarding-hermeticity (medium): Multiple suites now contain local fake-token reachability-skip logic. A hermetic fake Telegram API/onboarding validation would reduce dependency on live Telegram reachability and remove duplicated skip heuristics.
- Suggested test: Add a hermetic Telegram onboarding reachability E2E using a fake Telegram API endpoint for fake-token runs.

Dispatch hint

Workflow: nightly-e2e.yaml
jobs input: messaging-providers-e2e,channels-stop-start-e2e,token-rotation-e2e

github-actions · 2026-05-30T03:19:15Z

E2E Scenario Advisor Recommendation

Required scenario E2E: None
Optional scenario E2E: None

Workflow run

Full scenario advisor summary

E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required scenario E2E

None. Changes are limited to legacy test/e2e/ scripts and do not touch test/e2e-scenario/, scenario workflows, scenario metadata, expected-state contracts, validation suite catalog, or scenario runtime code. Scenario E2E is not required; legacy E2E coverage is owned by the general E2E advisor.

Optional scenario E2E

None.

Relevant changed files

None.

github-actions · 2026-05-30T03:20:42Z

PR Review Advisor

Findings: 0 needs attention, 2 worth checking, 0 nice ideas
Since last review: 3 prior items resolved, 0 still apply, 1 new item found

Review findings

🛠️ Needs attention

None.

🔎 Worth checking

Source-of-truth review needed: Telegram reachability bypass in E2E install/onboard scripts: The advisor marked localized patch analysis as needs_followup.
- Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
- Evidence: test-messaging-providers.sh and test-channels-stop-start.sh now skip only for fake-token conditions; test-token-rotation.sh skips if either rotation token is fake. The production source still drops Telegram on 401/404 in telegram-reachability.ts/onboard.ts.
Mixed fake/real token-rotation runs still bypass Telegram validation globally (test/e2e/test-token-rotation.sh:166): The revised scripts mostly fix the previous unconditional bypass, but token-rotation now exports NEMOCLAW_SKIP_TELEGRAM_REACHABILITY=1 when either TELEGRAM_BOT_TOKEN_A or TELEGRAM_BOT_TOKEN_B contains 'fake'. Because that environment variable persists for all later onboard phases, a mixed fake/real rotation run would skip Telegram reachability validation even for the real-token phase. This is likely low practical risk because this E2E is documented as rotation plumbing rather than live Telegram validation, but it leaves the real-token no-skip invariant untested.
- Recommendation: Either document that token-rotation is fake-token-only when any token is fake, or narrow the skip to the current token before each onboard phase. A lightweight static/shell test that real-token-only mode does not set NEMOCLAW_SKIP_TELEGRAM_REACHABILITY would also preserve the regression boundary.
- Evidence: test-token-rotation.sh sets the skip before the first install with: is_fake_telegram_token "$TELEGRAM_BOT_TOKEN_A" || is_fake_telegram_token "$TELEGRAM_BOT_TOKEN_B". Later phases rotate TELEGRAM_BOT_TOKEN and call 'nemoclaw onboard --non-interactive' while inheriting the same exported skip.

🌱 Nice ideas

None.

Since last review details

Current findings:

Source-of-truth review needed: Telegram reachability bypass in E2E install/onboard scripts: The advisor marked localized patch analysis as needs_followup.
- Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
- Evidence: test-messaging-providers.sh and test-channels-stop-start.sh now skip only for fake-token conditions; test-token-rotation.sh skips if either rotation token is fake. The production source still drops Telegram on 401/404 in telegram-reachability.ts/onboard.ts.
Mixed fake/real token-rotation runs still bypass Telegram validation globally (test/e2e/test-token-rotation.sh:166): The revised scripts mostly fix the previous unconditional bypass, but token-rotation now exports NEMOCLAW_SKIP_TELEGRAM_REACHABILITY=1 when either TELEGRAM_BOT_TOKEN_A or TELEGRAM_BOT_TOKEN_B contains 'fake'. Because that environment variable persists for all later onboard phases, a mixed fake/real rotation run would skip Telegram reachability validation even for the real-token phase. This is likely low practical risk because this E2E is documented as rotation plumbing rather than live Telegram validation, but it leaves the real-token no-skip invariant untested.
- Recommendation: Either document that token-rotation is fake-token-only when any token is fake, or narrow the skip to the current token before each onboard phase. A lightweight static/shell test that real-token-only mode does not set NEMOCLAW_SKIP_TELEGRAM_REACHABILITY would also preserve the regression boundary.
- Evidence: test-token-rotation.sh sets the skip before the first install with: is_fake_telegram_token "$TELEGRAM_BOT_TOKEN_A" || is_fake_telegram_token "$TELEGRAM_BOT_TOKEN_B". Later phases rotate TELEGRAM_BOT_TOKEN and call 'nemoclaw onboard --non-interactive' while inheriting the same exported skip.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

github-actions · 2026-05-30T05:54:45Z

Selective E2E Results — ✅ All requested jobs passed

Run: 26675728346
Target ref: fix/nightly-e2e-telegram-probe-fake-token-a5e7e63
Requested jobs: messaging-providers-e2e,channels-stop-start-e2e,token-rotation-e2e
Summary: 3 passed, 0 failed, 0 skipped

Job	Result
channels-stop-start-e2e	✅ success
messaging-providers-e2e	✅ success
token-rotation-e2e	✅ success

Keep live Telegram runs on the onboard validation path while preserving the fake-token E2E workaround for messaging provider, channel lifecycle, and token rotation tests. Signed-off-by: Carlos Villela <cvillela@nvidia.com>

cv · 2026-05-30T06:23:07Z

Addressed the advisor findings in b88c011: the Telegram reachability skip now applies only in fake-token mode, preserves live-token onboard validation, and documents the source boundary/removal condition in the affected E2E scripts. Verified with shfmt, shellcheck, and bash -n for the three changed scripts.

hunglp6d mentioned this pull request May 30, 2026

nightly-e2e: Telegram reachability probe breaks fake-token E2E jobs #4556

Closed

Merge remote-tracking branch 'origin/main' into fix/nightly-e2e-teleg…

dd8b33f

…ram-probe-fake-token-a5e7e63 # Conflicts: # test/e2e/test-messaging-providers.sh

sandl99 marked this pull request as ready for review May 30, 2026 05:28

sandl99 self-assigned this May 30, 2026

sandl99 added E2E End-to-end testing — Brev infrastructure, test cases, nightly failures, and coverage gaps Integration: Telegram Use this label to identify Telegram bot integration issues with NemoClaw. VRDC Issues and PRs submitted by NVIDIA VRDC test team. labels May 30, 2026

sandl99 requested a review from cv May 30, 2026 05:56

sandl99 added the v0.0.56 Release target label May 30, 2026

cv mentioned this pull request May 30, 2026

fix(onboard): revert Telegram non-interactive skip #4561

Closed

12 tasks

cv approved these changes May 30, 2026

View reviewed changes

test(e2e): scope Telegram probe skip to fake tokens

b88c011

Keep live Telegram runs on the onboard validation path while preserving the fake-token E2E workaround for messaging provider, channel lifecycle, and token rotation tests. Signed-off-by: Carlos Villela <cvillela@nvidia.com>

cv approved these changes May 30, 2026

View reviewed changes

cv merged commit e79461c into main May 30, 2026
30 checks passed

cv deleted the fix/nightly-e2e-telegram-probe-fake-token-a5e7e63 branch May 30, 2026 06:34

Conversation

hunglp6d commented May 30, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Root cause

Fix

Changes

Validation

Test plan

Summary by CodeRabbit

Uh oh!

copy-pr-bot Bot commented May 30, 2026

Uh oh!

coderabbitai Bot commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested labels

Suggested reviewers

Poem

❌ Failed checks (1 warning)

Uh oh!

github-actions Bot commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Advisor Recommendation

E2E Recommendation Advisor

Required E2E

Optional E2E

New E2E recommendations

Dispatch hint

Uh oh!

github-actions Bot commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Scenario Advisor Recommendation

E2E Scenario Advisor

Required scenario E2E

Optional scenario E2E

Relevant changed files

Uh oh!

github-actions Bot commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review Advisor

🛠️ Needs attention

🔎 Worth checking

🌱 Nice ideas

Uh oh!

github-actions Bot commented May 30, 2026

Selective E2E Results — ✅ All requested jobs passed

Uh oh!

cv commented May 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

hunglp6d commented May 30, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented May 30, 2026 •

edited

Loading

github-actions Bot commented May 30, 2026 •

edited

Loading

github-actions Bot commented May 30, 2026 •

edited

Loading

github-actions Bot commented May 30, 2026 •

edited

Loading