Skip to content

ci(test): distinguish "skipped due to missing tools" from "tested and passed" in startup script tests #3054

@bitflicker64

Description

@bitflicker64

Problem

The three startup regression test scripts exit 0 when required tools are missing from the environment:

# test-start-hugegraph.sh, test-start-hugegraph-pd.sh, test-start-hugegraph-store.sh
if ! command -v "$tool" >/dev/null 2>&1; then
    echo "SKIP: required tool '$tool' not found — skipping test suite"
    exit 0
fi

Exit code 0 is indistinguishable from "all tests ran and passed". If a CI runner is missing lsof, curl, java, or crontab, the step silently reports success and no one notices the tests never ran.

This was flagged as a non-blocking follow-up in the review of #3044 and #3047.

Expected behavior

When required tools are missing, the test scripts should exit with a distinct code (e.g. 77, the conventional POSIX skip code used by Automake and TAP) so CI can tell the difference between:

  • exit 0 — tested and passed
  • exit 77 — skipped due to missing environment
  • exit 1 — tested and failed

The CI workflow steps should handle 77 explicitly and emit a ::notice:: annotation instead of failing.

Files to change

Test scripts — change exit 0 to exit 77 in the tool-check block:

  • hugegraph-server/hugegraph-dist/src/assembly/travis/test-start-hugegraph.sh (lines 148–153)
  • hugegraph-server/hugegraph-dist/src/assembly/travis/test-start-hugegraph-pd.sh (lines 132–137)
  • hugegraph-server/hugegraph-dist/src/assembly/travis/test-start-hugegraph-store.sh (lines 113–127)

CI workflow steps — wrap the test call to handle exit 77:

  • .github/workflows/server-ci.yml — "Run start-hugegraph.sh foreground mode tests" step (lines 73–79)
  • .github/workflows/pd-store-ci.yml — "Run start-hugegraph-pd.sh foreground mode tests" step (lines 110–114) and the equivalent Store step

CI wrapper pattern:

run: |
  set +e
  $TRAVIS_DIR/test-start-hugegraph.sh $SERVER_DIR
  EXIT=$?
  set -e
  if [ $EXIT -eq 77 ]; then
    echo "::notice::Startup tests skipped — required tools not available in this environment"
    exit 0
  fi
  exit $EXIT

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingci-cdBuild or deploytestsAdd or improve test cases

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions