Skip to content

Allow e2e tests to use configurable executable#365

Merged
asmacdo merged 3 commits intocon:mainfrom
asmacdo:configurable-e2e-executable
Jan 9, 2026
Merged

Allow e2e tests to use configurable executable#365
asmacdo merged 3 commits intocon:mainfrom
asmacdo:configurable-e2e-executable

Conversation

@asmacdo
Copy link
Copy Markdown
Member

@asmacdo asmacdo commented Jan 9, 2026

Summary

  • Add DUCT_TEST_EXECUTABLES env var to configure which executable(s) the e2e tests use
  • By default, tests run twice: once with duct and once with con-duct run
  • External builds (e.g., PyInstaller) can override to test their executable

Usage

Default (tests both entry points):

pytest test/duct_main/test_e2e.py -v

External executable:

DUCT_TEST_EXECUTABLES="./dist/con-duct run" pytest test/duct_main/test_e2e.py -v

Test plan

Default test run (both entry points)
$ pytest test/duct_main/test_e2e.py::test_sanity -v
============================================= test session starts =============================================
platform linux -- Python 3.13.11, pytest-9.0.2, pluggy-1.6.0 -- /home/austin/.venvs/duct/bin/python
cachedir: .pytest_cache
rootdir: /home/austin/devel/duct
configfile: pyproject.toml (WARNING: ignoring pytest config in tox.ini!)
plugins: rerunfailures-16.1
collected 2 items                                                                                             

test/duct_main/test_e2e.py::test_sanity[duct] PASSED                                                    [ 50%]
test/duct_main/test_e2e.py::test_sanity[con-duct run] PASSED                                            [100%]

============================================== 2 passed in 0.93s ==============================================
PyInstaller executable test run
$ DUCT_TEST_EXECUTABLES="./linux-test-2 run" pytest test/duct_main/test_e2e.py::test_sanity -v
============================================= test session starts =============================================
platform linux -- Python 3.13.11, pytest-9.0.2, pluggy-1.6.0 -- /home/austin/.venvs/duct/bin/python
cachedir: .pytest_cache
rootdir: /home/austin/devel/duct
configfile: pyproject.toml (WARNING: ignoring pytest config in tox.ini!)
plugins: rerunfailures-16.1
collected 1 item                                                                                              

test/duct_main/test_e2e.py::test_sanity[./linux-test-2 run] PASSED                                      [100%]

============================================== 1 passed in 0.49s ==============================================

🤖 Generated with Claude Code

Add DUCT_TEST_EXECUTABLES env var to override the duct command used in
e2e tests. By default, tests run twice with both "duct" and "con-duct run"
entry points. External builds (e.g., PyInstaller) can set this to test
their executable against the same e2e suite.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings January 9, 2026 19:39
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for configurable executable testing in e2e tests through the DUCT_TEST_EXECUTABLES environment variable. By default, tests run with both duct and con-duct run entry points, and external builds (like PyInstaller) can override this to test their own executables.

Key changes:

  • Introduced DUCT_TEST_EXECUTABLES environment variable for specifying test executables
  • Created a pytest fixture duct_cmd that parametrizes tests across different executables
  • Updated all e2e test functions to accept and use the duct_cmd fixture

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread test/duct_main/test_e2e.py Outdated
asmacdo and others added 2 commits January 9, 2026 13:42
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Handle spaces after commas in DUCT_TEST_EXECUTABLES.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@asmacdo asmacdo requested a review from CodyCBakerPhD January 9, 2026 19:48
@asmacdo asmacdo added the semver-tests Add or improve existing tests label Jan 9, 2026
@asmacdo
Copy link
Copy Markdown
Member Author

asmacdo commented Jan 9, 2026

@CodyCBakerPhD ran against https://github.com/codycbakerphd/duct-releases/ and smoketests are green!

 duct  ~/devel/duct   configurable-e2e-executable ± DUCT_TEST_EXECUTABLES="./con-duct_v0.18.0_linux run" pytest test/duct_main/test_e2e.py -v             
============================================= test session starts =============================================
platform linux -- Python 3.13.11, pytest-9.0.2, pluggy-1.6.0 -- /home/austin/.venvs/duct/bin/python
cachedir: .pytest_cache
rootdir: /home/austin/devel/duct
configfile: pyproject.toml (WARNING: ignoring pytest config in tox.ini!)
plugins: rerunfailures-16.1
collected 17 items                                                                                            

test/duct_main/test_e2e.py::test_sanity[./con-duct_v0.18.0_linux run] PASSED                            [  5%]
test/duct_main/test_e2e.py::test_spawn_children[./con-duct_v0.18.0_linux run-1-plain] PASSED            [ 11%]
test/duct_main/test_e2e.py::test_spawn_children[./con-duct_v0.18.0_linux run-1-subshell] PASSED         [ 17%]
test/duct_main/test_e2e.py::test_spawn_children[./con-duct_v0.18.0_linux run-1-nohup] PASSED            [ 23%]
test/duct_main/test_e2e.py::test_spawn_children[./con-duct_v0.18.0_linux run-1-setsid] PASSED           [ 29%]
test/duct_main/test_e2e.py::test_spawn_children[./con-duct_v0.18.0_linux run-2-plain] PASSED            [ 35%]
test/duct_main/test_e2e.py::test_spawn_children[./con-duct_v0.18.0_linux run-2-subshell] PASSED         [ 41%]
test/duct_main/test_e2e.py::test_spawn_children[./con-duct_v0.18.0_linux run-2-nohup] PASSED            [ 47%]
test/duct_main/test_e2e.py::test_spawn_children[./con-duct_v0.18.0_linux run-2-setsid] PASSED           [ 52%]
test/duct_main/test_e2e.py::test_spawn_children[./con-duct_v0.18.0_linux run-10-plain] PASSED           [ 58%]
test/duct_main/test_e2e.py::test_spawn_children[./con-duct_v0.18.0_linux run-10-subshell] PASSED        [ 64%]
test/duct_main/test_e2e.py::test_spawn_children[./con-duct_v0.18.0_linux run-10-nohup] PASSED           [ 70%]
test/duct_main/test_e2e.py::test_spawn_children[./con-duct_v0.18.0_linux run-10-setsid] PASSED          [ 76%]
test/duct_main/test_e2e.py::test_session_modes[./con-duct_v0.18.0_linux run-new-session] PASSED         [ 82%]
test/duct_main/test_e2e.py::test_session_modes[./con-duct_v0.18.0_linux run-current-session] PASSED     [ 88%]
test/duct_main/test_e2e.py::test_session_mode_behavior_difference[./con-duct_v0.18.0_linux run] PASSED  [ 94%]
test/duct_main/test_e2e.py::test_logging_levels[./con-duct_v0.18.0_linux run] PASSED                    [100%]

============================================= 17 passed in 19.28s =============================================
 duct  ~/devel/duct   configurable-e2e-executable ± 

@CodyCBakerPhD
Copy link
Copy Markdown
Contributor

Great to hear!

@asmacdo asmacdo merged commit 8c5e906 into con:main Jan 9, 2026
26 of 27 checks passed
@github-actions
Copy link
Copy Markdown

🚀 PR was released in v0.20.0 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

released semver-tests Add or improve existing tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants