PR #25387 caused test discovery to fail

[AlanyTan]

Type: Bug

Behaviour

Unittest discovery with Doctest integration fails after PR #25387.
Expected behavior:
Test panel can show doctest converted to unittest cases (base Python feature) in the list of tests. (which it was able to, with our patch) but it no longer does.

Steps to reproduce:

Put the following into test_main.py file, then go to discover test in the testing action panel:

# Patch DocTestCase to hack the module path issue for VSCode testing grouping
original_init = doctest.DocTestCase.__init__


def patched_init(self, test, optionflags=0, setUp=None, tearDown=None, checker=None):
    # testing hacking the names
    if hasattr(test, 'name'):
        module_hierarchy = test.name.split('.')
        # Check if the test name is a module path
        hierarchy_level = 1
        module_name = ''
        while (hierarchy_level := hierarchy_level + 1) < len(module_hierarchy):
            try_module_name = ".".join(module_hierarchy[:hierarchy_level])
            try:
                if sys.modules.get(try_module_name):
                    module_name = try_module_name
            except Exception as e:
                break

        test_name = test.name.removeprefix(module_name + ".")
        if test.filename.endswith("__init__.py"):
            # If the test is in an __init__.py file, we need to adjust the module name
            test.name = module_name + ".__init__._module_." + test_name
        elif test.filename.endswith(f"/{test_name}.py"):
            test.name = module_name + "." + test_name + "._module_./"
        elif not "." in test_name:
            test.name = module_name + "._module_." + test_name
    # Call original init
    original_init(self, test, optionflags, setUp, tearDown, checker)


# Apply the patch
doctest.DocTestCase.__init__ = patched_init


def load_tests(loader, tests, ignore) -> unittest.TestSuite:
    """
    standard hook for unittest to load tests
    Load all doctests from the current package and its subpackages.
    """

    current_dir = os.path.dirname(__file__)
    for _, module_name, _ in pkgutil.walk_packages([current_dir]):
        # package_name = module_name.split(".")[0]
        # Fix duplicated levels in module names
        # if module_name.startswith(package_name + "."):
        #     module_name = module_name[len(package_name) + 1:]
        try:
            module = importlib.import_module(module_name)
            module.__name__ = module_name  # Ensure the module name is correctly set to avoid duplication
        except Exception as e:
            print(f"Error importing {module_name}: {e}")
            continue

        tests.addTests(doctest.DocTestSuite(module, optionflags=doctest.NORMALIZE_WHITESPACE | doctest.ELLIPSIS))
    return tests

Add doc test to any .py file, i.e.

# main.py
""" a python file with doctest
    Example:
    >>> 2 + 2
    4
"""

This will cause a test discovery error (only after PR #25387)

and in the OUTPUT panel, error messages like below:
Skipping doctest as it is not supported for the extension. Test case: / main.module

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

XXX

Extension version: 2025.14.0
VS Code version: Code 1.104.1 (0f0d87fa9e96c856c5212fc86db137ac0d783365, 2025-09-17T23:36:24.973Z)
OS version: Darwin arm64 24.6.0
Modes:
Remote OS version: Linux x64 6.8.12-060812-generic
Remote OS version: Linux x64 6.8.12-060812-generic
Remote OS version: Linux x64 6.8.12-060812-generic
Remote OS version: Linux x64 6.8.12-060812-generic
Remote OS version: Linux x64 6.8.12-060812-generic

• Python version (& distribution if applicable, e.g. Anaconda): 3.12.11.final.0
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): System
• Value of the python.languageServer setting: Default

User Settings

languageServer: "Pylance"

testing
• unittestArgs: "<placeholder>"
• unittestEnabled: true

experiments
• optInto: ["pythonTestAdapter"]

Installed Extensions

Extension Name	Extension Id	Version
autopep8	ms-	2025.2.0
copilot	Git	1.364.0
copilot-chat	Git	0.31.0
debugpy	ms-	2025.10.0
js-debug	ms-	1.104.0
python	ms-	2025.12.0
vscode-containers	ms-	2.1.0
vscode-eslint	dba	3.0.16
vscode-js-profile-table	ms-	1.0.10
vscode-pylance	ms-	2025.7.1
vscode-python-envs	ms-	1.2.0

System Info

Item	Value
CPUs	Apple M4 (10 x 2400)
GPU Status	2d_canvas: enabled direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: enabled_on trees_in_viz: disabled_off video_decode: enabled video_encode: enabled webgl: enabled webgl2: enabled webgpu: enabled webnn: disabled_off
Load (avg)	2, 2, 2
Memory (System)	16.00GB (0.64GB free)
Process Argv	--crash-reporter-id 06771ef1-d261-4c01-b65d-327c27831e79
Screen Reader	no
VM	0%

Item	Value
Remote	Dev Container: AIOps_Platform @ fw12
OS	Linux x64 6.8.12-060812-generic
CPUs	12th Gen Intel(R) Core(TM) i5-1240P (16 x 1786)
Memory (System)	62.51GB (40.75GB free)
VM	0%

Item	Value
Remote	Dev Container: cust_proj_CONA_agenticAI @ fw12
OS	Linux x64 6.8.12-060812-generic
CPUs	12th Gen Intel(R) Core(TM) i5-1240P (16 x 2610)
Memory (System)	62.51GB (40.75GB free)
VM	0%

Item	Value
Remote	Dev Container: sap_on_azure_AI_monitor @ fw12
OS	Linux x64 6.8.12-060812-generic
CPUs	12th Gen Intel(R) Core(TM) i5-1240P (16 x 1786)
Memory (System)	62.51GB (40.75GB free)
VM	0%

Item	Value
Remote	Dev Container: Python 3 @ fw12
OS	Linux x64 6.8.12-060812-generic
CPUs	12th Gen Intel(R) Core(TM) i5-1240P (16 x 1786)
Memory (System)	62.51GB (40.75GB free)
VM	0%

Item	Value
Remote	Dev Container: Python 3 @ fw12
OS	Linux x64 6.8.12-060812-generic
CPUs	12th Gen Intel(R) Core(TM) i5-1240P (16 x 2857)
Memory (System)	62.51GB (40.75GB free)
VM	0%

A/B Experiments

vsliv368:30146709
pythonvspyt551:31249599
binariesv615:30325510
nativeloc1:31344060
dwcopilot:31170013
dwoutputs:31242946
copilot_t_ci:31333650
e5gg6876:31282496
996jf627:31283433
pythonrdcb7:31342333
usemplatestapi:31297334
6518g693:31334701
aj953862:31281341
nesew2to5:31336538
agentgpt4:31387437
82j33506:31327384
nes-set-on:31351930
onetestforazureexp:31335613
6abeh943:31336334
envsactivate1:31353494
0927b901:31350571
ovsp_t:31383296
0cj2b977:31352657
0574c672:31362109
cloudbuttont:31379625
todos-1:31384886
v2-prompt:31370220
control_gpt5applypatchexclusively:31387916
3efgi100_wstrepl:31382709
metis-embeddings:31387854
trigger-command-fix:31379601

[eleanorjboyd]

gotcha- so you had a patch to support doc tests?

[AlanyTan]

Yes. We have a patch to fix the doctest naming convention so it follows module.class.method convention and it also pads module level and functions (non-class) level doctest.

It worked pretty well for us for the past year until this PR is merged and arbitrarily rejected our doctest irrespective of if the name is already following the expectation.

Feel free to take the patch, or if you want a PR from us, let us know where do you like it to be submitted.

The only piece that we are still trying to improve is the line number mapping - this patch will link you back to the file where doctest are define, but not able to link you to the exact line yet...