Add end to end test devoted to ctest #4186

hippo91 · 2024-11-24T10:30:33Z

This changes test system

The following changes are proposed:

This PR is a proposal to add end to end test devoted to ctest part of the extension.

It adds a directory named single-root-ctest in the end-to-end-tests directory.

It also changes slightly the source directory in order:

to avoid early return in the refreshTests method (ctest.ts file) when in testing mode;
to change the return code of the ctest method (cmakeProject.ts) so that successfull test return 0 and failure returns -1.

The purpose of this change

When doing change to the ctest part of the extension, i did not feel very comfortable not being backed up by tests.
This PR tries to circumvent this.

hippo91 · 2025-03-22T14:29:49Z

@gcampbell-msft i integrated these tests in the PR build pipelines.

I configured C and CXX compilers for linux and windows systems. I do not have any windows os at home so i'm not able to test that. Neither can i on Mac OS. I was counting on the CI to check that.

* updated templates * wording updates * updated changelog * template updates

* Adds the getAllCMakeProjects method * Loop over all CMake projects of the workspace to provide tasks. Also pass the workspaceFolder to the provideTask method * Add workspacefolder property to the CustomBuildTaskTerminal class. This is necessary to make each task run in the correct project * Simplify getAllCMakeProjects method. It is a simple delegation to the eponym method of projectController * The scope is now always a workspaceFolder * Adds the getDefaultPresetNameForProject function * Renames getDefaultPresetName function into getDefaultPresetNameForActiveProject * Renames getActiveProject method into getProject method Using workspaceFolder scope allows to retrieve the project using this folder * Adds doc * Get the preset for the specified workspace using the new getDefaultPresetNameForProject function * Fix trailing space * Adds comment * Adds the method getProjectForFolder in the extension. It is mostly a delegation to the eponym method of the projectController * Update CHANGELOG.md (microsoft#3232) --------- Co-authored-by: Garrett Campbell <86264750+gcampbell-msft@users.noreply.github.com>

* feat: add `cmake.ctest.testSuiteDelimiter.debugLaunchTarget` * fix: setting name * fix: description name * chore: add changelog entry --------- Co-authored-by: Garrett Campbell <86264750+gcampbell-msft@users.noreply.github.com>

…icrosoft#4321) * ctest: Don't require backtrace in presence of DEF_SOURCE_LINE * docs: add CHANGELOG entry --------- Co-authored-by: Garrett Campbell <86264750+gcampbell-msft@users.noreply.github.com>

* Fix lowercase include paths on Mac * Update CHANGELOG for version 1.21 --------- Co-authored-by: Garrett Campbell <86264750+gcampbell-msft@users.noreply.github.com>

… due to trailing \r, and false parsing of new "multiple definitions" error (microsoft#4341) * Fix gnuld error parsing false positive on make errors, false negative due to trailing \r, and false parsing of new "multiple definitions" error * Move bug fix entry in CHANGELOG to 1.21 section

* graphics * more images * readme updates * whitespace * html sanitization * made picture clarity better * Update README.md * Update README.md * rewording suggestion from Garrett Co-authored-by: Garrett Campbell <86264750+gcampbell-msft@users.noreply.github.com> * rewording suggestion from Garrett Co-authored-by: Garrett Campbell <86264750+gcampbell-msft@users.noreply.github.com> * rewording suggestion from Garrett Co-authored-by: Garrett Campbell <86264750+gcampbell-msft@users.noreply.github.com> * rewording suggestion from Garrett Co-authored-by: Garrett Campbell <86264750+gcampbell-msft@users.noreply.github.com> * reordered features * rewording and fixing broken contributing link * removed extra period * made feature and banner updates * updated alt text * reverted wording * removed cache editor feature image * updated wording for presets support feature * removed extra image --------- Co-authored-by: Garrett Campbell <86264750+gcampbell-msft@users.noreply.github.com>

* Update README.md added a brief description of the features we support * updated readme

Co-authored-by: csigs <csigs@users.noreply.github.com>

hippo91 · 2025-04-05T10:30:11Z

@gcampbell-msft i tried to debug the failing CI tests but was unable to understand what was failing in the build step.

I suppose this is the build of the C++ test project but cannot confirm.
For now i try to run the CI locally using act. If you have any idea or hint about how i can debug the CI i'll be happy to try them.

gcampbell-msft · 2025-04-11T16:15:58Z

@hippo91 Update, it looks like the builds are failing tests due to an unexpected temp folder, possibly due to differences in OS's. Could you investigate?

OR, please let us know if you'd like help with this PR, and we can aim to help out on this PR for milestone 1.21.

hippo91 · 2025-04-12T10:54:17Z

@gcampbell-msft i am trying to investigate but i do not know how i can have access to build output or log. Have you got any idea?
Another approach that i'am trying is to run the CI locally thanks to act. Maybe is there another possibility?

gcampbell-msft · 2025-04-14T12:47:23Z

@hippo91 Are you not able to view the build log if you click on the github links?

Our CI pipelines run publicly in github. Here is a link to the Windows one, for example, https://github.com/microsoft/vscode-cmake-tools/actions/runs/14145701868/job/40095177801?pr=4186

hippo91 · 2025-04-16T20:15:28Z

@gcampbell-msft i am able to see the build log but for linux the only interesting thing is Warning: ls] 2025-04-07T11:17:46.863Z [warning] [proc] The command: /home/runner/work/_temp/-1604695791/cmake-3.18.3-Linux-x86_64/bin/cmake --build /home/runner/work/vscode-cmake-tools/vscode-cmake-tools/test/end-to-end-tests/single-root-ctest/project-folder/build -- exited with code: 1 but i have no access to the inner log of this command.

It seems like the cmake command does not receive the correct config preset.
It is really weird because on my laptop (running debian) using act, the CI is successful.

gcampbell-msft · 2025-04-16T20:20:21Z

@hippo91 I'm also interested in the failure for the Windows build. However, I also can't see any more information. I'll do my best to make time to investigate this sometime in the next week or so. If I don't get to it, we do hope to take this in our 1.21 milestone, which we hope to release in the next month or two. Thanks for your patience.

hippo91 · 2025-04-16T20:34:11Z

@gcampbell-msft the problem with windows is due to the fact i dumbly hardcoded path to /tmp which of course does not exist on windows. I should be able to fix this. However the only way for me to check windows execution is through the CI.

And for Linux in the CI for now i have no clue.

Using /tmp is not compatible with windows use

hippo91 · 2025-04-17T18:32:22Z

@gcampbell-msft i think i fixed the windows problem. Can you relaunch the CI please?

hippo91 · 2025-04-17T19:37:55Z

@gcampbell-msft surprisingly linux ci seems to be ok.

I removed an unused CMAKE variable for windows. It was generating what could be an error.

hippo91 · 2025-04-20T09:37:57Z

@gcampbell-msft the linux part of the CI is ok but i'm struggling with the windows one (the macos one is failing in an unrelated test). If you could have a look it would be nice. Please tell me if i can do anything to help.

Guillaume Peillex and others added 30 commits November 24, 2024 11:08

Adds a new end to end test directory devoted to ctest tests

9c1b1f2

Fix non stopping breakpoints

e101ea0

Adds scripts entry for new end to end ctest

072859f

Adds configurations for new end to end ctest test

f562055

Do not forbid test refreshing even in test mode

24c9e3f

Fix end to end test devoted to ctest

2e83da2

Test is functionnal when running through debug

fa8a419

Add "allowParallelJobs" in the config

b4d2839

Removes useless files

237079b

Replaces TestCleanup target with simpler call in the teardown method

6c4642f

Add cmake.ctest.allowParallelJobs setting

c8d5ce6

Upgrade cmake minimum required version to avoid warnings

4cfe9fc

Sets up two tests. One with parallel jobs enabled and the other without

4a6202e

Remove useless file

add157a

Adds two tests dealing with testSuiteDelimiter

14d7963

Remove generated test file between each test

8c057e3

Do not delete file if it is not present

9d90005

Fix test suite delimiter

5107e59

Do not add a comma if no text is appended

cc818ae

Delete test file results before the test

f3dbbe2

Create cleanUpTestResultFiles to factorize code

3d9f137

Adds explicit message to failing assertions

838d8a2

Fix comment

cbe131c

Generalize output writing

2e7a64a

Protect empy successor when dealing with last element of the container

6c6290f

Adds doc and format

7f05cff

Move suite initialization in the right method

3858fff

Await for settings change promise

31119f2

Set CMT_TESTING to one

6dec1a4

Restore previous state

acfd55e

Select the preset according to the OS type

b1fc68c

snehara99 and others added 13 commits March 29, 2025 13:12

Updated Issue Template Selection (microsoft#4335)

a491c46

* updated templates * wording updates * updated changelog * template updates

feat: add cmake.ctest.debugLaunchTarget (microsoft#4273)

b392d58

* feat: add `cmake.ctest.testSuiteDelimiter.debugLaunchTarget` * fix: setting name * fix: description name * chore: add changelog entry --------- Co-authored-by: Garrett Campbell <86264750+gcampbell-msft@users.noreply.github.com>

fix(ctest): don't require backtrace in presence of DEF_SOURCE_LINE (m…

63357d2

…icrosoft#4321) * ctest: Don't require backtrace in presence of DEF_SOURCE_LINE * docs: add CHANGELOG entry --------- Co-authored-by: Garrett Campbell <86264750+gcampbell-msft@users.noreply.github.com>

Fix lowercase include paths on Mac (microsoft#4325)

cec677d

* Fix lowercase include paths on Mac * Update CHANGELOG for version 1.21 --------- Co-authored-by: Garrett Campbell <86264750+gcampbell-msft@users.noreply.github.com>

fix getProjectForFolder (microsoft#4348)

937d162

Don't require LCOV to be found in test project (microsoft#4351)

c2579d7

Update README.md (microsoft#4350)

f0fb4e8

* Update README.md added a brief description of the features we support * updated readme

be more explicit in what we check (microsoft#4352)

faa7327

Localization - Translated Strings (microsoft#4344)

8d6a836

Co-authored-by: csigs <csigs@users.noreply.github.com>

Localization - Translated Strings (microsoft#4353)

1888d74

Co-authored-by: csigs <csigs@users.noreply.github.com>

Merge branch 'main' into add_end_to_end_ctest_tests

eaa4ff0

itodirel assigned gcampbell-msft Apr 15, 2025

hippo91 added 2 commits April 17, 2025 19:16

Merge branch 'main' into add_end_to_end_ctest_tests

c197f40

Changes the location of temp directory holding the tests (TESTS_DIR).

46ba27e

Using /tmp is not compatible with windows use

Remove useless CMAKE variable

6be0025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add end to end test devoted to ctest #4186

Add end to end test devoted to ctest #4186

Uh oh!

hippo91 commented Nov 24, 2024

Uh oh!

hippo91 commented Mar 22, 2025

Uh oh!

hippo91 commented Apr 5, 2025

Uh oh!

gcampbell-msft commented Apr 11, 2025

Uh oh!

hippo91 commented Apr 12, 2025

Uh oh!

gcampbell-msft commented Apr 14, 2025

Uh oh!

hippo91 commented Apr 16, 2025

Uh oh!

gcampbell-msft commented Apr 16, 2025 •

edited

Loading

Uh oh!

hippo91 commented Apr 16, 2025

Uh oh!

hippo91 commented Apr 17, 2025

Uh oh!

hippo91 commented Apr 17, 2025

Uh oh!

hippo91 commented Apr 20, 2025

Uh oh!

Uh oh!

Add end to end test devoted to ctest #4186

Are you sure you want to change the base?

Add end to end test devoted to ctest #4186

Uh oh!

Conversation

hippo91 commented Nov 24, 2024

This changes test system

The purpose of this change

Uh oh!

hippo91 commented Mar 22, 2025

Uh oh!

hippo91 commented Apr 5, 2025

Uh oh!

gcampbell-msft commented Apr 11, 2025

Uh oh!

hippo91 commented Apr 12, 2025

Uh oh!

gcampbell-msft commented Apr 14, 2025

Uh oh!

hippo91 commented Apr 16, 2025

Uh oh!

gcampbell-msft commented Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hippo91 commented Apr 16, 2025

Uh oh!

hippo91 commented Apr 17, 2025

Uh oh!

hippo91 commented Apr 17, 2025

Uh oh!

hippo91 commented Apr 20, 2025

Uh oh!

Uh oh!

gcampbell-msft commented Apr 16, 2025 •

edited

Loading