[ENHANCMENT] Implement a setting to switch between Test/Task Discovery mode

[mardukbp]

Describe the bug
When the heading Tasks is used no green triangles are drawn in the left gutter.

To Reproduce
Steps to reproduce the behavior:

1. Open a file containing the code below.

If possible add some example source code like:

*** Settings ***
Library    OperatingSystem

*** Tasks ***
PATH should contain C:\\Users
    ${PATH} =    Get Environment Variable    PATH
    Should Contain    ${PATH}    C:\\Users

Expected behavior
It should be possible to execute Tasks.

Screenshots/ Videos
With the heading Tasks:

With the heading Test Cases:

Logs
Copy the messages from VSCode "Output" view for RobotCode and RobotCode Language Server for the specific folder/workspace.

The output is empty

Desktop (please complete the following information):

• VS Code Version 1.64.2
• RobotCode Version 0.5.5
• OS: Windows 10
• Python Version: 3.8.10
• RobotFramework Version 4.1.3
• Additional tools like robocop, robotidy: None

Additional context
Add any other context about the problem here.

[d-biehl]

This is because Robotframework does not support mixing Tests Cases and Tasks in the same project.
If you open the test explorer you will get an error message like

"Parsing '*\tasktest.robot' failed: Conflicting execution modes. File has tasks but files parsed earlier have tests. Fix headers or use '--rpa' or '--norpa' options to set the execution mode explicitly."

If you create a project with only task, you will get the expected behavior.

[d-biehl]

on the other side there is an option for robot to treat all tasks as testcases and vice versa.
So I can implement something that you can be able to select which mode robot should use if discovering tests/tasks.

[d-biehl]

show error/warning if the mode does not match

[mardukbp]

Let's say I have a directory Project, which contains tests.robot and tasks.robot. Robot Framework does not support executing robot Project, but the following causes no problem

cd Project
robot tests.robot
robot tasks.robot

However, when I open Project in VS Code, RobotCode scans both files, pressumably in alphabetic order and, since tests.robot is the last file, it sets the mode to --norpa. However, this is not the desired behavior. As a user I expect that when the active file contains only a Test Cases section the mode is --norpa and when the active file contains only a Tasks section the mode is --rpa. This corresponds to the behavior of the CLI. I don't want to have to set the mode manually, because it can be set automatically.

[d-biehl]

@mardukbp Testdiscovery is depending on robotframework, and if you run your project with

robot .

you will get

[ ERROR ] Parsing '*\tasks.robot' failed: Conflicting execution modes. File has tasks but files parsed earlier have tests. Fix headers or use '--rpa' or '--norpa' options to set the execution mode explicitly.

Test discovery needs to know, what it should expect and the decision is taken by robot algorithm. Robotcode tries to be as close as possible to RobotFramework, and that's how RobotFramework works.

If you want to execute a single suite file, without test discovery, then you can run it with the "run menu", then only the current file is executed.

[d-biehl]

@mardukbp And it is not defined in which order the files are read.

[mardukbp]

When I have a .robot file open I expect to be able to execute each of the tests/tasks in the file individually by clicking on the green triangle. It should not matter what other files are present in the directory hierarchy. Executing a single test/task does not amount to executing robot .. There should be a way to execute robot . and a way to execute robot -t (with --rpa and without it). That way the VS Code GUI actually mirrors the robot CLI.

[d-biehl]

@mardukbp I understand you ;-)

Let me think about whether it makes sense to mix tests and tasks together in one project and run them intermixed.
Or if it makes sense to point out that in the project tests and task are mixed and he should decide if this test/task should run in RPA or in NONRPA mode.

Running a single test/task is not the problem, but when I want to run a suite or a whole project and I don't know anymore which are now tasks or tests. No matter how we do it, we get a mistake or an undefined behaviour of the test execution.

Maybe we can ask the community at Slack?

[mardukbp]

Now I understand you :) My suggestion would be:

• add a Status Bar Item displaying the current mode (like the Vim extension)
• add a Command to switch to the mode that is not currently active
• during the execution include or exclude test suites according to the active mode
  • a .robot file qualifies for RPA execution if it only contains a Tasks section
  • a directory qualifies for RPA execution if all the .robot files in it qualify for RPA execution
  • a .robot file qualifies for normal execution if it only contains a Test Cases section
  • a directory qualifies for normal execution if all the .robot files in it qualify for normal execution
• display an error toast if the active file contains both a Test Cases section and a Tasks section

I believe the above shoud lead to correct behavior.