Feature Requests/Ideas

[rcarriga]

Some ideas that may or may not be implemented in future:

• File watching: Auto run tests when relevant code has updated
• Code coverage: Display code coverage using signs/virtual text/highlighting
• Configurable discovery: Tweak test discovery for performance Performance problems on large repos #13
• Per-project configuration: Change adapters, discovery, potentially other config depending on project directory Performance problems on large repos #13

[olimorris]

File watching - I love the idea of having the summary split open and being able to see how my whole test suite is affected by my changes.

Code coverage - This would also be great to have and I use nvim-coverage now to pretty awesome effect:

Ideas from my side:

• Documentation: Would love there to be a wiki or guide on how to create adapters (I'd be happy to contribute). Having completed the RSpec one, most of it was really straight forward but the challenging part was collecting results and reaching out to yourself solved a big headache. Would be great to highlight and provide some solutions to those problems to help get the adapter community growing.
• Adapter tests: I tried to write some tests for the RSpec adapter but hit some weird error. Would be great to ensure that adapters work with specific versions of neotest (assuming you're going to move to some versioning mechanism at some point) and that we can easily run them against specific edge cases. For the RSpec adapter I'm envisaging users coming to me with edge cases that I need to manually test against. Danger is that I then break other cases in trying to account for those edge cases. Some form of adapter testing would be amazing.

[rcarriga]

Thanks for the the pointers on the docs! Will get around to making a fleshed out wiki entry to make things clearer.

What issue were you having with running tests? Fairly used to writing plenary tests at this point so should be able to help out. One obvious one is to make sure for any async functions to use the async it wrapper. Examples are in the neotest repo

Also neotest now follow semantic versioning so it'll be clear when there is a breaking change. Hopefully one day plugin managers can handle more than just equality versions so users can stay up to date until a breaking change.

[olimorris]

Regarding the tests, we've actually got some nice ones running over at https://github.com/olimorris/neotest-rspec. We were running into a "c-boundary" error which was resolved by making the required tests, async. One thing I've yet to do is write some full integration tests with neotest and the adapter working.

Also, regarding the guide on creating adapters...I'd be happy to make a first attempt.

[rcarriga]

It'd be great to have someone take a stab at docs. If you'd like to, I would appreciate it! Of course also happy to fill in the gaps if you find some things unclear

[gugahoi]

Just curious if there are any updates regarding File watching: Auto run tests when relevant code has updated?

This would be a great addition! Alternatively, if anyone is willing to share, how are others workflows?

[jfpedroza]

I actually hadn't tried it. I just tried it, and it didn't work, but it is an issue with the package and how I got the adapter to work. I'll see how to get it to work.

[jfpedroza]

I got it to work 😄. The only thing missing is a way to reset the tests back to "running" on save.

[rcarriga]

Brilliant! Is that just an aesthetic issue? As in, everything is running and results update correctly, the user just isn't shown that it is happening?

[jfpedroza]

Yes, it's only aesthetic. You save and when new results arrive, the sign is updated. But the user doesn't know tests are running again.

[rcarriga]

I've created an issue for this so I can keep track of it 👍 #169

[Issafalcon]

Hi!

So this might be quite specific to dotnet, but there is an issue that the users are facing at the moment (including myself) in that the way dotnet test commands work. It either looks for the nearest .csproj file in the directory specified in the command, or the nearest .sln file.

For the most part, running tests are currently done at the project level, and this works well. However, the dotnet test runner will overwrite the test output file multiple times, if it is run from the .sln root folder, because each projects tests are run independently, with results collected independently. The end result is that the last project to run it's tests will be the one with its results in the output file. See the result output when I run the whole test suite from a .sln root folder:

My first thought is that the solution to this would be if the adapter.build_spec function, could return an array of specs like:

return {
  {
    command = command_string_1,
    context = {
      results_path = results_path_project_1,
      file = position.path_of_project_1,
      id = position.id_of_project_1,
    },
  },
 {
    command = command_string_2,
    context = {
      results_path = results_path_project_2,
      file = position.path_of_project_2,
      id = position.id_of_project_2,
    },
  }

}

Then they could maybe run sequentially or in parallel, each with their own process and result file (like the case would be if I hit 'run' on each of the individual projects from the summary window in quick succession).

Forgive me if you can already do something like this, or there are ways that this can be done in the adapters! If either is the case, it would be great to see how to do it so I can add this feature to the dotnet adapter. Even if there is a away to recursively call this build_spec function in the adapter, as if each project was running tests in isolation.

[rcarriga]

You can actually already return an array of specs 😄

neotest/lua/neotest/adapters/interface.lua

Line 34 in 392808a

---@return nil | neotest.RunSpec | neotest.RunSpec[]

For reference here is the original reason for the feature #93 (comment)

[Issafalcon]

Aha! Fantastic! Should have checked that out first 😄