Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve instrumentation testability #387

Closed
6 tasks done
MarcoRossignoli opened this issue Apr 30, 2019 · 2 comments
Closed
6 tasks done

Improve instrumentation testability #387

MarcoRossignoli opened this issue Apr 30, 2019 · 2 comments
Labels
enhancement General enhancement request

Comments

@MarcoRossignoli
Copy link
Collaborator

MarcoRossignoli commented Apr 30, 2019

Check/debug code coverage is not so simple; languages and compilers abstracts IL.

Working on some issues #369 #344 #158 I found that could be very useful to have a way to "test" entire process to verify the lines we expect to cover, this is the core goal of coverlet.

My proposal is to discuss and do some architectural changes to allow "simple" F5 debug all process and also allow to add "complex" test to discover bugs related to compilers changes that sometimes invalidates assumptions, for instance async/await state machines.

Plan:

  • Remove static Coverage object #409 Find a way to remove all static from code, we have already an issue open to track it MSBuild data should not be passed between tasks via static fields #364
    In this way we can run "parallel" tests
  • Abstract file system, at the moment we cannot unload assemblies and we cannot "restore" instrumented locked module
    After some attempts I found unuseful at the moment abstract file system to improve instrumentation testability, in addition to file locking we've a problem with tracker shared statics. To avoid statics on tracker we need to re-engineer too much code for "testing" pourpose.
    Next idea is to write an out of process "runner".
  • Create an out of process helper to run full instrumentation test (Improve instrumentation tests #420))
  • Add specific test for current and future issues on coverage

Postponed(did some initial work on #393 but too drastic for now #393 (comment))

Changes above allows us to also measure and improve the performance(complete benchmarks) and verify if the instrumentation is ok(i.e. inspect IL) and to expose in future Coverlet.Core as package with "stable apis".

This steps should resolve

https://github.com/tonerdo/coverlet/blob/0956d1e8da6b7265b0b5e4f93af3499c4be99cc8/test/coverlet.core.tests/CoverageTests.cs#L23-L24

/cc @tonerdo @petli

@MarcoRossignoli MarcoRossignoli added the enhancement General enhancement request label Apr 30, 2019
@MarcoRossignoli
Copy link
Collaborator Author

If we agree on the plan, I could start to work on it.

@tonerdo
Copy link
Collaborator

tonerdo commented May 1, 2019

@MarcoRossignoli I'm in agreement. Especially the first item. Let me know if you need any help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement General enhancement request
Projects
None yet
Development

No branches or pull requests

2 participants