-
Notifications
You must be signed in to change notification settings - Fork 612
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
100% test coverage #75
Comments
Im a tech blogger , here is the link of my tech blog ( www.hackintoshrao.com ) , im interested in documentation and testing , on providing little guidance on how to go about it , i can help you guys out |
Would be interested in helping but I haven't the slightest clue how you write OS tests - looping in some people that may know how to help |
The main advantage of NodeOS is that the barebones layer is really small, and after that all is mostly build in Javascript and managed by NPM, so testing NodeOS is mostly write tests to its dependencies packages. The other advantage is that being minimalistic by design its footprint is small and there are very few packages to tests, so it could be feasable to have a 100% fully tested OS... :-) |
By using a package.json file] to define the NodeOS dependencies, it would be easy to recursively check them and see what's their code coverage and help them to improve it :-) |
I've just developed coverdeeps, a small tool that check a module dependencies and give a global tests coverage percentage instead of just only the own module, because having reliable dependencies are as important (or more!) than the module itself. It still lacks a lot of features, only support modules published on npm registry (not local ones or published only on a repository) and tests coverage info from coveralls.io so it's really pesimistic about the projects tests coverage, but at least it's a beginning :-) And as usual, pull-requests are welcome :-D |
This tool is fun to play with! I've been running 'coverdeeps' on everything for the last 20 minutes |
And what's your impression? :-D
|
I get the impression that I have a lot of tests to write for a lot ;-; |
I think this is a good thing, isn't it? ;-) |
hahaha nope! Because that means I have a lot of work, but it will be nice once that number raises haha! |
First of all, make coverdeeps more flexible... and get it to achieve 100%
|
Can we make a list of all things we need to test because i wann to help |
Coverdeeps should help on this by showing the tests coverage of each
|
For a list of tasks to complete, there is the roadmap. You can find a link On Tue, Apr 19, 2016, 1:27 AM Jesús Leganés Combarro <
|
Just for clarification, does NodeOS support native modules (compiled On Tue, Apr 26, 2016, 4:09 PM Lite McFish lite260@gmail.com wrote:
|
Not to install them with npm from inside NodeOS itself because we don't have yet a c compiler (probably nodeos-cross-toolchain would be reused for this), but you can add them on build time. |
Quick question: How are you defining coverage? If you're purely going by lines of code (which is what it appears that coverdeeps is doing), then you aren't necessarily covering all expected behaviors, since you have varying branches and whatnot. Are you accounting for branches, including branches implicit in operations such as division (which returns |
Good question. At this moment I only trust on what coveralls.io does, so if
|
Those things could work - having a metric like this is definitely a good first step to have. If you're trying to go for a perfectly bug free OS, however, then something that you might want to consider is actually going in and formally verifying the OS components (i.e. sitting down with the JS source and mathematically proving that each function actually does exactly what it's supposed to do). I believe there are tools that help automate this process (for instance, there's one called Coq that works for Haskell). It is an expensive process and not even possible in some cases (not to mention that Chrome V8 has bugs, so your proof can only be as correct as the V8 engine is), but if you can do it, then you never have to worry about bugs for that module ever again as long as it keeps the same feature set. And if code is so convoluted that you can't figure out what it's supposed to do, that might be an indication to use a different module. |
That's definitely dificult to achieve, since we are using both Linux and Node.js (and as you've said, this one use v8), and the two are really big projects, and as far as I've seen there's no way to get tests coverage statistics for them, imagine to get a mathematic audit of the code... The best we could achieve is to claim that there's a perfect score of 100% on NodeOS core modules and dependencies, this itself would bring a HUGE confidence on the system, isn't it? :-)
Or split it and make it simpler :-) |
Yes, that we could. I'm just saying that if we can prove that the JS that drives NodeOS core is correct, then we know that any bugs in NodeOS that we do find are V8's fault or the Linux kernel's fault, not NodeOS's fault. And then our JS code can serve as a reproducible example that we can draw up really quickly for the bug report. |
That's exactly what I was telling you. |
OK, that makes sense. :) |
Recently i encountered the exit code 71 which i traced back to line NodeOS#75 in scripts/build and saw that it used a - (minus) instead of a _ (underscore) so i changed the concatenation of the strings
Recently i encountered the exit code 71 which i traced back to line NodeOS#75 in scripts/build and saw that it used a - (minus) instead of a _ (underscore) so i changed the concatenation of the strings
If some software really needs to be reliable, that's the OS. We need to document and tests all the processes, and we must work to achieve a 100% testing coverage to be sure there's no hidden bugs.
The text was updated successfully, but these errors were encountered: