New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document Test Type Definitions #26

Closed
kennylavender opened this Issue Nov 30, 2018 · 6 comments

Comments

@kennylavender
Copy link
Contributor

kennylavender commented Nov 30, 2018

There is still some uncertainty around which category a test falls in issues/prs.

There doesn't seem to globally accepted test definitions in software development, so I think it would help if we document our test type definitions.

Here are a couple links to definitions that don't completely agree. Just providing for information and guidance, not suggesting we follow one or the other. We had originally agreed to follow Eric's but we never documented it and we could add more clarification for ourselves.

@lautarodragan

This comment has been minimized.

Copy link
Member

lautarodragan commented Nov 30, 2018

Agreed 100%. Great initiative 🚀

I find Eric's, RCM's and every other definition I've read vague at best.

Like you said, we can use their (and other's) definitions for guidance but we'll have to define our own.

Personally I like this approach:

Functional Tests

Functional tests treat the application being tested as a black box. They are not aware of whether inside the application there are controllers, DAOs or whatever. They do have knowledge of the external dependencies of the application, such as MySQL or MongoDB, and are free to mock them or not, but they can only "inject" these mocks via the normal usage of the application (passing the URL via environment variables for example).

This seems similar to what RCM defines as System Tests. System feels less confusing to me than functional, though.

I've mostly seen automated UI tests get called functional tests, too, even though they are completely different. All UI tests are rather intrusive and most times don't treat the application as a black box.

For example, GET /works is what I call a functional test.

Integration Tests

Regarding integration tests, I agree with RCM's vision, although I find no reason why only the architect / tech lead should write them.

To me, integration test is a catch-all term referring to any test that is in the middle of a functional and unit test. That is, they don't treat the application as a black box, they can be aware of specific functions, classes or files, but they are broader in scope then unit tests.

For example, BlockInfoDAO.insertBlockInfo is what I call an integration test.

Unit Tests

In the case of unit tests, from RCM's definition, what he calls micro-test is what I call unit-test. Under most definitions, unit tests means literally anything. Everything can be considered a unit.

In my mind, a test is a unit test if it tests a pure function or composition of pure functions.

Maybe pure test would be a more intuitive name under that description?

Notes

Unfortunately, too many programmers think that the rules for unit tests are different – that they can be “junk code” written using any ad hoc style that they find convenient.

Couldn't agree more. In every project I've worked, as the tests grew larger and larger, the also grew more disorganized and copy-pasted.

@krobi64

This comment has been minimized.

Copy link
Contributor

krobi64 commented Nov 30, 2018

All UI tests are rather intrusive and most times don't treat the application as a black box.

How so? UI tests approach an application the way the end user does, who definitely treats the application as a black box. Yes, they are fragile because of how automating interaction with the UI works today, but if you put a human as the UI testing engine, the application is a black box.

From Robert C Martin wrt integration AND system tests:

These are plumbing tests. They are not business rule tests. Their purpose is to confirm that the sub-assembly (or entire system) has been integrated and configured correctly.

Those are tests that ensure that containers, applications, etc. are simply able to communicate with each other correctly. Can Frost-API reach poet-node, can poet-node reach rabbitmq. That does not appear to be our use of functional or integration. And that is why he thinks the Tech Lead or architect should write them: to ensure the system is communicating among the various sub-parts as designed.

That the View Module displays a claim uploaded to the API module and eventually displays the block Info associated with that claim appears to fall under Acceptance Testing.

@krobi64 krobi64 closed this Nov 30, 2018

@krobi64 krobi64 reopened this Nov 30, 2018

@krobi64

This comment has been minimized.

Copy link
Contributor

krobi64 commented Nov 30, 2018

oops

@geoffturk

This comment has been minimized.

Copy link
Contributor

geoffturk commented Dec 1, 2018

Please review our current testing process doc and suggest changes/additions:

https://github.com/poetapp/documentation/blob/master/process/testing.md

@geoffturk geoffturk added the docs label Dec 1, 2018

@geoffturk geoffturk added this to the PML1 milestone Dec 1, 2018

@kennylavender

This comment has been minimized.

Copy link
Contributor Author

kennylavender commented Dec 1, 2018

Oops, guess we did document it, completely forgot about that.

@geoffturk

This comment has been minimized.

Copy link
Contributor

geoffturk commented Dec 2, 2018

Yeah, well have a read and let me know if you would change anything we wrote there. I put that together with Warren, but evidently did not bring enough attention to it! 😃

@geoffturk geoffturk closed this Dec 2, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment