New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TODO] Get rid of unittest.TestCase #42

Closed
webknjaz opened this Issue Jul 31, 2017 · 24 comments

Comments

Projects
None yet
3 participants
@webknjaz
Copy link
Member

webknjaz commented Jul 31, 2017

This is needed to fully take advantage of py.test testing framework.

@webknjaz webknjaz self-assigned this Jul 31, 2017

@webknjaz

This comment has been minimized.

@jaraco

This comment has been minimized.

Copy link
Member

jaraco commented Jul 31, 2017

Fully support.

@jaraco

This comment has been minimized.

Copy link
Member

jaraco commented Nov 21, 2017

One consideration is some environments may be relying on WebCase as a unittest.TestCase subclass. I suggest separating the web testing functionality into a WebTest class that can be instantiated without any unittest functionality, but also use it as a mix-in for WebCase to continue to provide that functionality in the WebCase.

webknjaz added a commit that referenced this issue Dec 2, 2017

@webknjaz

This comment has been minimized.

Copy link
Member

webknjaz commented Dec 2, 2017

I've dropped a few classes in #67

webknjaz added a commit that referenced this issue Dec 14, 2017

webknjaz added a commit that referenced this issue Dec 14, 2017

webknjaz added a commit that referenced this issue Dec 22, 2017

@stale

This comment has been minimized.

Copy link

stale bot commented Jan 31, 2018

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jan 31, 2018

@webknjaz

This comment has been minimized.

Copy link
Member

webknjaz commented Feb 3, 2018

Next thing to achieve on this is to deprecate webtest and try to make some public test fixture/helper module

@stale stale bot removed the stale label Feb 3, 2018

@jaraco

This comment has been minimized.

Copy link
Member

jaraco commented Feb 5, 2018

In my experience - CherryPy's webtest is helpful, but it also masks errors (because every request happens on a remote side of a call, it's impossible to debug with something like pdb). I've found instead the webtest project and more interestingly pyriform, though admittedly I haven't used the latter yet.

@webknjaz

This comment has been minimized.

Copy link
Member

webknjaz commented Feb 5, 2018

I was also thinking of pylons.webtest and officially recommending to defer WSGI testing to third party, but didn't explore the possibilities and effect of such decision.

@webknjaz

This comment has been minimized.

Copy link
Member

webknjaz commented Feb 5, 2018

So I looked through pyriform and it looks simple and easy to use. There's no pytest fixtures shipped though, which might be useful. On the other side, we could provide our own fixtures based on this library. I foresee that it would fit in our ecosystem :)

@jaraco

This comment has been minimized.

Copy link
Member

jaraco commented Feb 10, 2018

I don't think there is anything called pylons.webtest and when I find Pylons/webtest, it seems to be one and the same project as the one I referenced.

And pyriform builds on that same framework.

I know the developer of pyriform and I bet he'd accept pull requests for pytest fixtures, so something definitely worth considering contributing upstream.

@stale

This comment has been minimized.

Copy link

stale bot commented Apr 11, 2018

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Apr 11, 2018

@stale stale bot closed this Apr 18, 2018

@webknjaz

This comment has been minimized.

Copy link
Member

webknjaz commented Apr 18, 2018

unstale

@webknjaz webknjaz reopened this Apr 18, 2018

@stale stale bot removed the stale label Apr 18, 2018

@stale

This comment has been minimized.

Copy link

stale bot commented Jun 17, 2018

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jun 17, 2018

@webknjaz

This comment has been minimized.

Copy link
Member

webknjaz commented Jun 18, 2018

unstale

@stale stale bot removed the stale label Jun 18, 2018

@stale

This comment has been minimized.

Copy link

stale bot commented Aug 19, 2018

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Aug 19, 2018

@webknjaz webknjaz removed the stale label Aug 19, 2018

@jeffvanvoorst

This comment has been minimized.

Copy link

jeffvanvoorst commented Sep 3, 2018

I would like to tackle this. I am familiar with the requests, pytest, and unittest modules.

I did see that pyriform links the requests and WebTest modules. Is it OK to have all 3 modules and dependencies as requirements for unit tests?

My reason for asking is that I was reading a CherryPy PR with a comment that "we probably don't want to introduce this dependency right now". cherrypy/cherrypy#1715.

@webknjaz

This comment has been minimized.

Copy link
Member

webknjaz commented Sep 3, 2018

@jeffvanvoorst it'd be nice to get some help!

I've done some refactoring introducing some replacements for server creation and client generation as fixtures (as opposed to https://github.com/cherrypy/cheroot/blob/master/cheroot/test/helper.py#L35 and https://github.com/cherrypy/cheroot/blob/master/cheroot/test/webtest.py#L99, which we're trying to make easily deprecatable in future).
Also, I've tried introducing some SSL tests (test_ssl.py), which temporarily use requests directly + there's a contribution from @dpeschman (test_https.py) where he modified that old client to support SSL client cert and introduced ddt dependency (pytest.mark.parametrized is not compatible with old helpers)

So we want to:

  • remove unittest completely
  • remove ddt in favor of pytest.mark.parametrize, but it doesn't work with classes inherited from unittest.TestCase
  • stop using old helper testcase class bases
  • start using pyriform directly, it's okay to use it, but not put requests/webtest as a direct dependency
  • stop using requests directly (in favor of pyriform) for testing WSGI
  • create a better framework for testing (as a replacement of old helpers), I've got some helpers in cheroot.testing and cheroot.test.conftest, and fixtures in test modules themselves, but there's a load of copy-paste, which I desire to reduce and make more elegant

N.B. I don't want to use pure requests here, because we implement HTTP parser and low-level protocol, so we need to test (1) malicious requests, (2) tune some low-level stuff for testing, which requests being a very high-level tool don't quite facilitate.

@jeffvanvoorst

This comment has been minimized.

Copy link

jeffvanvoorst commented Sep 4, 2018

I am thinking that given my time, it would be best to tackle one thing at a time. For example, it seems reasonable to try to remove unittest from WebCase as a first step. Then can continue to complete items.

@webknjaz

This comment has been minimized.

Copy link
Member

webknjaz commented Sep 4, 2018

@jeffvanvoorst do it! it comes from https://github.com/cherrypy/cheroot/pull/52/files#r213702146
I think it might be possible to also use trustme for certificate generation and setting up CA into SSLContext in that https module.

@jeffvanvoorst

This comment has been minimized.

Copy link

jeffvanvoorst commented Sep 6, 2018

@webknjaz it is taking me some time to wrap my head around the pieces. My plan keeps morphing. I think that I finally grokked how to use pyriform -- pass in the actual webserver instead of wsgi app. I will try some PoCs when I get time. After that, I should be able to more quickly code up a pytest implementation (leveraging pyriform) to test certificate handling. If nothing else, pyriform knowledge seems to be useful -- it would be great to unit/component test some custom requests code + wsgi app for a different project.

@webknjaz

This comment has been minimized.

Copy link
Member

webknjaz commented Sep 6, 2018

Yeah, I see... Maybe it's easier to start with refactoring other test modules, which already use pytest, to create a foundation for further refactoring?

@stale

This comment has been minimized.

Copy link

stale bot commented Nov 5, 2018

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Nov 5, 2018

@webknjaz

This comment has been minimized.

Copy link
Member

webknjaz commented Nov 5, 2018

unstale

@stale stale bot removed the stale label Nov 5, 2018

@webknjaz webknjaz referenced this issue Jan 1, 2019

Open

[TODO] Design a good cheroot.testing API #161

1 of 3 tasks complete
@webknjaz

This comment has been minimized.

Copy link
Member

webknjaz commented Jan 1, 2019

This is complete, I've moved unrelated items to #161.

@webknjaz webknjaz closed this Jan 1, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment