This will allow running the tests when installing on gentoo linux for example.
For context @jkbr, other distributions (like debian and arch) prefer this too. You don't have to force them to run on install either, you can simply include them and when packaging for different distributions, the package maintainers can modify it to run them.
Most of the tests talk to htptbin.org, so they will fail while offline or when httpbin.org is down. Also, there are additional dependencies for running the tests. Maybe it could be changed that a local instance of httpbin is used instead, but what about the extra dependencies?
This is not a problem on gentoo. Tests are enabled using a useflag which the user could enable/disable. Dependencies are enabled conditionally for tests. This is already done for lot of packages, for example requests.
@jkbr you also have the advantage of being assured that httpbin.orgrarely changes. With that in mind, you could use something like Betamax so that you could develop offline. You could then ship the cassettes with the tests and they would run quickly.
If you specify your test dependencies in setup.py they will be downloaded but not installed globally so you won't augment the user's system.
Now the tests use a local httpbin instance via pytest-httpbin which makes running tests a bit easier. Are there any good examples of packages that ship with a test suite?