Added the Hitch integration testing framework.

[crdoconnor]

No description provided.

[osantana]

👍

[vinta]

@crdoconnor @osantana I'm not familiar with hitch, what are its strengths?

[crdoconnor]

It's a high level integration testing framework.

While unit testing frameworks like py.test are ideally suited to testing logical code (sorting lists, finding anagrams, matrix calculations, financial calculations), I find they're lacking for situations which involve integration code - especially high level end to end tests - e.g. testing a web app that talks to lots of different systems - databases, SMTP servers, message queues, etc. Hitch shines in this area.

Hitch provides the boilerplate to let you write clear, readable, reliable tests for these scenarios and minimize the problem of "brittle and slow" end to end tests. Some example features:

• Postgres plug in that lets you run your code with a specific version of postgresql (e.g. exact version you have in production) with a database environment that is isolated to your test (ever had one functional test affect how another behaved because of what the previous one did to the database? this is avoided in hitch). Download/set up of postgres is automated.
• Python plug in that lets you run your code in a specific version of python (or multiple versions of python). Again, download and set up of python is automated (system python is not used). Project specific virtualenv(s) for running your code in is/are is created for you with a couple of lines of code.
• Selenium plug in with built in step library to write clear, readable browser steps that can easily be configured to run with XVFB or without.
• Command line plug in with step library that lets you write steps to interact with command line applications.
• The test description language is simple, readable, declarative, YAML and templateable via jinja2 (kinda like ansible or salt).
• Adding step code for your test description language to call is easy and does not require you to write any regular expressions.
• Deep integration with IPython.
• Make your services "travel in time" by using libfaketime to lie to them about the system time.
• Everything is loosely coupled and most things except the core are plug-ins. The python plug in is so that you can test python code more easily. You don't have to use hitch to test python code, though.
• One step set up of testing environment - either for test driven development or continuous integration.
• Works on most major linux distros and Mac OS X without much, if any, additional configuration/changes (largely done via integration with another project I wrote: unixpackage).

[techtonik]

@crdoconnor looks like you are the author and this now looks like self-promotion. Who can prove your words that Hitch is awesome? Who is using it in production?

[crdoconnor]

Yeah I am and it is. I assumed that was ok. Nothing in CONTRIBUTING says otherwise.

It's still a pretty new thing and I haven't done much promotion yet so the number of users isn't enormous. No risk-averse big corporations that I know of yet, for instance.

It's integrated into cookiecutter-django and gets used via that though. I have customers who use it too. I assume at least some of the people who starred it on github are also using it.

[techtonik]

Promotion doesn't matter. You product doesn't have reviews and you're biased towards it, so it is hard to say if it is awesome. It may have an awesome ideas undercover, but buggy as hell on Python 3 like my libraries, for example.

[crdoconnor]

It was unstable when I issued the pull request 2 months ago and quality-wise it won't be production-hardened for probably another ~2-3 months. It's beta quality, currently. If that disqualifies it for inclusion, let me know. I'll re-open the PR once it's hardened.

I have a dogfooded regression test suite for the framework which I run regularly on Mac OS Yosemite, Ubuntu, Debian, Fedora, Arch and 3 different versions of python 3.

I'd recommend trying it to get a flavor of what it's capable of (and to ascertain its bugginess or lack thereof). You can do so by installing the bootstrapper::

$ sudo pip install hitch

And then check out and run the example project::

(Ensure brew, firefox and node-less are all pre-installed if using Mac OS X)

$ git clone --recursive https://github.com/hitchtest/django-remindme.git
$ cd django-remindme/django-remindme-tests

# This creates a python 3 virtualenv to run the testing code in and installs required packages:
$ hitch init

# First test run will take ~25 minutes. Runs the same scenario in python 2 and python 3:
$ hitch test simple_reminder.test

# Subsequent test run will take ~40 seconds:
$ hitch test simple_reminder.test

You can also try it out with cookiecutter-django, where it runs an example test::

$ cookiecutter https://github.com/crdoconnor/cookiecutter-django.git
$ cd my_project/tests
$ hitch init
$ hitch test .

If you can think of pythonistas whose opinion you'd trust who might be interested in trying out this new approach to testing, please let me know.

[techtonik]

I am on Windows, sorry.

[techtonik]

This list is not just a list of all ideas and frameworks that people came up with, because Python is awesome. This list is for tools that are awesome for Python. So far I don't see any formal proof that Hitch is awesome - it is marked as an alpha quality, doesn't have clearly expressed architecture depicted on diagram, not cross-platform, doesn't have any public users and public installations. Everything it has as a proof is a authors' opinion and prebaked demo video. I don't want to disappoint you - the Hitch may be awesome, but I don't have enough time to be your tester.

[techtonik]

And also AGPL is a bad license choice if your code contains Python libs that are meant to be imported for setting the harness.