New Release / Project Status

[still-dreaming-1]

I would like to inquire about the status of simpletest. The last release was in October of 2015, so getting close to 2 years ago now. I'm guessing there are 4 possibilities as to why this is. 1) The code has been unfit for release since then. 2) There have not been significant enough changes to warrant a release. 3) The project is not actively maintained enough for someone to prepare a release even though they are believed in by the maintainer. 4) You don't believe in releases and always consider the master branch to be the latest release and fit for use.

[jakoch]

Hello Jesse!

I didn't tag the dev-master branch as a new release version, because it's not green.
I work on the project every once in a while. That's clearly not enough to stabilize it.

As developer i'm stuck on the problem of handling mock objects and references.

As maintainer i'm merging only smaller contributions, which aren't blocked by the need for refactoring. Merged contributions contained fixes and new features, which means the project is still in use somewhere.
But i received no contributions from the community working towards getting the master branch "green".

The project is on hiatus; waiting for my next commits as developer or merges by a maintainer.

Best, Jens

[still-dreaming-1]

Thanks for the info! Just out of curiosity, what is your personal interest in SimpleTest based on? What I mean is, of course PHPUnit is more popular and widely used. Is there something keeping you interested in SimpleTest instead of just going with what is popular? I'm not suggesting there should only be one option or that making alternatives is bad and everyone should go with what is popular. SimpleTest has served me well at the company I work for.

For us we used to use SimpleTest a long time ago because I found it easier to get reports viewable from a website and more flexible for running tests. Then I gave up on automated testing for a time. Then I got back into it and started trying to use PHPUnit. But PHPUnit seems to enforce PSR-4, as in it is completely impossible to run tests for code not organized into PSR-4, and our code is not quite there yet even though we are working toward it. Once all of our code honors PSR-4, we will have the option of going back to PHPUnit.

The following comments about SimpleTest refer to v1.1.7. From my perspective, as someone that has some experience, but is lacking much knowledge, PHPUnit and SimpleTest both have aspects that are better designed than the other. SimpleTest is less coupled to your architecture and easier to get working. PHPUnit seems better designed in terms of how you write custom assertions and get custom, helpful fail messages from them. Although recently SimpleTest did start mysteriously reporting better fail messages for our custom assertions... If I had to guess, the difference seems to be related to a slightly different way we are organizing the tests now. We are creating private assertion methods that start with the name "assert" that don't do anything special to assert things other than call other assertion methods. The rule we are following is a test method can only call one assert method, but an assert method can call as many assert methods as it needs to in order to complete that one conceptual assertion. Most of those assert methods are not intended to be super reusable, they are only reusable in that one context, from that one TestCase, which is why we leave them private.

Once we have all our code organized according to PSR-4, I may try to decouple the tests from a particular framework, and maintain compatibility with at least 2 frameworks in order to ensure the modularity stays real. In my opinion, in the long run, that kind of decoupling ends up giving you enough return on investment to make it worth it. We probably will want to try other frameworks at some point, or possibly have to translate or even transpile the tests to a completely different language possibly to run on a different environment.

[TrueType]

Whatever. The main difference to PHPUnit is probably the ability to run integration tests on the HTTP interface of your web application or backends. And fast enough to do TDD here.

[still-dreaming-1]

@TrueType interesting. Do you think it is faster than PHPUnit? Or are you just saying there is no reason to use PHPUnit because SimpleTest is fast enough?

[still-dreaming-1]

@TrueType Oh I think I understand now. Are you saying even when you do TDD through the HTTP interface the tests still run fast enough to do your development that way?

[jakoch]

(There we go again with another episode of the famous PHPUnit vs. SimpleTest discussion.)

One can not really compare SimpleTest with PHPUnit, because SimpleTest is a legacy system. It's used by legacy PHP applications, which are still out there and have to be maintained somehow. Technical debt.

My personal interest in SimpleTest is to keep that life-support-machine running. :)

I've learned a lot while working on it and with it. For instance, to keep an eye on BC compatibility issues, while slowly raising PHP versions = usage of feature flags and PHP version guards. I'm also glad that i had the chance to do a really large refactoring on one of the oldest PHP projects. And i'm still learning on how to implement mocking ¯\(ツ)/¯

For new projects you might simply use PHPUnit or any other testing library, e.g. Atoum, Storyteller, Peridot, Kahlan, Behat, phpSpec, phake, mockery, SimpleTest or a wrapper like CodeCeption.
Use whatever works for you, your team and your project context. Some like TDD, some BDD, some functional testing, some cli-based test runs, some old school web-based testing.
(I mean, if you need PHP7 support and a solid mocking layer right now, then the decision is clearly not SimpleTest, but PHPUnit+mockery, right?)

The main difference to PHPUnit is probably the ability to run integration tests...

A lot of developers find it very intuitive to run tests in the browser. You basically switch between your app/script tab and the test-runner tab. Here it's an out of the box feature, but such a runner exists for PHPUnit as an add-on, too.

Because everyone runs tests on the CLI nowadays (waiting for red/green flags coming from CI servers), the opposite direction and original idea (of Fitnesse and Slim) was never fully explored in the PHP world: from static web-based test runners over ajax runners to ajax+wiki based runners. https://vimeo.com/5861020

translate or even transpile the tests

There are also bridges/adapters available to run SimpleTest cases with PHPUnit and vice versa. This helps in the migration/transition phase from one test tool to the other.

Do you think it is faster than PHPUnit?

What comes to mind is autoloading.
ST might be faster, because there is no autoloading. We still include files directly.
On the other hand: with opcache enabled it doesn't matter.
And oh: we don't have a parallel test runner.

[still-dreaming-1]

Thanks! If you would be willing to answer just one more opinion question, it would totally make my week. If you were starting a new web project, and for some reason you chose PHP as the language, what testing tools do you think you would use if you only had a very small team of experienced, patient, and insightful developers?

[jakoch]

For unit- and group-testing (integration): PHPUnit.
For web- and acceptance testing: Selenium+ChromeHeadless or Behat+Mink or CodeCeption.
For static analysis: phpstan.
For code-style fixes: php-cs-fixer.