Enable all tests on Travis and document the needed options #165

Closed
wants to merge 6 commits into
from

Projects

None yet

4 participants

@rombert
Mantis Bug Tracker member

Please note that this is yet untested, since:

  • I can't run publican ( but maybe it's a good idea to add a publican validate step on Travis )
  • I can't validate that the before_script is correct outside of Travis
rombert added some commits Apr 17, 2014
@rombert rombert Document the needed configuration options to enable all SOAP tests
Affects #17229: Ensure that all tests are executed on Travis
8556ebd
@rombert rombert Ensure that all tests are executed on Travis CI
Fixes #17229: Ensure that all tests are executed on Travis
4aa3204
@rombert rombert Follow-up fix to make before_script work on Travis ( will be squashed )
Fixes #17229: Ensure that all tests are executed on Travis
4e3e6d6
@rombert rombert Follow-up fix to make before_script work on Travis part 2 ( will be s…
…quashed )

Fixes #17229: Ensure that all tests are executed on Travis
304a317
@rombert rombert Add missing trailing comma to code listing ( will be squashed )
Fixes #17229: Ensure that all tests are executed on Travis
0e751d0
@grangeway

-1 = As our unit tests should try and cover as much of mantis as possible (hence my question the other day of what the required config was at the moment, and the pull request on the config changes so it's easier to change configs within a test)

My purpose of asking (initially) what tests options were covered was to save time getting the current test suite to pass.

@rombert
Mantis Bug Tracker member

@grangeway - I don't see an either/or between this PR and #164 . Also, I don't see how #164 allows me to configure MantisBT from the SOAP API, which is what these tests run against.

Please reconsider your -1 , or at least let's get the conversation flowing a bit more :-)

@grangeway

The purpose of #164 is to allow a unit test (against mantis) to define configs - i'll admit that testing 'mantis' vs testing 'soap' is a bit different - but we need to be able to test the soap api for isntance with the backend having both documentation ON + OFF

@vboctor
Mantis Bug Tracker member

Is there a way to make the config overrides happen as part of running a set of test cases? For example, imagine there is a config_inc.php copy that is standard, then a text fixture can override that and at the end revert to the standard. This way multiple fixtures can apply their overrides and revert when done.

In such case, we shouldn't need to document the settings. Since even if a developer happens to run the test cases locally, they would apply the overrides at the beginning of the fixture. Otherwise, documentation is likely to become stale.

@grangeway

Victor, this was the purpose of submitting #164 - as I'd like to be able to do this. And also hence why I originally said -1 to doing this.

@rombert
Mantis Bug Tracker member

What I might be able to do is - add a new parameter to bootstrap.php which controls the location of the config_inc.php file. If this parameter is defined, the SOAP ( not unit ) tests will attempt to set configuration options as needed in that file. It's not completely straightforward ( needs PHP code parsing and must be careful about cleaning up ) but I think that's the only way to make this completely automatic.

The changes would be appended at the end of the file and marked for cleanup, for easier testing, e.g.

$g_allow_no_category = ON; // PHPUnit-autoclean

Thoughts?

@rombert
Mantis Bug Tracker member

@grangeway - but this still doesn't work for the SOAP tests, as they don't run in the same process as the code they're testing. So #164 simply isn't related to this PR.

@grangeway
@vboctor
Mantis Bug Tracker member

Given that this is not an extensibility, db schema or end user feature, I'm OK with getting in a solution and then improving it later (in a week, month or year - doesn't really matter). That is unless of course, @grangeway ends up getting the go ahead to get his change in and it would resolve the same issue.

Is it possible to set an environment variable that points to a config file for the test fixture? For example, we already have MANTIS_CONFIG now and with #163 we'll have MANTIS_CONFIG_FOLDER.

@grangeway
@rombert rombert Revert "Document the needed configuration options to enable all SOAP …
…tests"

This reverts commit 8556ebd.

Conflicts:
	docbook/Developers_Guide/en-US/Contributors.xml
e72f8fc
@rombert
Mantis Bug Tracker member

So how about this one? Just enable all tests, to make sure we don't have any regressions.

@dregad
Mantis Bug Tracker member

I think it's good to merge this change in so that the Travis builds at least cover all cases by default.

Could you rebase your branch on top of latest master to ensure the build works ? You might also have to fix the current error, maybe you need to set MANTIS_CONFIG=config/config_inc.php

@dregad dregad added a commit that referenced this pull request Jun 23, 2014
@dregad dregad Enable all tests on Travis
Fixes #17229, #165
e1ab941
@dregad
Mantis Bug Tracker member

Thanks @rombert for this.

It's now fixed in both master and 1.2.x (sorry about the mess in the latter, I inadvertently pushed to the wrong remote while testing)

@dregad dregad closed this Jun 23, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment