New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable testing with MySQL and Postgres on Travis-CI #171

Open
wants to merge 6 commits into
base: develop
from

Conversation

Projects
None yet
3 participants
@atodorov
Contributor

atodorov commented Mar 10, 2017

This PR enables MySQL and Postgres on Travis. I have a support ticket opened with their team to figure out how to add MariaDB alongside MySQL.

The build will fail b/c there are some issue with both MySQL and Postgres. I will try to fix them in subsequent commits.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 10, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling c6c2794 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 10, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling c6c2794 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 12, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling f486a3d on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 12, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling f486a3d on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 12, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling 297333c on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 12, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling 297333c on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 12, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling b9e8ab6 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 12, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling b9e8ab6 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 12, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling b9e8ab6 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 12, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling b9e8ab6 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 12, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling cefa25a on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 12, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling cefa25a on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 12, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling 6f9fdcd on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 12, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling 6f9fdcd on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 12, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling e34804a on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 12, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling e34804a on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 12, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling 1c2d2fd on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 12, 2017

Coverage Status

Coverage increased (+0.01%) to 45.189% when pulling 1c2d2fd on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 12, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling bad6c4c on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 12, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling bad6c4c on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 12, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling bad6c4c on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 12, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling bad6c4c on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 12, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 482f977 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 12, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 482f977 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 12, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 482f977 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 12, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 482f977 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 12, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 482f977 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 12, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 482f977 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 12, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 443c5bb on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 12, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 443c5bb on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 12, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 443c5bb on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 12, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 443c5bb on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 13, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling a5920d7 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 13, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling a5920d7 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 13, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 414963e on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 13, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 414963e on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 13, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 288c9fc on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 13, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 288c9fc on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 13, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 288c9fc on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 13, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 288c9fc on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 13, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 11b4fb9 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 13, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 11b4fb9 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 13, 2017

Coverage Status

Coverage decreased (-0.003%) to 45.175% when pulling bf0b9d8 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 13, 2017

Coverage Status

Coverage decreased (-0.003%) to 45.175% when pulling bf0b9d8 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 13, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 492669d on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 13, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 492669d on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 13, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 492669d on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 13, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 492669d on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@atodorov

This comment has been minimized.

Show comment
Hide comment
@atodorov

atodorov Mar 13, 2017

Contributor

@tkdchen sorry for the noise here. I've managed to fix the failing tests on MySQL but I'm having troubles configuring MariaDB to use utf8 instead of latin1. I have opened a support ticket for that.

Contributor

atodorov commented Mar 13, 2017

@tkdchen sorry for the noise here. I've managed to fix the failing tests on MySQL but I'm having troubles configuring MariaDB to use utf8 instead of latin1. I have opened a support ticket for that.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 13, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 6822ea6 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

coveralls commented Mar 13, 2017

Coverage Status

Coverage increased (+0.03%) to 45.209% when pulling 6822ea6 on MrSenko:travisci_more_db_testing into ae3745a on Nitrate:develop.

@tkdchen

This comment has been minimized.

Show comment
Hide comment
@tkdchen

tkdchen Mar 13, 2017

Member

It's okay. Thanks for making this improvement. Looking forward to the success.

Member

tkdchen commented Mar 13, 2017

It's okay. Thanks for making this improvement. Looking forward to the success.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 17, 2017

Coverage Status

Coverage increased (+0.05%) to 45.448% when pulling ef493e2 on MrSenko:travisci_more_db_testing into 980b07b on Nitrate:develop.

coveralls commented Mar 17, 2017

Coverage Status

Coverage increased (+0.05%) to 45.448% when pulling ef493e2 on MrSenko:travisci_more_db_testing into 980b07b on Nitrate:develop.

@atodorov

This comment has been minimized.

Show comment
Hide comment
@atodorov

atodorov Mar 17, 2017

Contributor

@tkdchen this is ready for merge.

I have update the Travis CI config to use their Ubuntu Trusty infrastructure b/c I was having lots of problems with MariaDB on their older infrastructure.

I have also updated some of the tests, mostly to avoid duplicate objects so they now PASS.

Note1: I have cherry-picked #157 here and later modified it to special case the migration for PostgreSQL. This turned out to be a bug in Django 1.8, which is documented and already fixed (reference in comments and commit log) but not backported. Also note that we can't use RunSQL for SQLite here b/c the necessary syntax isn't supported. Django ORM goes behind the scenes and provides the migration for us (add column, copy data, drop column, rename, etc).

Note 2: Currently tests for PostgreSQL fail. At least one of them is related to my other PRs where I refactor the SQL. My vote is to merge this PR as-is and I will work to add more tests to my other PRs and work to fix the PostgreSQL failures there.

Contributor

atodorov commented Mar 17, 2017

@tkdchen this is ready for merge.

I have update the Travis CI config to use their Ubuntu Trusty infrastructure b/c I was having lots of problems with MariaDB on their older infrastructure.

I have also updated some of the tests, mostly to avoid duplicate objects so they now PASS.

Note1: I have cherry-picked #157 here and later modified it to special case the migration for PostgreSQL. This turned out to be a bug in Django 1.8, which is documented and already fixed (reference in comments and commit log) but not backported. Also note that we can't use RunSQL for SQLite here b/c the necessary syntax isn't supported. Django ORM goes behind the scenes and provides the migration for us (add column, copy data, drop column, rename, etc).

Note 2: Currently tests for PostgreSQL fail. At least one of them is related to my other PRs where I refactor the SQL. My vote is to merge this PR as-is and I will work to add more tests to my other PRs and work to fix the PostgreSQL failures there.

@atodorov

This comment has been minimized.

Show comment
Hide comment
@atodorov

atodorov Mar 21, 2017

Contributor

@tkdchen ping? Any comment on this PR. I can't proceed further without feedback.

Contributor

atodorov commented Mar 21, 2017

@tkdchen ping? Any comment on this PR. I can't proceed further without feedback.

@tkdchen

This comment has been minimized.

Show comment
Hide comment
@tkdchen

tkdchen Mar 27, 2017

Member

@atodorov Hi, I'm just back from vocation. I'll be back to review PRs later.

Member

tkdchen commented Mar 27, 2017

@atodorov Hi, I'm just back from vocation. I'll be back to review PRs later.

@atodorov

This comment has been minimized.

Show comment
Hide comment
@atodorov

atodorov Apr 9, 2017

Contributor

Depends on #157

Contributor

atodorov commented Apr 9, 2017

Depends on #157

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Apr 11, 2017

Coverage Status

Coverage increased (+0.03%) to 45.41% when pulling e777204 on MrSenko:travisci_more_db_testing into 8f45beb on Nitrate:develop.

coveralls commented Apr 11, 2017

Coverage Status

Coverage increased (+0.03%) to 45.41% when pulling e777204 on MrSenko:travisci_more_db_testing into 8f45beb on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Apr 11, 2017

Coverage Status

Coverage increased (+0.03%) to 45.41% when pulling e777204 on MrSenko:travisci_more_db_testing into 8f45beb on Nitrate:develop.

coveralls commented Apr 11, 2017

Coverage Status

Coverage increased (+0.03%) to 45.41% when pulling e777204 on MrSenko:travisci_more_db_testing into 8f45beb on Nitrate:develop.

@atodorov

This comment has been minimized.

Show comment
Hide comment
@atodorov

atodorov Apr 11, 2017

Contributor

@tkdchen I think this is ready to merge so we can continue forward. I hate to merge with failing Postgres but MySQL, MariaDB and sqlite all work fine and some of the fixes are covered into other PRs. I don't think we have a lot of other options here otherwise this PR will become very big and there are other problems to worry about too.

Contributor

atodorov commented Apr 11, 2017

@tkdchen I think this is ready to merge so we can continue forward. I hate to merge with failing Postgres but MySQL, MariaDB and sqlite all work fine and some of the fixes are covered into other PRs. I don't think we have a lot of other options here otherwise this PR will become very big and there are other problems to worry about too.

@tkdchen

This comment has been minimized.

Show comment
Hide comment
@tkdchen

tkdchen Apr 12, 2017

Member

This is a big step forward to the testing. However, the problem is there is dependency of Travis in Nitrate source code, and this testing matrix cannot run locally by developer. I prefer to a solution to avoid these two major disadvantages.

Basically, the testing matrix should

  • allow developer to run tests locally without any other external services required.
  • allow developer to run tests by selecting specific database, SQLite, MySQL, MariaDB or PostgreSQL, or even all of them at once whatever in parallel or in sequence.
  • by default, tests could be run with SQLite, which is a much fast way to run tests and verify something quickly.

The second ability should be able to be used in Travis-CI to run the testing matrix.

In my mind, docker would be a good tool to help us to build this testing matrix, and obviously some scripts would be required.

Currently, to run whole test suite, I can simple run make test in a well-provisioned vagrant machine. However, running test suite from scratch should be an easy thing now in Nitrate as following steps,

  1. provision machine, installing necessary packages required in next steps
  2. make a virtual environment
  3. install dependencies from requirements/devel.txt
  4. make test

It should be easy to make these steps happen in a docker container. Fedora 25 docker image can be used and it is ok to use the version of MySQL, MariaDB and PostgreSQL in Fedora repo.

Finally, there could be these ways to run tests matrix

  • make test, run with SQLite by default
  • DB=[dbname] make test, dbname could be mysql, mariadb, or pgsql
  • DB=all make test, run tests with mysql, mariadb, and pgsql individually

Thank you for making this improvement. It's valuable and develop our ideas. Let's make it better and better.

I'll look at other commits in this PR from tomorrow. Have a good day. :)

Member

tkdchen commented Apr 12, 2017

This is a big step forward to the testing. However, the problem is there is dependency of Travis in Nitrate source code, and this testing matrix cannot run locally by developer. I prefer to a solution to avoid these two major disadvantages.

Basically, the testing matrix should

  • allow developer to run tests locally without any other external services required.
  • allow developer to run tests by selecting specific database, SQLite, MySQL, MariaDB or PostgreSQL, or even all of them at once whatever in parallel or in sequence.
  • by default, tests could be run with SQLite, which is a much fast way to run tests and verify something quickly.

The second ability should be able to be used in Travis-CI to run the testing matrix.

In my mind, docker would be a good tool to help us to build this testing matrix, and obviously some scripts would be required.

Currently, to run whole test suite, I can simple run make test in a well-provisioned vagrant machine. However, running test suite from scratch should be an easy thing now in Nitrate as following steps,

  1. provision machine, installing necessary packages required in next steps
  2. make a virtual environment
  3. install dependencies from requirements/devel.txt
  4. make test

It should be easy to make these steps happen in a docker container. Fedora 25 docker image can be used and it is ok to use the version of MySQL, MariaDB and PostgreSQL in Fedora repo.

Finally, there could be these ways to run tests matrix

  • make test, run with SQLite by default
  • DB=[dbname] make test, dbname could be mysql, mariadb, or pgsql
  • DB=all make test, run tests with mysql, mariadb, and pgsql individually

Thank you for making this improvement. It's valuable and develop our ideas. Let's make it better and better.

I'll look at other commits in this PR from tomorrow. Have a good day. :)

@atodorov

This comment has been minimized.

Show comment
Hide comment
@atodorov

atodorov Apr 12, 2017

Contributor
make test, run with SQLite by default
DB=[dbname] make test, dbname could be mysql, mariadb, or pgsql
DB=all make test, run tests with mysql, mariadb, and pgsql individually

I think this can easily be done in the settings and then simply ask Travis to execute whichever command fits its matrix. In this way we move the infrastructure dependency in .travis.yml and not in the project source code.

I will rework the settings for that. It is up to the developer to prepare their local environment for whichever DB they want to test on. I will also update the docs with commands how to install dependencies and run the tests against different databases.

Note: vagrant or docker will be a great help but that's up to each developer to decide how to use. IMO this needs to come at a later stage because this PR is quite big already. What do you think?

Contributor

atodorov commented Apr 12, 2017

make test, run with SQLite by default
DB=[dbname] make test, dbname could be mysql, mariadb, or pgsql
DB=all make test, run tests with mysql, mariadb, and pgsql individually

I think this can easily be done in the settings and then simply ask Travis to execute whichever command fits its matrix. In this way we move the infrastructure dependency in .travis.yml and not in the project source code.

I will rework the settings for that. It is up to the developer to prepare their local environment for whichever DB they want to test on. I will also update the docs with commands how to install dependencies and run the tests against different databases.

Note: vagrant or docker will be a great help but that's up to each developer to decide how to use. IMO this needs to come at a later stage because this PR is quite big already. What do you think?

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Apr 13, 2017

Coverage Status

Coverage increased (+0.05%) to 45.425% when pulling bd74f39 on MrSenko:travisci_more_db_testing into 8f45beb on Nitrate:develop.

coveralls commented Apr 13, 2017

Coverage Status

Coverage increased (+0.05%) to 45.425% when pulling bd74f39 on MrSenko:travisci_more_db_testing into 8f45beb on Nitrate:develop.

@atodorov

This comment has been minimized.

Show comment
Hide comment
@atodorov

atodorov Apr 13, 2017

Contributor

@tkdchen see the updated documentation and the last 3 commits for the latest changes. It is not possible to execute the tests on the local machine by specifying which database to use. The same commands are integrated into Travis as well. All of the integration happens in .travis.yml.

Contributor

atodorov commented Apr 13, 2017

@tkdchen see the updated documentation and the last 3 commits for the latest changes. It is not possible to execute the tests on the local machine by specifying which database to use. The same commands are integrated into Travis as well. All of the integration happens in .travis.yml.

@tkdchen

This comment has been minimized.

Show comment
Hide comment
@tkdchen

tkdchen Apr 14, 2017

Member

Using Travis to run tests does not mean there is a dependency of Travis to run tests. Simply say I don't want this line of code appears in source code

if 'TRAVIS' in os.environ:

Documentation is good but not always. For running tests, I prefer to make it as easy as possible with just one command, no need of steps

  1. find the documentation
  2. read through the documentation to learn the steps
  3. try each step to provision environment and launch tests
  4. repeat again and again if something is wrong by misoperation

Instead, as mentioned in my previous comment, just make test or DB=pgsql make test is enough. Easy and save much time, isn't it?

If someone is interested in the steps to build testing environment, just open and read the relative scripts. That should be more straightforward than lots of words in documentation and easy understand.

Member

tkdchen commented Apr 14, 2017

Using Travis to run tests does not mean there is a dependency of Travis to run tests. Simply say I don't want this line of code appears in source code

if 'TRAVIS' in os.environ:

Documentation is good but not always. For running tests, I prefer to make it as easy as possible with just one command, no need of steps

  1. find the documentation
  2. read through the documentation to learn the steps
  3. try each step to provision environment and launch tests
  4. repeat again and again if something is wrong by misoperation

Instead, as mentioned in my previous comment, just make test or DB=pgsql make test is enough. Easy and save much time, isn't it?

If someone is interested in the steps to build testing environment, just open and read the relative scripts. That should be more straightforward than lots of words in documentation and easy understand.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Apr 14, 2017

Coverage Status

Coverage increased (+0.03%) to 45.411% when pulling 61355c2 on MrSenko:travisci_more_db_testing into 8f45beb on Nitrate:develop.

coveralls commented Apr 14, 2017

Coverage Status

Coverage increased (+0.03%) to 45.411% when pulling 61355c2 on MrSenko:travisci_more_db_testing into 8f45beb on Nitrate:develop.

@atodorov

This comment has been minimized.

Show comment
Hide comment
@atodorov

atodorov Apr 14, 2017

Contributor

if 'TRAVIS' in os.environ:

Mistake, this has now been updated. The rest (make test commands) are as requested and described in more details in the docs. Anything else missing ?

Contributor

atodorov commented Apr 14, 2017

if 'TRAVIS' in os.environ:

Mistake, this has now been updated. The rest (make test commands) are as requested and described in more details in the docs. Anything else missing ?

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Apr 14, 2017

Coverage Status

Coverage increased (+0.03%) to 45.406% when pulling 8b42d6f on MrSenko:travisci_more_db_testing into 8f45beb on Nitrate:develop.

coveralls commented Apr 14, 2017

Coverage Status

Coverage increased (+0.03%) to 45.406% when pulling 8b42d6f on MrSenko:travisci_more_db_testing into 8f45beb on Nitrate:develop.

@tkdchen

This comment has been minimized.

Show comment
Hide comment
@tkdchen

tkdchen Apr 15, 2017

Member

@atodorov We talked a lot in IRC channel #nitrate. You did a good job to make it possible to run tests on different database engines. Hopefully, you could continue improving based on the changes in this PR to make running tests more easily.

I'd like to repeat my thought again

  1. Do not depend on Travis-CI to run tests.
  2. To achieve the first goal, if those database provision steps in .travis.yml are necessary, they should happen locally, not in Travis-CI.
  3. As a developer, I can run tests simply in following steps without the need of reading whole documentation before that.
    1. clone Nitrate
    2. DB=mysql make test

Detailed documentation in this PR is good, I think it could be as reference for developers who are interested in hacking the tests for themselves.

Is this clear and make sense?

Member

tkdchen commented Apr 15, 2017

@atodorov We talked a lot in IRC channel #nitrate. You did a good job to make it possible to run tests on different database engines. Hopefully, you could continue improving based on the changes in this PR to make running tests more easily.

I'd like to repeat my thought again

  1. Do not depend on Travis-CI to run tests.
  2. To achieve the first goal, if those database provision steps in .travis.yml are necessary, they should happen locally, not in Travis-CI.
  3. As a developer, I can run tests simply in following steps without the need of reading whole documentation before that.
    1. clone Nitrate
    2. DB=mysql make test

Detailed documentation in this PR is good, I think it could be as reference for developers who are interested in hacking the tests for themselves.

Is this clear and make sense?

@tkdchen

This comment has been minimized.

Show comment
Hide comment
@tkdchen

tkdchen Apr 15, 2017

Member

I merged these 3 commits

a1c47ec Compare PKs with what was written in DB, not hard-coded
fc2343c Update pre_check_product() and pre_process_ids() to include long
7d10b49 Don't use unique test object names

Member

tkdchen commented Apr 15, 2017

I merged these 3 commits

a1c47ec Compare PKs with what was written in DB, not hard-coded
fc2343c Update pre_check_product() and pre_process_ids() to include long
7d10b49 Don't use unique test object names

Show outdated Hide outdated tcms/testruns/tests.py
@@ -37,7 +36,7 @@ def setUpTestData(cls):
cls.client = test.Client()
def test_404_if_run_does_not_exist(self):
nonexisting_run_pk = TestRun.objects.count() + 1
nonexisting_run_pk = 999999

This comment has been minimized.

@tkdchen

tkdchen Apr 15, 2017

Member

It would be helpful to understand if you can add explanation to this change. Then, I'll merge it.

@tkdchen

tkdchen Apr 15, 2017

Member

It would be helpful to understand if you can add explanation to this change. Then, I'll merge it.

This comment has been minimized.

@atodorov

atodorov Apr 15, 2017

Contributor

OK, for some reason ( I think because PKs are global autoincrements on MySQL, generated via LAST_INSERT_ID())

TestRun.objects.count() == 1
nonexisting_run_pk == 2

but the only TestRun object in the database has an actual ID of 2 hence the failure!

On other DB engines the only TestRun object has an ID of 1. I will update this code with
TestRun.objects.last().pk + 1 which will be more robust.

However I have found one more thing. In testruns/urls/run_urls.py there are two URLs which use the same view:
.../ordercase/$ and .../ordercaserun/$. Do you know why is that? Sounds like a bug (let me know and I will send a separate PR for it).

@atodorov

atodorov Apr 15, 2017

Contributor

OK, for some reason ( I think because PKs are global autoincrements on MySQL, generated via LAST_INSERT_ID())

TestRun.objects.count() == 1
nonexisting_run_pk == 2

but the only TestRun object in the database has an actual ID of 2 hence the failure!

On other DB engines the only TestRun object has an ID of 1. I will update this code with
TestRun.objects.last().pk + 1 which will be more robust.

However I have found one more thing. In testruns/urls/run_urls.py there are two URLs which use the same view:
.../ordercase/$ and .../ordercaserun/$. Do you know why is that? Sounds like a bug (let me know and I will send a separate PR for it).

atodorov added some commits Mar 10, 2017

Enable testing with MySQL and Postgres on Travis-CI
Fix #169

Signed-off-by: Mr. Senko <atodorov@mrsenko.com>
Configure MariaDB for utf8 and move to Ubuntu Trusty
- the older Travis CI images have problems with MariaDB not picking
  up its configuration
- also manually create DB and test DB and grant privileges to travis
  b/c connection will fail otherwise.

Signed-off-by: Mr. Senko <atodorov@mrsenko.com>
Fix test failures on MySQL for non-existing PKs
On MySQL the default AutoField() translates to AUTO_INCREMENT
which is shared between all tables!

Signed-off-by: Mr. Senko <atodorov@mrsenko.com>
Create separate files for DB testing
testing is now possible via the following commands:

	make test (uses SQlite)
	TEST_DB=all make test (will test on all DBs)
	TEST_DB=MySQL make test
	TEST_DB=MariaDB make test
	TEST_DB=Postgres make test

In addition when preparing the local virtualenv we now have DB
specific requirements files:

	requirements/mysql.txt (for MySQL and MariaDB)
	requirements/postgres.txt (for Postgres)
	requirements/base.txt (for SQLite)
	requirements/devel.txt (additional development libraries)

Signed-off-by: Mr. Senko <atodorov@mrsenko.com>
Handle imports if not running on MySQL
Signed-off-by: Mr. Senko <atodorov@mrsenko.com>
Update documentation to reflect new test commands
also updates the usage of new requirements files

Signed-off-by: Mr. Senko <atodorov@mrsenko.com>
@atodorov

This comment has been minimized.

Show comment
Hide comment
@atodorov

atodorov Apr 15, 2017

Contributor

Updated and rebased.

@tkdchen let's move this comment #171 (comment) into a separate issue. I understand the value proposition of it and it is definitely doable even in the short term. However not everyone would like to follow exactly the same steps so we shouldn't force it. Plus a separate issue will allow us to discuss into more details. What do you think?

Contributor

atodorov commented Apr 15, 2017

Updated and rebased.

@tkdchen let's move this comment #171 (comment) into a separate issue. I understand the value proposition of it and it is definitely doable even in the short term. However not everyone would like to follow exactly the same steps so we shouldn't force it. Plus a separate issue will allow us to discuss into more details. What do you think?

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Apr 15, 2017

Coverage Status

Coverage increased (+0.04%) to 45.406% when pulling fa98cb2 on MrSenko:travisci_more_db_testing into a1c47ec on Nitrate:develop.

coveralls commented Apr 15, 2017

Coverage Status

Coverage increased (+0.04%) to 45.406% when pulling fa98cb2 on MrSenko:travisci_more_db_testing into a1c47ec on Nitrate:develop.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Apr 15, 2017

Coverage Status

Coverage increased (+0.04%) to 45.406% when pulling fa98cb2 on MrSenko:travisci_more_db_testing into a1c47ec on Nitrate:develop.

coveralls commented Apr 15, 2017

Coverage Status

Coverage increased (+0.04%) to 45.406% when pulling fa98cb2 on MrSenko:travisci_more_db_testing into a1c47ec on Nitrate:develop.

@atodorov atodorov referenced this pull request Apr 26, 2017

Closed

Fix Issue #181 #182

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment