Skip to content
This repository has been archived by the owner on Aug 20, 2018. It is now read-only.

Bug 1019672 Add steps: pip, setup.py and running unit tests. #139

Merged
merged 8 commits into from
Jul 16, 2014
120 changes: 84 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,26 @@
oneanddone
==========

The "One and Done" initiative, previously known as "QA Taskboard", is a workflow
where Mozilla community contributors can pick tasks and work on them - one at a
time, one day at a time - and feel good about doing them.

One and Done is written with [Playdoh][playdoh] and [Django][django].

If you're interested in helping us out, please read through
[the blog post][blogpost] and reach out to us!
If you're interested in helping us out, please read through the
[project wiki][wiki] and reach out to us!

About the project:
>Contribute to Mozilla QA - One task at a time, One day at a time.
>
>One and Done gives users a wide variety of ways to contribute to Mozilla.
>You can pick an easy task that only takes a few minutes - or take on a
>bigger challenge. This includes working on manual testing, automation, bug
>verification, mobile testing and more. Tasks are from all QA teams - so you
>can get involved with Automation, Firefox OS, Desktop Firefox, Mozilla
>websites, Services, or Thunderbird.

[django]: http://www.djangoproject.com/
[playdoh]: https://github.com/mozilla/playdoh
[blogpost]: https://quality.mozilla.org/2013/10/qa-taskboard-development-call-for-participation/
[wiki]: https://wiki.mozilla.org/QA/OneandDone
[persona]: https://developer.mozilla.org/Persona/The_implementor_s_guide/Testing
[django-browserid]: https://github.com/mozilla/django-browserid


Development Setup
Expand All @@ -39,46 +47,73 @@ you don't have `pip` installed, you can install it with `easy_install pip`.
$ source venv/bin/activate
```

3. Install the compiled requirements:
3. Set up MySQL locally. The [MySQL Installation Documentation][mysql] explains how to do this.


4. Create the initial empty database; make sure it's utf8:
```
# Start the MySQL server
$ mysql.server start
# Once successfully started, log into the console
# using your username and password
$ mysql -uroot -p
```
In the mysql console:
```mysql
CREATE DATABASE oneanddone
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add the much needed semicolons.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's all one command, only one semi-colon is needed at the very end.

DEFAULT CHARACTER SET utf8
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Semicolon.

DEFAULT COLLATE utf8_general_ci;
```
To run all parts of the application, you will eventually need to populate this empty database with some example data, especially Tasks. There are many ways to populate the database. The method you choose may depend on the kind of data you want to add.
* Use the create/edit features of your local One and Done instance. For example sign in with an administrator account and go to the `/tasks/create/` URL of the app to create Tasks.
* Use the Django admin section of your local One and Done instance by going to the `/admin` URL -- this also relies on an admin account. You can define Task Teams here, for example.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My change still mentions the Django admin even though it cannot be used for Tasks. I'm trying to make it clear that it might be useful for other db modifications.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough.

* Use an external tool like MySQL Workbench.
* Ask another active developer for a mysqldump of their local database.
5. Install the compiled and development requirements:
```sh
$ pip install -r requirements/compiled.txt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried running through the instructions from scratch, to verify if they work and to check on that setup.py requirement, and I found that I get an error when trying to run the above command. I am on OS X 10.8.5. I know I was able to get this installed in the past (as I have it all running locally now), but this concerns me as I imagine others might run into the same issue. The message from pip looks like this:

~/gitRepos/tmp/oneanddone[master*]$ pip install -r requirements/compiled.txt
Downloading/unpacking MySQL-python==1.2.3c1 (from -r requirements/../vendor/src/funfactory/funfactory/requirements/compiled.txt (line 1))
  Downloading MySQL-python-1.2.3c1.tar.gz (89kB): 89kB downloaded
  Running setup.py (path:/Users/bsilverberg/.virtualenvs/oneanddone-pr139/build/MySQL-python/setup.py) egg_info for package MySQL-python

Downloading/unpacking Jinja2==2.5.5 (from -r requirements/../vendor/src/funfactory/funfactory/requirements/compiled.txt (line 2))
  Downloading Jinja2-2.5.5.tar.gz (438kB): 438kB downloaded
  Running setup.py (path:/Users/bsilverberg/.virtualenvs/oneanddone-pr139/build/Jinja2/setup.py) egg_info for package Jinja2

    warning: no previously-included files matching '*' found under directory 'docs/_build'
    warning: no previously-included files matching '*.pyc' found under directory 'jinja2'
    warning: no previously-included files matching '*.pyc' found under directory 'docs'
    warning: no previously-included files matching '*.pyo' found under directory 'jinja2'
    warning: no previously-included files matching '*.pyo' found under directory 'docs'
Downloading/unpacking py-bcrypt==0.3 (from -r requirements/../vendor/src/funfactory/funfactory/requirements/compiled.txt (line 5))
  Downloading py-bcrypt-0.3.tar.gz
  Running setup.py (path:/Users/bsilverberg/.virtualenvs/oneanddone-pr139/build/py-bcrypt/setup.py) egg_info for package py-bcrypt

Installing collected packages: MySQL-python, Jinja2, py-bcrypt
  Running setup.py install for MySQL-python
    building '_mysql' extension
    clang -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -pipe -Dversion_info=(1,2,3,'gamma',1) -D__version__=1.2.3c1 -I/usr/local/mysql/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _mysql.c -o build/temp.macosx-10.8-intel-2.7/_mysql.o -Os -g -fno-strict-aliasing -arch x86_64
    clang: error: unknown argument: '-mno-fused-madd' [-Wunused-command-line-argument-hard-error-in-future]
    clang: note: this will be a hard error (cannot be downgraded to a warning) in the future
    error: command 'clang' failed with exit status 1
    Complete output from command /Users/bsilverberg/.virtualenvs/oneanddone-pr139/bin/python -c "import setuptools, tokenize;__file__='/Users/bsilverberg/.virtualenvs/oneanddone-pr139/build/MySQL-python/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/sk/1kj23lvn46n7pvr7_xk074wm0000gn/T/pip-O1DeTX-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/bsilverberg/.virtualenvs/oneanddone-pr139/include/site/python2.7:
    running install

running build

running build_py

creating build

creating build/lib.macosx-10.8-intel-2.7

copying _mysql_exceptions.py -> build/lib.macosx-10.8-intel-2.7

creating build/lib.macosx-10.8-intel-2.7/MySQLdb

copying MySQLdb/__init__.py -> build/lib.macosx-10.8-intel-2.7/MySQLdb

copying MySQLdb/converters.py -> build/lib.macosx-10.8-intel-2.7/MySQLdb

copying MySQLdb/connections.py -> build/lib.macosx-10.8-intel-2.7/MySQLdb

copying MySQLdb/cursors.py -> build/lib.macosx-10.8-intel-2.7/MySQLdb

copying MySQLdb/release.py -> build/lib.macosx-10.8-intel-2.7/MySQLdb

copying MySQLdb/times.py -> build/lib.macosx-10.8-intel-2.7/MySQLdb

creating build/lib.macosx-10.8-intel-2.7/MySQLdb/constants

copying MySQLdb/constants/__init__.py -> build/lib.macosx-10.8-intel-2.7/MySQLdb/constants

copying MySQLdb/constants/CR.py -> build/lib.macosx-10.8-intel-2.7/MySQLdb/constants

copying MySQLdb/constants/FIELD_TYPE.py -> build/lib.macosx-10.8-intel-2.7/MySQLdb/constants

copying MySQLdb/constants/ER.py -> build/lib.macosx-10.8-intel-2.7/MySQLdb/constants

copying MySQLdb/constants/FLAG.py -> build/lib.macosx-10.8-intel-2.7/MySQLdb/constants

copying MySQLdb/constants/REFRESH.py -> build/lib.macosx-10.8-intel-2.7/MySQLdb/constants

copying MySQLdb/constants/CLIENT.py -> build/lib.macosx-10.8-intel-2.7/MySQLdb/constants

running build_ext

building '_mysql' extension

creating build/temp.macosx-10.8-intel-2.7

clang -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -pipe -Dversion_info=(1,2,3,'gamma',1) -D__version__=1.2.3c1 -I/usr/local/mysql/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _mysql.c -o build/temp.macosx-10.8-intel-2.7/_mysql.o -Os -g -fno-strict-aliasing -arch x86_64

clang: error: unknown argument: '-mno-fused-madd' [-Wunused-command-line-argument-hard-error-in-future]

clang: note: this will be a hard error (cannot be downgraded to a warning) in the future

error: command 'clang' failed with exit status 1

----------------------------------------
Cleaning up...
Command /Users/bsilverberg/.virtualenvs/oneanddone-pr139/bin/python -c "import setuptools, tokenize;__file__='/Users/bsilverberg/.virtualenvs/oneanddone-pr139/build/MySQL-python/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/sk/1kj23lvn46n7pvr7_xk074wm0000gn/T/pip-O1DeTX-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/bsilverberg/.virtualenvs/oneanddone-pr139/include/site/python2.7 failed with error code 1 in /Users/bsilverberg/.virtualenvs/oneanddone-pr139/build/MySQL-python
Storing debug log for failure in /var/folders/sk/1kj23lvn46n7pvr7_xk074wm0000gn/T/tmpsJh6Ic

Is this something you can look into, @mjzffr?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I just figured it out myself (thanks Google). Based on a blog post [1], I added an environment variable to the beginning of the pip command, so my full command line is:

ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install -r requirements/compiled.txt

I think we should add a note to the readme to do this if that error is encountered.

[1] http://kaspermunck.github.io/2014/03/fixing-clang-error/

$ pip install -r requirements/dev.txt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The packages here are specifically aimed for developer version as mentioned here [0]
We can mention it here if we want to add this.
[0] https://github.com/mozilla/oneanddone/blob/master/requirements/dev.txt#L1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what you mean, @bitgeeky. These instructions are for a development setup so I don't think it's necessary to further clarify that dev.txt refers to packages useful to developers.

```
_Note_: On OS X (in particular 10.8.5, Xcode 5.1), you may encounter the following error: `clang: error: unknown argument. '-mno-fused-madd'`. Try running pip with the `ARCHFLAGS` environment variable set, as follows: `ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install -r requirements/compiled.txt`

4. Set up a local MySQL database. The [MySQL Installation Documentation][mysql]
explains how to do this. Make sure your DB is utf8.

5. Configure your local settings by copying `oneanddone/settings/local.py-dist` to
6. Configure your local settings by copying `oneanddone/settings/local.py-dist` to
`oneanddone/settings/local.py` and customizing the settings in it:

```sh
$ cp oneanddone/settings/local.py-dist oneanddone/settings/local.py
```

The file is commented to explain what each setting does and how to customize
them. One item in the local.py settings file you are going to want to change, if
you are running this locally and not over HTTPS, is the following.

Open up local.py, find and uncomment SESSION_COOKIE_SECURE = False

them. Here are some highlights:
* If you are running this locally and not over HTTPS, uncomment `SESSION_COOKIE_SECURE = False`
* The `HMAC_KEYS` dictionary should not be empty.
* Provide a value for `SECRET_KEY`.

7. Create the initial empty database:

7. Initialize your database structure:
```sh
# Start the MySQL server
$ mysql.server start
# Once successfully started, log into the console
# using your username and password
$ mysql -uroot -p
# Create the database
mysql> create database oneanddone;
$ python manage.py syncdb
```
If you are asked to create a superuser, do so. Don't worry if you miss this step: see the [Users](#users) section below for more information.

8. Initialize your database structure:
Once finished, the `syncdb` command should produce a message about which models have been synced. At the bottom, the message will include something like this:

```sh
$ python manage.py syncdb
```
Not synced (use migrations):
- oneanddone.tasks
- oneanddone.users
- rest_framework.authtoken
```

This means that you must also run `./manage.py migrate [model]` for each model that is not synced with `syncdb`. More information about South migrations is included under the [Applying Migrations](#applying-migrations) section below.

Users
-----

Playdoh uses [BrowserID][django-browserid], a.k.a. Mozilla Persona, for user authentication. To add users to your local database, simply sign into your local One and Done instance. You may want to use dummy email accounts as described in Mozilla's guide to [testing Persona][persona].

You need at least one superuser to be able to develop and test administrative features of the project. You can create as many superusers as you like with `python manage.py createsuperuser`. After that, sign into your local One and Done instance with the superuser's email address as usual.


Applying Migrations
-------------------
Expand All @@ -90,22 +125,24 @@ run the following:
$ ./manage.py migrate oneanddone.tasks && ./manage.py migrate oneanddone.users
```

If you make changes to an existing model you will need to regeneratre the schema migration as follows:
If you make changes to an existing model, say `oneanddone.tasks`, you will need to regeneratre the schema migration as follows:

```sh
$ ./manage.py schemamigration oneanddone.tasks --auto
```

To generate a blank schema migration:
To generate a blank data migration:

```sh
$ ./manage.py datamigration oneanddone.mymodel data_migration_name
# ./manage.py datamigration [model] [data_migration_name]
# Example:
$ ./manage.py datamigration oneanddone.tasks task_data
```

Then fill in the generated file with logic, fixtures, etc. You can then apply this migration as above with:

```sh
$ ./manage.py migrate oneanddone.mymodel
$ ./manage.py migrate oneanddone.tasks
```


Expand All @@ -124,8 +161,21 @@ You can launch the development server like so:
$ python manage.py runserver
```

If you are asked to create a super user, just enter no and let the process complete.
Running Unit Tests
------------------
You can run all the unit tests in verbose mode as follows:

```sh
$ python manage.py test -v 2
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good point. Please also add a command to run a particular test or tests in a particular module.

You can also run spefic tests:
```sh
# All tests in tasks/tests/test_helpers module.
$ python manage.py test oneanddone.tasks.tests.test_helpers -v 2
# Just one test (PageUrlTests.test_basic)
$ python manage.py test oneanddone.tasks.tests.test_helpers:PageUrlTests.test_basic -v 2

```

REST API Support
----------------
Expand Down Expand Up @@ -169,7 +219,5 @@ Functional (Selenium) tests for oneanddone are maintained by the Web QA team and

License
-------
This software is licensed under the `Mozilla Public License v. 2.0`_. For more
This software is licensed under the [Mozilla Public License v. 2.0](http://mozilla.org/MPL/2.0/). For more
information, read the file ``LICENSE``.

.. _Mozilla Public License v. 2.0: http://mozilla.org/MPL/2.0/