Skip to content

Commit

Permalink
Update documentation.
Browse files Browse the repository at this point in the history
  • Loading branch information
ThomasThoren committed Jun 24, 2016
1 parent d980618 commit 4d0ecb9
Showing 1 changed file with 28 additions and 44 deletions.
72 changes: 28 additions & 44 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,14 @@
# New Orleans real estate

[http://vault.thelensnola.org/realestate](http://vault.thelensnola.org/realestate)
[https://vault.thelensnola.org/realestate](https://vault.thelensnola.org/realestate)

This app scrapes the latest property sales in New Orleans, stores the records in a database and publishes the results with a map.

[![Build Status](https://travis-ci.org/TheLens/realestate.svg?branch=master)](https://travis-ci.org/TheLens/realestate) [![Documentation Status](https://readthedocs.org/projects/realestate/badge/?version=latest)](https://readthedocs.org/projects/realestate/?badge=latest) [![Coverage Status](https://coveralls.io/repos/TheLens/realestate/badge.svg?branch=master)](https://coveralls.io/r/TheLens/realestate?branch=master)
[![Build Status](https://travis-ci.org/TheLens/realestate.svg?branch=master)](https://travis-ci.org/TheLens/realestate) [![Coverage Status](https://coveralls.io/repos/TheLens/realestate/badge.svg?branch=master)](https://coveralls.io/r/TheLens/realestate?branch=master)

Documentation: https://realestate.readthedocs.org/

Issues: https://github.com/TheLens/realestate/issues

Tests: https://travis-ci.org/TheLens/realestate

Testing coverage: https://coveralls.io/r/TheLens/realestate
- Issues: https://github.com/TheLens/realestate/issues
- Tests: https://travis-ci.org/TheLens/realestate
- Testing coverage: https://coveralls.io/r/TheLens/realestate

#### Dependencies

Expand All @@ -23,49 +19,37 @@ Testing coverage: https://coveralls.io/r/TheLens/realestate
* SQLAlchemy
* Virtualenvwrapper/virtualenv

#### Setup

(Not yet sure if this setup will work or not.)

`git clone https://github.com/TheLens/realestate.git`

`cd` into the `realestate` directory.

`pip install -r requirements.txt`

`createdb realestate`

`psql realestate < backups/realestate.sql` # For now, only available to The Lens employees. Access backup SQL file on S3.

#### Daily scripts

This will scrape, build, geocode, clean and publish the previous day's sales. These are summarized in `scripts/main.sh`, which is run on a cron job every night.

`python realestate/lib/scrape.py`

`python scripts/initialize.py`

There is also a cron task to run `scripts/backups.sh`, which creates a datestamped database backup on the server and then copies the file to S3.

#### Environment variables

Create environment variables in `~/.virtualenvs/realestate/bin/postactivate` locally. Do the same at `/home/ubuntu/.virtualenvs/realestate/bin/postactivate` on the server, but first remove "export" in each line.
Every day, these two commands are run to scrape and then build, geocode, clean and publish the previous day's sales. These are summarized in `scripts/main.sh`, which is run on a cron job every night.

```bash
export SERVER_ENGINE='postgresql://myuser:mypass@localhost/realestate'

export SERVER_CONNECTION='dbname=realestate user=myuser password=mypass'

export LRD_USERNAME='MyLandRecordsDivisionUsername'
python realestate/lib/scrape.py
python scripts/initialize.py
```

export LRD_PASSWORD='MyLandRecordsDivisionPassword'
Occasionally, due to bugs or password expiration, you will need to scrape and build beyond the previous day. You can specify the date range for both commands by using command-line arguments.

export DASHBOARD_USERNAME='myuser'
```bash
# <starting_date> <ending_date>
python realestate/lib/scrape.py 2016-05-01 2016-05-05
python scripts/initialize.py 2016-05-01 2016-05-05
```

export DASHBOARD_PASSWORD='mypass'
There is also a cron task to run `scripts/backups.sh`, which creates a date-stamped database backup on the server and then copies the file to S3. This runs every night.

export REAL_ESTATE_GMAIL_USERNAME=tthoren@thelensnola.org
#### Environment variables

export REAL_ESTATE_GMAIL_PASSWORD=mypass
Create environment variables in `~/.virtualenvs/realestate/bin/postactivate` locally. Do the same at `/home/ubuntu/.virtualenvs/realestate/bin/postactivate` on the server. You will need to include two lines in the server's postactivate file so that Upstart can find and use the variables.

```bash
REAL_ESTATE_LRD_USERNAME='MyLandRecordsDivisionUsername'
REAL_ESTATE_LRD_PASSWORD='MyLandRecordsDivisionPassword'
REAL_ESTATE_DATABASE_USERNAME='MyDatabaseUsername'
REAL_ESTATE_DATABASE_PASSWORD='MyDatabasePassword'

export REAL_ESTATE_LRD_USERNAME='MyLandRecordsDivisionUsername'
export REAL_ESTATE_LRD_PASSWORD='MyLandRecordsDivisionPassword'
export REAL_ESTATE_DATABASE_USERNAME='MyDatabaseUsername'
export REAL_ESTATE_DATABASE_PASSWORD='MyDatabasePassword'
```

0 comments on commit 4d0ecb9

Please sign in to comment.