Skip to content
Python R Shell
Branch: master
Clone or download

Latest commit

Fetching latest commit…
Cannot retrieve the latest commit at this time.

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docs
.nojekyll
README.md
_site.yml
environment.yml
index.Rmd
kijiji_scraper.py
preprocess_data.R
run_scrapper.sh
scrapper.py
stats_eng.Rmd
stats_habr.Rmd
summarize.py

README.md

Spatiotemporal statistics of Montreal Real Estate

Scripts

  • scrapper.py - scrapper from MLS system, dowload data about Montreal and the South Shore, stored information in gzipped json files
  • kijiji_scraper.py - generic kijiji scrapper, stored information in property.sqlite3 database
  • summarize.py - processed data from MLS system, from gzipped json file(s) and stores it in property.sqlite3
  • preprocess_data.R - R script , performing some basic preoprocessing and filtering data from property.sqlite3 outputs preprocessed.RData
  • environment.yml - conda environment description
  • run_scrapper.sh - template of a shell script tying it all together
  • index.Rmd - knitr script, used to generate http://www.ilmarin.info/re_mtl
  • stats_habr.Rmd - knitr script , used to generate http://www.ilmarin.info/re_mtl/stats_habr.html

Data files

these files can be downloaded from https://github.com/vfonov/re_mtl/releases/tag/v0.0

Requirements

  • R version 3.6

Installing

conda env create --name re_mtl -f environment.yml

# to download preprocessed data
curl -L https://github.com/vfonov/re_mtl/releases/download/v0.0/preprocessed.RData -o preprocessed.RData

# to download raw data
curl -L https://github.com/vfonov/re_mtl/releases/download/v0.0/property.sqlite3.xz -o property.sqlite3.xz
unxz property.sqlite3.xz 

Running

conda activate re_mtl

# to regenerate preprocessed.RData
Rscript preprocess_data.R

# to regenerate contents of http://www.ilmarin.info/re_mtl/
Rscript -e "rmarkdown::render_site('index.Rmd')" 
You can’t perform that action at this time.