Skip to content
Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Build Status codecov


hk0weather is an open source web scraper project using Scrapy to collect the useful weather data from Hong Kong Observatory website.

Scrapy can output collected weather data into the machine-readable formats (eg. CSV, JSON, XML).

Optionally, this project supports a Django app 'openweather' to store the collected weather data to Django web framework, and the data can be shown on web through the Django admin UI.

Available Spiders

  1. regional: Hong Kong Regional Weather Data in 10-minutes update from HKO.
  2. rainfall: Hong Kong Rainfall Data in hourly update from HKO.
  3. hkoforecast: Hong Kong Next 24 hour Weather Forecast Report from HKO Open Data.
  4. hko9dayforecast: Hong Kong 9-day Weather Report from HKO Open Data.

Installation Example

  1. Cloning and setup hk0weather in a Py3 virtual environment

    git clone  
    virtualenv hk0weatherenv  
    source hk0weatherenv/bin/activate  
    cd hk0weather   
    pip install -r requirements.txt    
  2. Optional: Setup hk0weather to use openweather

    pip install -r requirements-django.txt    
    cd ..   
    django-admin startproject yourweatherproject   
    cd yourweatherproject   
    git clone   

    Please add 'openweather' to INSTALLED_APPS in Django yourweatherproject/

    ./ makemigrations    
    ./ migrate   
    ./ createsuperuser   
    ./ runserver &    
    cd ../hk0weather     

    Django daemon is now running in the background, its web admin UI can be access at http://localhost:8000/admin.

    export PYTHONPATH=/your-full-path-to/yourweatherproject    
    export DJANGO_SETTINGS_MODULE=yourweatherproject.settings   

    Please export PYTHONPATH and DJANGO_SETTINGS_MODULE again after every activation of the Py3 virtual environment.

Run a Scrapy spider

Activate the Py3 virtual environment once before the first running of web spiders.

source hk0weatherenv/bin/activate  

Optionally, if Django is in use, export PYTHONPATH and DJANGO_SETTINGS_MODULE.

export PYTHONPATH=/your-full-path-to/yourweatherproject    
export DJANGO_SETTINGS_MODULE=yourweatherproject.settings   

Optionally, list all available spiders.

scrapy list 

Run a specific spider (eg. regional) in Scrapy

scrapy crawl regional   

and optionally use -t (file format) and -o (filename) to output the data in a json file.

scrapy crawl regional -t json -o test.json


Calvin Tsang.

Thanks for my sponsors, please consider to sponsor my works.



Web scraper project to collect the useful Hong Kong weather data from HKO website




No releases published

Sponsor this project



No packages published