AWS Lambda function to fetch radar images periodically from weather.gc.ca. The function should be configured to run daily (at any time), and then will visit weather.gc.ca historical imagery and download one image for each of the last 24 hours for each radar site for each image type. The images will then be pushed to AWS S3.
- Create a lambda function through the AWS Console.
- Configure the lambda function to run daily.
.envand set the target bucket name and desired sites inside.
- Ensure you've run
- Run the deploy script in this repository:
Thats it! The lambda function will now run daily and grab the last 24 URLs for each radar site for each image type. The images will be uploaded to the s3 bucket specified in the
The radar collector can be run manually for a single date, or for a range of dates.
# Single Date bin/fetch-for-date.js 2017-01-15 # Range of Dates bin/fetch-for-range.sh 2016-01-01 2017-01-01
This might take awhile to run, so it is best to setup a spot instance to run this for you. This will install dependencies required to run on a fresh instance. Be sure to copy over an env file.
sudo yum update -y; sudo yum install -y tmux git curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.1/install.sh | bash source $HOME/.nvm/nvm.sh git clone https://github.com/tesera/lambda-fetch-radar.git cd lambda-fetch-radar nvm install npm install cp sample.env .env vi .env # Copy your env parameters here ./bin/fetch-for-range.sh 2016-01-01 2017-01-01