Node.js, preferably using NVM.
npm install serverlessInstalls Serverless, a command line utility that simplifies the deployment of services that run on AWS Lambda, Azure Functions, or others.
npm install --save serverless-python-requirementsInstalls a Serverless plugin that will download the PIP requirements specified in
requirements.txtbefore deploying to AWS. (Note: you need Docker installed for this to run correctly.)
- To deploy to AWS, you’ll need to set up an AWS account, if you haven’t
already. You should also configure a
cornerwiseprofile in your AWS credentials. See here for details about setting up a profile and the privileges the AWS user requires
credentials.jsonand modify the variables to use your Socrata credentials.
- If everything is correctly configured, you should be able to
cdto this directory and type
serverless deploy -vto fully deploy the lambda function and corresponding API Gateway interface to AWS.
Somerville, MA Reports and Decisions
- Scrapes the OSPCD’s Reports and Decisions page.
Somerville, MA PB/ZBA Event Scraper
- Scrapes the city’s events page, finds events for the Planning Board and Zoning Board of Appeals, and scrapes the attached Agenda for related case numbers.
Somerville, MA Capital Projects
- Published annually by the Somerville Capital Projects Committee, the dataset includes “infrastructure projects, building improvements, park redesigns, and equipment purchases.”
Green Line Extension
- Scrapes the “Upcoming Meetings” section of the Green Line Extension home page.
Using the Scrapers
The interface to the scrapers is intentionally simple. Place a GET request to
the scraper’s URL. You may optionally supply a
since query parameter
yyyymmdd. The scraper will respond with JSON conforming to the
Cornerwise scraper schema.