delineate.io

portray or describe (something) precisely.

Scheduled Cloud Functions

Demonstrate using Terraform to provision scheduled cloud functions securely on Google Cloud
Explore the docs »

Report Bug · Request Feature

Table of Contents

1. About The Project
   • Built With
2. Getting Started
   • Prerequisites
   • Installation
3. Usage
4. Roadmap
5. Contributing
6. License
7. Contact
8. Acknowledgements

About The Project

The purpose of this repo is to demonstrate scheduling cloud functions on Google Cloud Platform.

Built With

• NodeJS used as the language to implement the cloud function
• Google Functions Framework enabling better developer workflow and simplifying local development
• Hashicorp Vagrant to provide a consistent and isolated developer desktop
• Hashicorp Terraform used to provision cloud resources for the scheduled function
• Google Cloud Platform used as the hosting platform for the cloud resources

Getting Started

To get a local copy up and running follow these simple steps.

Prerequisites

This is an example of how to list things you need to use the software and how to install them.

In future the GCP bootstrapping can be automated using a bootstrapping script to smooth setup.

# installs vagrant
brew install --cask vagrant

# installs httpie
brew install httpie

GCP APIs

Before running the terraform provisioning the following two Google Cloud APIs must be enabled:

• Cloud Resource Manager API
• Identity and Access Management API

When using gcloud with a correctly authenticated and permissioned the APIs can be enabled with the following command

# enables the service
gcloud services enable "${SERVICE}" --async

Terraform Service Account

Once the right APIs are enabled a service account for terraform needs to be created with the following roles:

• App Engine Creator
• Cloud Functions Admin
• Cloud Scheduler Admin
• Service Account Admin
• Create Service Accounts
• Service Account User
• Project IAM Admin
• Service Usage Admin
• Storage Admin
• Storage Object Admin

Installation

# clone the repo
git clone https://github.com/delineateio/scheduled-cloud-functions.git

Once the repo is cloned create ./box.yml in the root of the project. This is required to configure gcloud when the vagrant box is provisioned.

gcloud:
  account:  # service_account_email
  key:      # service_account_key (json)
  project:  # gcp_project
  region:   # gcp_region
  zone:     # gcp_zone

Usage

Local Testing

The below instructions can be used to run the function locally for testing purposes, you can find out more here. Additionally, you can find out more by watching the Functions Framework video.

# navigate function
cd ./dev

# install and run the server
npm install
npm start

# use preferred tool from host (curl, httpie etc)
http :8080

Provisioning in GCP

It is important to note that some of the GCP resources are eventually consistent. What this means in practice, is that if the scheduled job is immediately manually run from the console it sometimes fails. However it stabilises after some time as proven by testing.

# navigate to infra
cd ./ops

# initalises the modules
terraform init

# applies the infra
terraform apply

# destroy the infrastructure (when required)
terraform destroy

Roadmap

See the open issues for a list of proposed features (and known issues).

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

If you would like to contribute to any Capco Digital OSS projects please read:

• Code of Conduct
• Contributing Guidelines

License

Distributed under the MIT License. See LICENSE for more information.

Acknowledgements

• Best README Template