Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump color-string from 1.5.4 to 1.6.0 #41

Open
wants to merge 8 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .cloudbuild.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
steps:
- id: 'build-image'
name: 'gcr.io/cloud-builders/docker'
args:
- 'build'
- '-t'
- 'eu.gcr.io/$PROJECT_ID/$REPO_NAME/$BRANCH_NAME/$REPO_NAME:$SHORT_SHA'
- '-t'
- 'eu.gcr.io/$PROJECT_ID/$REPO_NAME/$BRANCH_NAME/$REPO_NAME:latest'
- '.'
- '-f'
- './Dockerfile'
timeout: 1200s
- id: 'push-to-registry'
name: 'gcr.io/cloud-builders/docker'
args:
- 'push'
- 'eu.gcr.io/$PROJECT_ID/$REPO_NAME/$BRANCH_NAME/$REPO_NAME'
- id: 'deploy-to-gke'
name: 'gcr.io/cloud-builders/gcloud'
env:
- 'KUBECONFIG=/.kube/config'
entrypoint: 'bash'
args:
- '-c'
- |
gcloud container clusters get-credentials soils-revealed-cluster --project=$PROJECT_ID --zone=europe-west4-a
kubectl set image deployment/$REPO_NAME --namespace=$BRANCH_NAME soils-revealed=eu.gcr.io/$PROJECT_ID/$REPO_NAME/$BRANCH_NAME/$REPO_NAME:$SHORT_SHA
kubectl rollout restart deployment $REPO_NAME --namespace=$BRANCH_NAME
18 changes: 18 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# ignore folder on copy (more for local build)
*.md
.git
.cache
.history
node_modules
npm-debug.log
.prettierrc
.gitignore
.github
.git
.eslintrc
.env.default
.editorconfig
.dockerignore
.env
Dockerfile
.cloudbuild.yaml
4 changes: 2 additions & 2 deletions .env.default
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
PORT=3000
PORT=3001
MAPBOX_API_KEY=
API_URL=
ANALYSIS_API_URL=
DEPLOYMENT_KEY=
GOOGLE_ANALYTICS_KEY=
AWS_REGION=
AWS_BUCKET_NAME=
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_MAX_Z_TILE_STORAGE=
DEPLOYMENT_KEY=
61 changes: 0 additions & 61 deletions .github/workflows/production.yml

This file was deleted.

61 changes: 0 additions & 61 deletions .github/workflows/staging.yml

This file was deleted.

5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,8 @@ yarn-error.log*

# keys
gee.key.json

#VScode
.history/


29 changes: 29 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
FROM docker.io/debian:stable-slim@sha256:a939c03c4d3e3f53e3ef4ef6e75cb681a3ad56537842f95bf89755da86559b13

# Source: https://www.kabisa.nl/tech/nvm-in-docker/
# docker build --force-rm --no-cache -t soils-revealed:latest .
# docker run -p3001:3001 --env-file .env soils-revealed:latest

SHELL ["/bin/bash","-l","-c"]
ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update \
&& apt-get install -y tini curl \
&& rm -rf /var/lib/apt/lists/*

RUN mkdir /soils-revealed
WORKDIR /soils-revealed
COPY . .

RUN curl --silent -o- https://raw.githubusercontent.com/creationix/nvm/master/install.sh | bash
RUN nvm install && nvm use
RUN echo -e "Implementing node: $(node --version)\nFrom: $(which node)"

RUN npm install -g yarn
RUN npm install -g pm2
RUN yarn install --frozen-lockfile

RUN DEPLOYMENT_KEY=$(date +%s) && echo "DEPLOYMENT_KEY=$DEPLOYMENT_KEY" > .env

ENTRYPOINT ["/usr/bin/tini","-g","--"]
CMD ["/soils-revealed/run.sh","production"]
38 changes: 30 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,15 @@ In order to start modifying the app, please make sure to correctly configure you
5. Use the correct Node.js version for this app by running `nvm use`; if you didn't install NVM (step 2), then manually install the Node.js version described in `.nvmrc`
6. Install the dependencies: `yarn`
7. Create a `.env` file at the root of the project by copying `.env.default` and giving a value for each of the variables (see next section for details)
8. Create a `gee.key.json` file at the root of the project with the Google Earth Engine's private key inside
8. Create a gee.key.json file at the root of the project with the Google Earth Engine's private key inside.

9. Run the server: `yarn dev`

You can access a hot-reloaded version of the app on [http://localhost:3000](http://localhost:3000).

The application is built using [React](https://reactjs.org/) and the framework [Next.js](https://nextjs.org/). The styles use [Sass](https://sass-lang.com/) and the [Bootstrap](https://getbootstrap.com/) framework.

A continuous deployment system is in place. Each time you push to the `master` branch, the application is deployed to production through a GitHub Action defined in `.github/workflows/production.yml`. Each time you push to `develop`, the application is deployed to staging through an action defined in `.github/workflows/staging.yml`. You can see the status of the build in the “Actions” tab of the repository on GitHub.
A continuous deployment system is in place. Each time you push to the `master` branch, the application is deployed to production. Github will trigger an event on google cloud run and make a deployment. The same for the `develop` branch.

## Environment variables

Expand All @@ -43,19 +44,40 @@ Below is a description of each of the keys.
| AWS_BUCKET_NAME | Name of the AWS S3 bucket storing the tiles of the soils layers |
| AWS_ACCESS_KEY_ID | Access key ID of the AWS server storing the tiles of the soils layers |
| AWS_SECRET_ACCESS_KEY | Secret access key of the AWS server storing the tiles of the soils layers |
| AWS_MAX_Z_TILE_STORAGE | Maximum zoom at which tiles generated on-the-fly will be saved in the AWS S3 bucket |
| AWS_MAX_Z_TILE_STORAGE | Maximum zoom at which tiles generated on-the-fly will be saved in the AWS S3 bucket

## Deployment

As explained before, the application is automatically deployed to staging when pushing new changes to the `develop` branch, and deployed to production when pushing changes to `master`. This is achieved through GitHub Actions defined in `.github/workflows`.
### Local computer

It is possible to run a local test deployment using the docker image. Docker implement an agnostic build and then during run it will pickup the container's env variables, and properly set the system

`run.sh` accepts 2 arguments: production or develop. Production argument will run `yarn start` and the code will production ready, while `develop` runs nodejs in development mode, necessary if you are testing content and changes.

```bash
docker build --force-rm --no-cache -t soils-revealed:latest .
docker run -p3001:3001 --env-file .env soils-revealed:latest /soils-revealed/run.sh production
```

**Note:** We have created a `.env` file on the project root with all variables, and this will be used to run nodejs in production mode. This is a deployment to test nodejs in production mode.

**Note:** Dockerfile has `CMD` implementing production.


When an action is executed, it connects via SSH to the server hosting the application. The server's credentials are stored in GitHub's “secrets” vault. A script is then executed: the running instance of the application is stopped, the code is pulled, the correct version of node is selected, the dependencies are installed, a local `.env` file is generated, a local `gee.key.json` file is generated, and the application is restarted.
### Google GKE

The `.env` file is programmatically generated on the server because it differs for each environment. Some of the keys are hard coded in the `.github/workflows/XXX.yml` file and others are pulled from GitHub's “secrets” vault.
Public deployment is based on Google Cloud build and file `.cloudbuild.yaml`. Up on push to `master` or `develop`, the following steps will happen:

The `gee.key.json` file contains the credentials for the Google Earth Engine library. It is also programmatically generated as its value is stored in GitHub's “secrets” vault.
1. Github will trigger a Google Cloud run trigger
2. Google cloud will pull the branch content.
3. Docker build will be iniciated, using `Dockerfile` and `.cloudbuild.yaml`
4. After completed Docker image is stored on a private repository, using tags `latest` and `$SHORT_SHA`
5. Google Cloud build will update the image on GKE and make a `kubectl rollout restart`
6. GKE contains a specific `ConfigMap` with all .env necessary for deployment.
7. `gee.key.json` is added to the pods using a `ConfigMap` mount


Overall, deploying to either environment takes between 1 to 2 minutes to complete.
Overall, deploying to either environment takes between 5 to 10 minutes to complete. If deployment is not successful GKE will continue implementing the previous deployment.

## Architecture

Expand Down
1 change: 1 addition & 0 deletions components/explore/component.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import throttle from 'lodash/debounce';

import { Router } from 'lib/routes';
import { logEvent } from 'utils/analytics';
import { isModalShown } from 'utils/explore';
import { useHasMounted, useDesktop } from 'utils/hooks';
import { toggleBasemap, toggleLabels, toggleRoads } from 'utils/map';
import {
Expand Down
2 changes: 1 addition & 1 deletion components/explore/tour/component.js
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ const ExploreTour = props => {
previousShowTour.current = showTour;
}
}
}, [previousShowTour.current, showTour, props, stepIndex, setOpened, updateShowTour]);
}, [showTour, props, stepIndex, setOpened, updateShowTour]);

const onChange = useCallback(
({ action, type }) => {
Expand Down
2 changes: 1 addition & 1 deletion components/footer/component.js
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ const Footer = () => {
</button>
</li>
<li>
<a href="mailto:soilsrevealed@tnc.org" className="nav-link pb-0">
<a href="mailto:info@soilsrevealed.org" className="nav-link pb-0">
Contact
</a>
</li>
Expand Down
6 changes: 5 additions & 1 deletion components/forms/radio/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import PropTypes from 'prop-types';

import './style.scss';

const Radio = ({ id, name, disabled, checked, onChange, children, className }) => (
const Radio = ({ id, name, disabled, checked, onChange, children, className, required }) => (
<div
className={[
'custom-control',
Expand All @@ -20,12 +20,14 @@ const Radio = ({ id, name, disabled, checked, onChange, children, className }) =
name={name}
checked={checked}
onChange={onChange}
required={required}
/>
<label className="custom-control-label" htmlFor={id}>
{children}
</label>
</div>
);

Radio.propTypes = {
id: PropTypes.string.isRequired,
name: PropTypes.string.isRequired,
Expand All @@ -34,13 +36,15 @@ Radio.propTypes = {
onChange: PropTypes.func,
children: PropTypes.node.isRequired,
className: PropTypes.string,
required: PropTypes.bool,
};

Radio.defaultProps = {
disabled: false,
checked: false,
onChange: null,
className: null,
required: false,
};

export default Radio;
2 changes: 1 addition & 1 deletion components/map/constants.js
Original file line number Diff line number Diff line change
Expand Up @@ -1082,7 +1082,7 @@ exports.LAYERS = {
'soc-experimental': {
label: 'Soil organic carbon experimental approach',
description:
'These maps display soil organic carbon in Argentina in any year between 1982 and 2017. You can toggle between soil organic carbon concentration and soil organic carbon stock, as well as between soil organic carbon maps for any given year and change maps between years. Selecting “Change” will display where Argentina has experienced a gain in soil organic carbon and which parts a loss for a chosen time period. To produce these maps, we used a novel machine learning algorithm based on field samples to map soil organic carbon over space and time.\n\nA global implementation of this approach is in process. If you would like to join this effort, please [contact us](mailto:soilsrevealed@tnc.org), and join our growing partnership.',
'These maps display soil organic carbon in Argentina in any year between 1982 and 2017. You can toggle between soil organic carbon concentration and soil organic carbon stock, as well as between soil organic carbon maps for any given year and change maps between years. Selecting “Change” will display where Argentina has experienced a gain in soil organic carbon and which parts a loss for a chosen time period. To produce these maps, we used a novel machine learning algorithm based on field samples to map soil organic carbon over space and time.\n\nA global implementation of this approach is in process. If you would like to join this effort, please [contact us](mailto:info@soilsrevealed.org), and join our growing partnership.',
group: 'soc',
attributions: [],
bbox: [
Expand Down
1 change: 1 addition & 0 deletions google5dcefa970456a1f5.html
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
google-site-verification: google5dcefa970456a1f5.html
1 change: 1 addition & 0 deletions index.js
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ const handle = app.getRequestHandler();
// Initialize GEE
let geePrivateKey;
try {

geePrivateKey = require('./gee.key.json');
ee.data.authenticateViaPrivateKey(
geePrivateKey,
Expand Down
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,9 @@
"@turf/area": "^6.0.1",
"@turf/bbox": "^6.0.1",
"@zeit/next-sass": "^1.0.1",
"axios": "^0.19.2",
"airtable": "^0.11.1",
"aws-sdk": "^2.808.0",
"axios": "^0.19.2",
"bootstrap": "^4.4.1",
"classnames": "^2.2.6",
"dayjs": "^1.9.6",
Expand Down
Loading