Skip to content

Commit

Permalink
📝 Update README.md and Makefile
Browse files Browse the repository at this point in the history
  • Loading branch information
nkmr-jp committed Jun 12, 2022
1 parent 13a07d8 commit 07f0858
Show file tree
Hide file tree
Showing 2 changed files with 52 additions and 5 deletions.
10 changes: 6 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ FUNC_NAME=fetch
ENTRY_POINT=Fetch
TOPIC_NAME=$(FUNC_NAME)-topic
BUCKET_NAME=$(PROJECT_ID)-fetch
VERSION=$(shell git rev-parse --short HEAD)
# VERSION=$(shell git rev-parse --short HEAD)
VERSION=$(shell git describe --abbrev=0 --tags)

init:
@echo
Expand All @@ -33,7 +34,9 @@ init:
-gsutil lifecycle set ./lifecycle.json gs://$(BUCKET_NAME)-test
@echo
@echo "---- check resources in google cloud console. ----"
make open
open https://console.cloud.google.com/iam-admin/serviceaccounts?project=$(PROJECT_ID)
open https://console.cloud.google.com/cloudpubsub/topic/detail/$(FUNC_NAME)-topic
open https://console.cloud.google.com/storage/browser?project=$(PROJECT_ID)

test:
export BUCKET_NAME=$(BUCKET_NAME)-test && go test -v
Expand Down Expand Up @@ -62,10 +65,9 @@ log:
gcloud beta functions logs read $(FUNC_NAME) --gen2 --limit=100

open:
open https://console.cloud.google.com/iam-admin/serviceaccounts?project=$(PROJECT_ID)
open https://console.cloud.google.com/cloudpubsub/topic/detail/$(FUNC_NAME)-topic
open https://console.cloud.google.com/storage/browser?project=$(PROJECT_ID)
open https://console.cloud.google.com/functions/details/$(REGION)/$(FUNC_NAME)?env=gen2
open "https://console.cloud.google.com/logs/query;query=%2528resource.type%20%3D%20%22cloud_function%22%0Aresource.labels.function_name%20%3D%20%22$(FUNC_NAME)%22%0Aresource.labels.region%20%3D%20%22$(REGION)%22%2529%0A%20OR%20%0A%2528resource.type%20%3D%20%22cloud_run_revision%22%0Aresource.labels.service_name%20%3D%20%22$(FUNC_NAME)%22%0Aresource.labels.location%20%3D%20%22$(REGION)%22%2529%0A%20severity%3E%3DDEFAULT;?project=$(PROJECT_ID)"

lint:
golangci-lint run --fix
47 changes: 46 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,55 @@ Cloud Functions that fetch data from public APIs and store it in Google Cloud St
- Fetch data is managed by [Object Versioning](https://cloud.google.com/storage/docs/object-versioning).
- GCS [price](https://cloud.google.com/storage/pricing) are optimized by [Object Lifecycle Management](https://cloud.google.com/storage/docs/lifecycle).
- The storage class is set to change from Standard to Coldline after 7 days from object creation and from Coldline to Archive after 30 days (easy to change).
- [zl](https://github.com/nkmr-jp/zl) (zap based logger) for logging by level.
- [zl](https://github.com/nkmr-jp/zl) (zap based logger) for logging by severity level.
- Logs in JSON format so it can check the element contents in detail with [Cloud Logging](https://cloud.google.com/logging).
- You can also check the contents of CloudEvents triggered by Functions in the log.

<details>
<summary>Cloud Logging's log example</summary>

```json

{
"insertId": "xxxxxxxxxxxxxxxxxxxxxx",
"jsonPayload": {
"timestamp": "2022-06-12T00:45:17.427119741Z",
"function": "github.com/nkmr-jp/gcf-fetch.parseEvent",
"cloudEventContext": "Context Attributes,\n specversion: 1.0\n type: google.cloud.pubsub.topic.v1.messagePublished\n source: //pubsub.googleapis.com/projects/[your project id]/topics/fetch-topic\n id: xxxxxxxxxxx\n time: 2022-06-12T00:45:14.378Z\n datacontenttype: application/json\n",
"cloudEventData": {
"subscription": "projects/[your project id]/subscriptions/eventarc-asia-northeast1-fetch-xxxxx-sub-xxx",
"message": {
"publishTime": "2022-06-12T00:45:14.378Z",
"messageId": "4863463195745766",
"data": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
}
},
"caller": "https://github.com/nkmr-jp/gcf-fetch/blob/v1.0.0/fetch.go#L89",
"version": "v1.0.0",
"message": "CLOUD_EVENT_RECEIVED"
},
"resource": {
"type": "cloud_run_revision",
"labels": {
"service_name": "fetch",
"project_id": "[your project id]",
"configuration_name": "fetch",
"revision_name": "fetch-xxxx-xiv",
"location": "asia-northeast1"
}
},
"timestamp": "2022-06-12T00:45:17.427272Z",
"severity": "INFO",
"labels": {
"goog-managed-by": "cloudfunctions",
"instanceId": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
},
"logName": "projects/[your project id]/logs/run.googleapis.com%2Fstderr",
"receiveTimestamp": "2022-06-12T00:45:17.670946465Z"
}
```

</details>

# Prepare

Expand Down

0 comments on commit 07f0858

Please sign in to comment.