Skip to content

Commit

Permalink
Added contributing guide
Browse files Browse the repository at this point in the history
  • Loading branch information
prabhatsharma committed Feb 23, 2022
1 parent 7ad98d2 commit 6785b02
Show file tree
Hide file tree
Showing 3 changed files with 188 additions and 6 deletions.
110 changes: 110 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# Contributing to Zinc
## Setting up development environment

### Prerequisite

Zinc uses Go (For server) and VueJS (For Web UI)

You must have follwing installed:

1. Git
2. Go 1.16 +
3. nodejs v14+ and npm v6+

## Building from source code

### Lets clone the repo and get started

```shell
git clone https://github.com/prabhatsharma/zinc
cd zinc
```

### Now let's build the UI

```shell
cd web
npm install
npm run build
cd ..
```

Output will be stored in web/dist folder. web/dist will be embedded in zinc binary when zinc go application is built.

It is important that you build the web app every time you make any changes to javascript code as the built code is then embedded in go application.

### Time to build the go application now

Download the dependencies

```shell
go get -d -v # this will download the go libraries used by zinc
```

Simple:
```shell
go build -o zinc cmd/zinc/main.go # will build the zinc binary
```

Advanced

```shell
CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -ldflags="-s -w -X github.com/prabhatsharma/zinc/pkg/meta/v1.Version=${VERSION} -X github.com/prabhatsharma/zinc/pkg/meta/v1.CommitHash=${COMMIT_HASH} -X github.com/prabhatsharma/zinc/pkg/meta/v1.BuildDate=${BUILD_DATE}" -o zinc cmd/zinc/main.go
```

Setting GOOS and GOARCH appropriately allows for cross platform compilation. Check [Official docs](https://go.dev/doc/install/source#environment) for all possible values and combinations. This [gist](https://gist.github.com/asukakenji/f15ba7e588ac42795f421b48b8aede63) is also great.


Setting CGO_ENABLED=0 allows for static linking, which results in a single binary output that has no dependencies.

setting up ldflags allows for passing values like version number to the binary at build time instead of hardcoding the value in cource code. Generally the version number is set by the CI pipeline during build by using the git tag.

## Developing

Once you have the source code cloned you can start development.

There are 2 areas of development.

1. UI
1. Server


### Server

```shell
go get -d -v
ZINC_FIRST_ADMIN_USER=admin ZINC_FIRST_ADMIN_PASSWORD=Complexpass#123 go run cmd/zinc/main.go
```

This will start the Zinc API server on port 4080

environment variables ZINC_FIRST_ADMIN_USER and ZINC_FIRST_ADMIN_PASSWORD are required only first time when zinc is started.

### UI

```shell
cd web
npm install
npm run serve
```
This will start UI server on port 8080

In order for you to effectively use the UI you would wnat to have the Zinc API server running in a seperate window that will accept requests from the UI.


## Build docker image

Make sure that you have [docker](https://docs.docker.com/get-docker/).

Simple build:

```shell
docker build --tag zinc:latest . -f Dockerfile.hub
```
Multi-arch build

In order to build multi-srach builds you will need [buildx](https://docs.docker.com/buildx/working-with-buildx/) installed. You will need to pass the platform flag for the platform that you want to build.

```shell
docker buildx build --platform linux/amd64 --tag zinc:latest-linux-amd64 . -f Dockerfile.hub
```
68 changes: 68 additions & 0 deletions Dockerfile.hub
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# syntax=docker/dockerfile:experimental
############################
# STEP 1 build executable binary
############################
# FROM golang:alpine AS builder
FROM --platform=$BUILDPLATFORM golang:latest as builder
ARG VERSION
ARG COMMIT_HASH
ARG BUILD_DATE
ARG TARGETOS
ARG TARGETARCH

RUN update-ca-certificates
# RUN apk update && apk add --no-cache git
# Create appuser.
ENV USER=appuser
ENV UID=10001
# See https://stackoverflow.com/a/55757473/12429735RUN
RUN adduser \
--disabled-password \
--gecos "" \
--home "/nonexistent" \
--shell "/sbin/nologin" \
--no-create-home \
--uid "${UID}" \
"${USER}"
WORKDIR $GOPATH/src/github.com/prabhatsharma/zinc/
COPY . .
# Fetch dependencies.
# Using go get.
RUN go get -d -v
# Using go mod.
# RUN go mod download
# RUN go mod verify
# Build the binary.
# to tackle error standard_init_linux.go:207: exec user process caused "no such file or directory" set CGO_ENABLED=0.
# CGO_ENABLED=0 builds a statically linked binary.
# docs for -ldflags at https://pkg.go.dev/cmd/link
# -w : Omit the DWARF symbol table.
# -s : Omit the symbol table and debug information.
# Omit the symbol table and debug information will reduce the binary size.
# RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-w -s" -o zinc cmd/zinc/main.go
ENV VERSION=$VERSION
ENV COMMIT_HASH=$COMMIT_HASH
ENV BUILD_DATE=$BUILD_DATE

RUN CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -ldflags="-s -w -X github.com/prabhatsharma/zinc/pkg/meta/v1.Version=${VERSION} -X github.com/prabhatsharma/zinc/pkg/meta/v1.CommitHash=${COMMIT_HASH} -X github.com/prabhatsharma/zinc/pkg/meta/v1.BuildDate=${BUILD_DATE}" -o zinc cmd/zinc/main.go
############################
# STEP 2 build a small image
############################
# FROM public.ecr.aws/lts/ubuntu:latest
FROM scratch
# Import the user and group files from the builder.
COPY --from=builder /etc/passwd /etc/passwd
COPY --from=builder /etc/group /etc/group

# Copy the ssl certificates
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt

# Copy our static executable.
COPY --from=builder /go/src/github.com/prabhatsharma/zinc/zinc /go/bin/zinc

# Use an unprivileged user.
USER appuser:appuser
# Port on which the service will be exposed.
EXPOSE 4080
# Run the zinc binary.
ENTRYPOINT ["/go/bin/zinc"]
16 changes: 10 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,32 +50,36 @@ Join slack channel

## Download / Installation / Run

Check installation [installation docs](https://docs.zincsearch.io/04_installation/)
Check installation [installation docs](https://docs.zinclabs.io/04_installation/)


## Data ingestion

### Single record

Check [single record ingestion docs](https://docs.zincsearch.io/ingestion/single-record/)
Check [single record ingestion docs](https://docs.zinclabs.io/ingestion/single-record/)

### Bulk ingestion

Check [bulk ingestion docs](https://docs.zincsearch.io/ingestion/bulk-ingestion/#bulk-ingestion)
Check [bulk ingestion docs](https://docs.zinclabs.io/ingestion/bulk-ingestion/#bulk-ingestion)

### Fluent bit

Check [fluet-bit ingestion docs](https://docs.zincsearch.io/ingestion/fluent-bit/)
Check [fluet-bit ingestion docs](https://docs.zinclabs.io/ingestion/fluent-bit/)

### Syslog-ng

Check [syslog-ng ingestion docs](https://docs.zincsearch.io/ingestion/syslog-ng/)
Check [syslog-ng ingestion docs](https://docs.zinclabs.io/ingestion/syslog-ng/)

## API Reference

Check [Zinc API docs](https://docs.zincsearch.io/API%20Reference/)
Check [Zinc API docs](https://docs.zinclabs.io/API%20Reference/)


# How to develop and contribute to Zinc

Check the [contributing guide](./CONTRIBUTING.md)

# Who uses Zinc (Known users)?

1. [Quadrantsec](https://quadrantsec.com/)
Expand Down

0 comments on commit 6785b02

Please sign in to comment.