Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/minio #9

Merged
merged 6 commits into from
Sep 20, 2021
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .chglog/CHANGELOG.tpl.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{{ range .Versions }}
<a name="{{ .Tag.Name }}"></a>
## {{ if .Tag.Previous }}[{{ .Tag.Name }}]({{ $.Info.RepositoryURL }}/compare/{{ .Tag.Previous.Name }}...{{ .Tag.Name }}){{ else }}{{ .Tag.Name }}{{ end }} ({{ datetime "2006-01-02" .Tag.Date }})

{{ range .CommitGroups -}}
### {{ .Title }}

{{ range .Commits -}}
* {{ .Subject }}
{{ end }}
{{ end -}}

{{- if .NoteGroups -}}
{{ range .NoteGroups -}}
### {{ .Title }}

{{ range .Notes }}
{{ .Body }}
{{ end }}
{{ end -}}
{{ end -}}
{{ end -}}
173 changes: 173 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@

<a name="v0.8.0"></a>
## [v0.8.0](https://github.com/neo9/mongodb-backups/compare/v0.7.0...v0.8.0) (2021-07-30)

### Config

* Add variable to choose tmpPath


<a name="v0.7.0"></a>
## [v0.7.0](https://github.com/neo9/mongodb-backups/compare/v0.6.1...v0.7.0) (2020-12-18)

### Ci

* Move on github actions

### Dump

* Fix removefile after dump

### Gs

* Add gs bucket support

### Job

* Add instant backup example

### Removefile

* Make this function exportable

### Restore

* Use configured timeout on restore


<a name="v0.6.1"></a>
## [v0.6.1](https://github.com/neo9/mongodb-backups/compare/v0.6.0...v0.6.1) (2020-08-26)

### Dump

* Remove tmp file after backup

### K8s

* Increase memory limit and request to 1Gi

### Prom

* Add alert examples


<a name="v0.6.0"></a>
## [v0.6.0](https://github.com/neo9/mongodb-backups/compare/v0.5.1...v0.6.0) (2019-08-09)

### Doc

* Add new prometheus metrics

### Pkg

* Add last successful backup timestamp


<a name="v0.5.1"></a>
## [v0.5.1](https://github.com/neo9/mongodb-backups/compare/v0.5.0...v0.5.1) (2019-08-09)

### Travis

* Set file glob to true
* Add darwin build support


<a name="v0.5.0"></a>
## [v0.5.0](https://github.com/neo9/mongodb-backups/compare/v0.4.1...v0.5.0) (2019-08-09)

### Pkg

* Add --dump support


<a name="v0.4.1"></a>
## [v0.4.1](https://github.com/neo9/mongodb-backups/compare/v0.4.0...v0.4.1) (2019-08-09)

### Doc

* Add config example
* Add S3 policy example

### K8s

* Improve deployment specs

### Pkg

* Fix metrics inconsistent label cardinality


<a name="v0.4.0"></a>
## [v0.4.0](https://github.com/neo9/mongodb-backups/compare/v0.3.0...v0.4.0) (2019-08-09)

### Pkg

* Fix current metrics types


<a name="v0.3.0"></a>
## [v0.3.0](https://github.com/neo9/mongodb-backups/compare/v0.2.0...v0.3.0) (2019-08-08)

### Pkg

* Add restore command


<a name="v0.2.0"></a>
## [v0.2.0](https://github.com/neo9/mongodb-backups/compare/v0.1.0...v0.2.0) (2019-08-06)

### Cmd

* Add port override


<a name="v0.1.0"></a>
## v0.1.0 (2019-08-02)

### Ci

* Fix docker tag
* Fix travis
* Add travis

### Doc

* Fix image names
* Add travis badge
* Add docker
* Add README

### Docker

* Fix command line name
* Add ca-certificates package
* Add Dockerfile

### Metrics

* Add basic support

### Mongodb

* Add better file cleanup

### Pkg

* Add authentication database
* Add log timeout error
* Add retention metrics
* Fix MongoDB size metrics
* Add metrics duration
* Add MongoDB logs output
* Fix metrics error
* Add retention support

### Refactor

* Add new backup logic
* Types and bucket logic

### Travis

* Add build command

26 changes: 26 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ Parameters:
- `region`: bucket region
- `gs`:
- `name`: bucket name
- `minio`:
- `name`: bucket name
- `host`: bucket hostname (and port if required)
- `region`: (optional) bucket region
- `ssl`: (optional) Enable SSL

Example:

Expand Down Expand Up @@ -87,9 +92,21 @@ All metrics have the label `name` equals to the config `name` key.

## Environment variables

### Global

- `MONGODB_USER`: MongoDB user
- `MONGODB_PASSWORD`: MongoDB password

### AWS

- `AWS_ACCESS_KEY_ID`: AWS secret key ID
- `AWS_SECRET_ACCESS_KEY`: AWS secret access key

### Minio

- `MINIO_ACCESS_KEY_ID`: Minio secret key ID
- `MINIO_SECRET_ACCESS_KEY`: Minio secret access key

## AWS

S3 policy example:
Expand Down Expand Up @@ -118,6 +135,15 @@ S3 policy example:

## Development

The changelog is generated by [git-chglog](https://github.com/git-chglog/git-chglog)

You must follow the given commit syntax: `<item>: <description>`

Generate the changelog:
```sh
git-chglog -o CHANGELOG.md
```

### Run

```bash
Expand Down
6 changes: 3 additions & 3 deletions cmd/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,16 @@ package main
import (
"flag"
"fmt"
"os"
"runtime"

"github.com/neo9/mongodb-backups/pkg/api"
"github.com/neo9/mongodb-backups/pkg/config"
"github.com/neo9/mongodb-backups/pkg/mongodb"
"github.com/neo9/mongodb-backups/pkg/restore"
"github.com/neo9/mongodb-backups/pkg/scheduler"
"github.com/neo9/mongodb-backups/pkg/utils"
log "github.com/sirupsen/logrus"
"os"
"runtime"
)

func printVersion() {
Expand Down Expand Up @@ -119,4 +120,3 @@ func main() {
log.SetFormatter(&log.JSONFormatter{})
}
}

5 changes: 3 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,17 @@ require (
github.com/golang/snappy v0.0.1 // indirect
github.com/howeyc/gopass v0.0.0-20170109162249-bf9dde6d0d2c // indirect
github.com/jessevdk/go-flags v1.4.0 // indirect
github.com/minio/minio-go/v7 v7.0.13
github.com/mongodb/mongo-tools-common v0.0.0 // indirect
github.com/prometheus/client_golang v1.1.0
github.com/prometheus/common v0.6.0
github.com/robfig/cron v1.2.0
github.com/sirupsen/logrus v1.4.2
github.com/sirupsen/logrus v1.8.1
github.com/xdg/scram v0.0.0-20180814205039-7eeb5667e42c // indirect
github.com/xdg/stringprep v1.0.0 // indirect
golang.org/x/arch v0.0.0-20190312162104-788fe5ffcd8c // indirect
google.golang.org/api v0.32.0
google.golang.org/appengine v1.6.6
gopkg.in/mgo.v2 v2.0.0-20180705113604-9856a29383ce // indirect
gopkg.in/yaml.v2 v2.2.2
gopkg.in/yaml.v2 v2.4.0
)
Loading