Skip to content

Commit

Permalink
feat: move docker folder to container
Browse files Browse the repository at this point in the history
  • Loading branch information
rahulguptajss committed Apr 5, 2023
1 parent 0e9cde6 commit 0024698
Show file tree
Hide file tree
Showing 19 changed files with 27 additions and 27 deletions.
10 changes: 5 additions & 5 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -742,7 +742,7 @@ The Unix collector is unable to monitor pollers running in containers. See [#249

- New `include_contains` plugin allows you to select a subset of objects. e.g. selecting only volumes with custom-defined ONTAP metadata

- We've included more out-of-the-box [Prometheus alerts](https://github.com/NetApp/harvest/blob/main/docker/prometheus/alert_rules.yml). Keep sharing your most useful alerts!
- We've included more out-of-the-box [Prometheus alerts](https://github.com/NetApp/harvest/blob/main/container/prometheus/alert_rules.yml). Keep sharing your most useful alerts!

- 7mode workflows continue to be improved :heart: Harvest now collects Qtree and Quotas counters from 7mode filers (these are already collected in cDOT)

Expand Down Expand Up @@ -775,7 +775,7 @@ The Unix collector is unable to monitor pollers running in containers. See [#249

- Harvest should provide a `include_contains` label agent plugin for filtering #735 Thanks to @chadpruden for reporting

- Improve Harvest's container compatibility with K8s via kompose. [#655](https://github.com/NetApp/harvest/pull/655) See [also](https://github.com/NetApp/harvest/tree/main/docker/k8) and [discussion](https://github.com/NetApp/harvest/discussions/827)
- Improve Harvest's container compatibility with K8s via kompose. [#655](https://github.com/NetApp/harvest/pull/655) See [also](https://github.com/NetApp/harvest/tree/main/container/k8) and [discussion](https://github.com/NetApp/harvest/discussions/827)

- The ZAPI cli tool should include counter types when querying ZAPIs #663

Expand All @@ -786,13 +786,13 @@ The output of one plugin can be fed into the input of the next one. #736 Thanks

- Harvest should collect Antivirus counters when ONTAP offbox vscan is configured [#346](https://github.com/NetApp/harvest/issues/346) Thanks to @burkl and @Falcon667 for reporting

- [Document](https://github.com/NetApp/harvest/tree/main/docker/containerd) how to run Harvest with `containerd` and `Rancher`
- [Document](https://github.com/NetApp/harvest/tree/main/container/containerd) how to run Harvest with `containerd` and `Rancher`

- Qtree counters should be collected for 7-mode filers #766 Thanks to @jmg011 for raising this issue and iterating with us on a solution

- Harvest admin node should work with pollers running in Docker compose [#678](https://github.com/NetApp/harvest/pull/678)

- [Document](https://github.com/NetApp/harvest/tree/main/docker/podman) how to run Harvest with Podman. Several RHEL customers asked about this since Podman ships as the default container runtime on RHEL8+.
- [Document](https://github.com/NetApp/harvest/tree/main/container/podman) how to run Harvest with Podman. Several RHEL customers asked about this since Podman ships as the default container runtime on RHEL8+.

- Harvest should include a Systemd service file for the HTTP service discovery admin node [#656](https://github.com/NetApp/harvest/pull/656)

Expand Down Expand Up @@ -1150,7 +1150,7 @@ On RHEL and Debian, the example Unix collector does not work at the moment due t

- Improve client certificate authentication [documentation](https://github.com/NetApp/harvest/issues/314#issuecomment-882120238)

- Improve Docker deployment documentation [4019308](https://github.com/NetApp/harvest/tree/main/docker/onePollerPerContainer)
- Improve Docker deployment documentation [4019308](https://github.com/NetApp/harvest/tree/main/container/onePollerPerContainer)

### Fixes

Expand Down
4 changes: 2 additions & 2 deletions cmd/tools/generate/generate.go
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,7 @@ func generateDocker(path string, kind int) {
filesd = append(filesd, fmt.Sprintf("- targets: ['%s:%d']", pollerInfo.ServiceName, pollerInfo.Port))
}

t, err := template.New("docker-compose.tmpl").ParseFiles("docker/onePollerPerContainer/docker-compose.tmpl")
t, err := template.New("docker-compose.tmpl").ParseFiles("container/onePollerPerContainer/docker-compose.tmpl")
if err != nil {
panic(err)
}
Expand Down Expand Up @@ -404,7 +404,7 @@ func init() {

dFlags.BoolVarP(&opts.showPorts, "port", "p", false, "Expose poller ports to host machine")
_ = dockerCmd.MarkPersistentFlagRequired("output")
fFlags.StringVar(&opts.filesdPath, "filesdpath", "docker/prometheus/harvest_targets.yml",
fFlags.StringVar(&opts.filesdPath, "filesdpath", "container/prometheus/harvest_targets.yml",
"Prometheus file_sd target path. Written when the --output is set")
fFlags.IntVar(&opts.promPort, "promPort", 9090, "Prometheus Port")
fFlags.IntVar(&opts.grafanaPort, "grafanaPort", 3000, "Grafana Port")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,12 @@ After Rancher Desktop installs, start it `Cmd + Space` type: Rancher and wait fo

## Create Harvest Docker image

You only need to [create a new image](https://github.com/NetApp/harvest/tree/main/docker/onePollerPerContainer#building-harvest-docker-image) if you've made changes to Harvest. If you just want to use the latest version of Harvest, skip this step.
You only need to [create a new image](https://github.com/NetApp/harvest/tree/main/container/onePollerPerContainer#building-harvest-docker-image) if you've made changes to Harvest. If you just want to use the latest version of Harvest, skip this step.

These are the same steps outline on [Building Harvest Docker Image](https://github.com/NetApp/harvest/tree/main/docker/onePollerPerContainer#building-harvest-docker-image) except we replace `docker build` with `nerdctl` like so:
These are the same steps outline on [Building Harvest Docker Image](https://github.com/NetApp/harvest/tree/main/container/onePollerPerContainer#building-harvest-docker-image) except we replace `docker build` with `nerdctl` like so:

```sh
nerdctl build -f docker/onePollerPerContainer/Dockerfile -t harvest:latest . --no-cache
nerdctl build -f container/onePollerPerContainer/Dockerfile -t harvest:latest . --no-cache
```

## Generate a Harvest compose file
Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ You only need to build a Harvest Docker image if you want a custom image. If ins
Build Docker Image

```
docker build -f docker/onePollerPerContainer/Dockerfile -t harvest:latest . --no-cache
docker build -f container/onePollerPerContainer/Dockerfile -t harvest:latest . --no-cache
or
nerdctl build -f docker/onePollerPerContainer/Dockerfile -t harvest:latest . --no-cache
nerdctl build -f container/onePollerPerContainer/Dockerfile -t harvest:latest . --no-cache
```
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion deb/build-deb.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ mkdir -p "$BUILD/opt/harvest/bin/"
cp -r "$SRC/grafana" "$SRC/conf" "$BUILD/opt/harvest/"
cp "$SRC/harvest.yml" "$SRC/prom-stack.tmpl" "$SRC/harvest.cue" "$BUILD/opt/harvest/"
cp -r "$SRC/.github/" "$BUILD/opt/harvest/"
cp -r "$SRC/pkg/" "$SRC/cmd/" "$SRC/docs/" "$SRC/docker/" "$BUILD/opt/harvest/"
cp -r "$SRC/pkg/" "$SRC/cmd/" "$SRC/docs/" "$SRC/container/" "$BUILD/opt/harvest/"
cp -r "$SRC/rpm/" "$SRC/deb/" "$SRC/service/" "$SRC/cert/" "$SRC/autosupport/" "$SRC/.git" "$BUILD/opt/harvest/"
cp "$SRC/Makefile" "$SRC/README.md" "$SRC/LICENSE" "$SRC/go.mod" "$SRC/go.sum" "$BUILD/opt/harvest/"
if [ -d "$SRC/vendor" ]; then
Expand Down
8 changes: 4 additions & 4 deletions docs/install/containers.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Harvest is container-ready and supports several deployment options:
- [Stand-up Prometheus, Grafana, and Harvest via Docker Compose](#docker-compose). Choose this if
you want to hit the ground running. Install, volume and network mounts automatically handled.

- [Poller-per-container model](https://github.com/NetApp/harvest/tree/main/docker/onePollerPerContainer) that offers
- [Poller-per-container model](https://github.com/NetApp/harvest/tree/main/container/onePollerPerContainer) that offers
more flexibility in configuration. This deployment enables a broad range of orchestrators (Nomad, Mesosphere, Swarm,
K8, etc.) since you pick-and-choose what gets built and how it's deployed, stronger familiarity with containers is
recommended.
Expand All @@ -15,10 +15,10 @@ Harvest is container-ready and supports several deployment options:
stands up Harvest, Grafana, and Prometheus.

- Want to run Harvest on a Mac
via [containerd and Racher Desktop](https://github.com/NetApp/harvest/tree/main/docker/containerd)? We got you
via [containerd and Racher Desktop](https://github.com/NetApp/harvest/tree/main/container/containerd)? We got you
covered.

- [Local K8 Deployment](https://github.com/NetApp/harvest/blob/main/docker/k8/README.md) via Kompose
- [Local K8 Deployment](https://github.com/NetApp/harvest/blob/main/container/k8/README.md) via Kompose

## Docker Compose

Expand Down Expand Up @@ -80,7 +80,7 @@ bin/harvest generate docker full --output harvest-compose.yml

1. Creates a Docker compose file with a container for each Harvest poller defined in your `harvest.yml`
2. Creates a matching Prometheus service discovery file for each Harvest poller (located
in `docker/prometheus/harvest_targets.yml`). Prometheus uses this file to scrape the Harvest pollers.
in `container/prometheus/harvest_targets.yml`). Prometheus uses this file to scrape the Harvest pollers.

### Start everything

Expand Down
4 changes: 2 additions & 2 deletions docs/prometheus-exporter.md
Original file line number Diff line number Diff line change
Expand Up @@ -303,8 +303,8 @@ Prometheus includes out-of-the-box support for simple alerting. Alert rules are
file. Setup and details can be found in the Prometheus
guide on [alerting](https://prometheus.io/docs/practices/alerting/).

Harvest also includes [ems alerts](https://github.com/NetApp/harvest/blob/main/docker/prometheus/ems_alert_rules.yml)
and [sample alerts](https://github.com/NetApp/harvest/blob/main/docker/prometheus/alert_rules.yml) for reference.
Harvest also includes [ems alerts](https://github.com/NetApp/harvest/blob/main/container/prometheus/ems_alert_rules.yml)
and [sample alerts](https://github.com/NetApp/harvest/blob/main/container/prometheus/alert_rules.yml) for reference.
Refer [EMS Collector](https://github.com/NetApp/harvest/blob/main/cmd/collectors/ems/README.md) for more details about
EMS events.

Expand Down
6 changes: 3 additions & 3 deletions integration/Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ pipeline {
echo "Installing"
export BUILD_PATH=$NATIVE
export VERSION=$VERSION
cp /u/mpeg/harvest/harvest_targets.yml $WORKSPACE/harvest/docker/prometheus/
cp /u/mpeg/harvest/harvest_targets.yml $WORKSPACE/harvest/container/prometheus/
bash $WORKSPACE/harvest/integration/test/test.sh install_native
'''
}
Expand Down Expand Up @@ -104,7 +104,7 @@ pipeline {
echo "Installing"
export BUILD_PATH=http://harvest-jenkins.rtp.openenglab.netapp.com:8080/job/harvest2_0/job/artifcats/lastSuccessfulBuild/artifact/harvest-21.11.0-1.x86_64.rpm
export VERSION=$VERSION
cp /u/mpeg/harvest/harvest_targets.yml $WORKSPACE/harvest/docker/prometheus/
cp /u/mpeg/harvest/harvest_targets.yml $WORKSPACE/harvest/container/prometheus/
bash $WORKSPACE/harvest/integration/test/test.sh install_rpm
'''
}
Expand Down Expand Up @@ -165,7 +165,7 @@ pipeline {
./bin/harvest stop
./bin/harvest generate docker full --port --output harvest-compose.yml
docker login -u $DOCKER_USER -p $DOCKER_PWD
docker build -f docker/onePollerPerContainer/Dockerfile -t rahulguptajss/harvest:latest -t cr.netapp.io/harvest:latest -t ghcr.io/netapp/harvest:latest . --no-cache --build-arg VERSION=$VERSION
docker build -f container/onePollerPerContainer/Dockerfile -t rahulguptajss/harvest:latest -t cr.netapp.io/harvest:latest -t ghcr.io/netapp/harvest:latest . --no-cache --build-arg VERSION=$VERSION
docker-compose -f prom-stack.yml -f harvest-compose.yml up -d --remove-orphans
sleep 1m
bash $WORKSPACE/harvest/integration/test/test.sh install_docker
Expand Down
2 changes: 1 addition & 1 deletion integration/test/installer/prometheus.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ func (p *Prometheus) Install() bool {
ipAddress := utils.GetOutboundIP()
cmd := exec.Command("docker", "run", "-d", "-p", utils.PrometheusPort+":"+utils.PrometheusPort,
"--add-host=localhost:"+ipAddress,
"-v", path+"/../../docker/prometheus/:/etc/prometheus/",
"-v", path+"/../../container/prometheus/:/etc/prometheus/",
p.image)
cmd.Stdout = os.Stdout
err := cmd.Start()
Expand Down
4 changes: 2 additions & 2 deletions jenkins/artifacts/jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ pipeline {
sh '''
targetLocation=$targetParentLocation$VERSION-$RELEASE-$BRANCH
docker login
docker build -f docker/onePollerPerContainer/Dockerfile --build-arg VERSION=$VERSION --build-arg RELEASE=$RELEASE --build-arg ASUP_MAKE_TARGET=$ASUP_MAKE_TARGET --build-arg GIT_TOKEN=$GIT_TOKEN -t ${imageName}:latest -t ${imageName}:$VERSION-$RELEASE -t ${jfrogImagePrefix}:latest -t ${jfrogImagePrefix}:$VERSION-$RELEASE -t ${ghcrImage}:latest -t ${ghcrImage}:$VERSION-$RELEASE . --no-cache
docker build -f container/onePollerPerContainer/Dockerfile --build-arg VERSION=$VERSION --build-arg RELEASE=$RELEASE --build-arg ASUP_MAKE_TARGET=$ASUP_MAKE_TARGET --build-arg GIT_TOKEN=$GIT_TOKEN -t ${imageName}:latest -t ${imageName}:$VERSION-$RELEASE -t ${jfrogImagePrefix}:latest -t ${jfrogImagePrefix}:$VERSION-$RELEASE -t ${ghcrImage}:latest -t ${ghcrImage}:$VERSION-$RELEASE . --no-cache
docker save -o ${targetLocation}/docker_harvest.tar ${imageName}:latest
'''
}
Expand Down Expand Up @@ -221,7 +221,7 @@ pipeline {
git push https://$GIT_TOKEN@github.com/NetApp/harvest.git --delete nightly
fi
/opt/home/gh_2.8.0_linux_386/bin/gh release create nightly $targetLocation/*.rpm $targetLocation/*.deb $targetLocation/*.gz --notes "Nightly builds may include bugs and other issues. You might want to use the stable releases instead." --title "Harvest Nightly Release" --prerelease --target main
docker build -f docker/onePollerPerContainer/Dockerfile --build-arg VERSION=$VERSION --build-arg RELEASE=$RELEASE --build-arg ASUP_MAKE_TARGET=$ASUP_MAKE_TARGET --build-arg GIT_TOKEN=$GIT_TOKEN -t ${imageName}:latest -t ${imageName}:nightly -t ${jfrogImagePrefix}:latest -t ${jfrogImagePrefix}:nightly -t ${ghcrImage}:latest -t ${ghcrImage}:nightly . --no-cache
docker build -f container/onePollerPerContainer/Dockerfile --build-arg VERSION=$VERSION --build-arg RELEASE=$RELEASE --build-arg ASUP_MAKE_TARGET=$ASUP_MAKE_TARGET --build-arg GIT_TOKEN=$GIT_TOKEN -t ${imageName}:latest -t ${imageName}:nightly -t ${jfrogImagePrefix}:latest -t ${jfrogImagePrefix}:nightly -t ${ghcrImage}:latest -t ${ghcrImage}:nightly . --no-cache
echo $GIT_TOKEN | docker login ghcr.io -u $GIT_USERNAME --password-stdin
docker push ${ghcrImage}:nightly
'''
Expand Down
2 changes: 1 addition & 1 deletion prom-stack.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ services:
container_name: prometheus
image: prom/prometheus:v2.33.1
volumes:
- ./docker/prometheus/:/etc/prometheus/
- ./container/prometheus/:/etc/prometheus/
- prometheus_data:/prometheus
command:
- '--config.file=/etc/prometheus/prometheus.yml'
Expand Down
2 changes: 1 addition & 1 deletion rpm/build-rpm.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ mkdir -p "$BUILD/harvest/bin"
cp -r "$SRC/.git" "$BUILD/harvest"
cp -r "$SRC/cmd/" "$BUILD/harvest/"
cp -r "$SRC/conf/" "$BUILD/harvest/"
cp -r "$SRC/docker/" "$BUILD/harvest/"
cp -r "$SRC/container/" "$BUILD/harvest/"
cp -r "$SRC/docs/" "$BUILD/harvest/"
cp -r "$SRC/grafana/" "$BUILD/harvest/"
cp -r "$SRC/pkg/" "$BUILD/harvest/"
Expand Down

0 comments on commit 0024698

Please sign in to comment.