Documentation Addition for Creating a service

[MaxDiOrio]

First - VM is amazing so far. Thank you.

I'd love to see you add documentation on running VM as a service in systemd. Here's what I did to make it work on CentOS 7, but this should work on any systemd OS.

Create a directory to hold the PID file:

mkdir /run/victoriametrics

Create /etc/systemd/system/victoriametrics.service. Copy and paste the below into this new file, adjusting the ExecStart line as needed.

[Unit]
Description=VictoriaMetrics
After=network.target

[Service]
Type=simple
StartLimitBurst=5
StartLimitInterval=0
Restart=on-failure
RestartSec=1
PIDFile=/run/victoriametrics/victoriametrics.pid
ExecStart=/usr/local/bin/victoriametrics -storageDataPath /data -retentionPeriod 6
ExecStop=/bin/kill -s SIGTERM $MAINPID

[Install]
WantedBy=multi-user.target

Set the file limits for the service:
mkdir /etc/systemd/system/victoriametrics.service.d
In this folder create a file ulimit.conf with the following:

[Service]
LimitNOFILE=32000
LimitNPROC=32000

Enable the service to start automatically:

systemctl enable victoriametrics

Start the service:

systemctl start victoriametrics

[valyala]

@MaxDiOrio , thanks for these instructions! I prettified your message and linked it from the README.md. Feel free updating this info as needed.

[mihaibalaci]

Quick question: if we deploy it in aks/eks is it possible to store data in something like s3 or adls?

[valyala]

Quick question: if we deploy it in aks/eks is it possible to store data in something like s3 or adls?

Currently the data may be stored in network block storage such as Amazon EBS or Google compute disks. There are plans for adding blob storage support such as S3 or GCS in the future - see this issue for details.

[kmansoft]

If anyone is interested, I can add build system scripts to create .rpm packages (did this recently for my project, went from "know nothing about RPM" to "know enough to be dangerous").

This would cover CentOS / Fedora, at least those versions that use systemd.

[valyala]

If anyone is interested, I can add build system scripts to create .rpm packages (did this recently for my project, went from "know nothing about RPM" to "know enough to be dangerous").

This sounds good. Please, coordinate your work with @patsevanton regarding his yum build for VictoriaMetrics.

[kmansoft]

@patsevanton I'm thinking of adding new code to the makefiles / build scripts.

It would dynamically generate the .spec file with the right paths (which do vary by version and by actual built path) and then use rpmbuild -bb --target to package.

Here is some code from my own build scripts:

            mkdir -p "${TEMPDIR_RPM}/${WHAT}/SPECS"
            cat > "${TEMPDIR_RPM}/${WHAT}/SPECS/clearview-${WHAT}.spec" <<EOF
Summary: Clearview agent
Name: clearview-agent
Version: ${VERSION}
Release: ${BUILD}
License: n/a
URL: https://clearview.rocks
Group: System
Packager: Kostya Vasilyev <kmansoft@gmail.com>
Requires: libpthread
Requires: libc

%description
Clearview is an easy to install, easy to use performance monitoring for
your Linux web servers. Inspired by Linode Longview.

%files
%attr(0744, root, root) /usr/sbin/*
%attr(0644, root, root) /lib/systemd/system/*

%prep
echo "BUILDROOT = \$RPM_BUILD_ROOT"

mkdir -p \$RPM_BUILD_ROOT/usr/sbin/
mkdir -p \$RPM_BUILD_ROOT/lib/systemd/system/

cp ${PWD}/package/temp-deb/agent/usr/sbin/${OUT_EXE} \$RPM_BUILD_ROOT/usr/sbin/
cp ${PWD}/package/agent/clearview-agent.service \$RPM_BUILD_ROOT/lib/systemd/system/
EOF

            cat "${TEMPDIR_RPM}/${WHAT}/SPECS/clearview-${WHAT}.spec"

            rpmbuild -bb --target "${RPM_ARCH}" \
                "${TEMPDIR_RPM}/${WHAT}/SPECS/clearview-${WHAT}.spec"

            cp "${HOME}/rpmbuild/RPMS/${RPM_ARCH}/${OUT_RPM}" "${PACKDIR}/${OUT_RPM}"

[patsevanton]

@kmansoft Can you create PR to https://github.com/patsevanton/victoriametrics-rpm
Thanks!

[kmansoft]

@patsevanton Hmmmm... I was thinking adding it to the existing build script(s) which build .deb packages, and therefore need to fork it off the complete Victoria repository.

[kmansoft]

Also @patsevanton can we drop support for non-systemd systems?

[patsevanton]

@kmansoft i think first merge #121

I dont know about drop support for non-systemd systems. Question to @valyala

[kmansoft]

RHEL 7 (has systemd) came out in 2014

For Debian, it's systemd only so that's "wheezy" and newer (I think), 2016 release date.

[kmansoft]

@kmansoft i think first merge #121

Got it, great, once that's done I'll fork the main repo again.

[dswarbrick]

Is there some particular reason why the LimitNOFILE and LimitNPROC are not part of the main service file?

Also, maybe it's not a great idea to run the service as root, if it's not really necessary.

[kmansoft]

Is there some particular reason why the LimitNOFILE and LimitNPROC are not part of the main service file?

If by "main" you mean the one for Debian - no reason, just oversight on my part.

Also, maybe it's not a great idea to run the service as root, if it's not really necessary.

Agreed. Maybe already fixed in the RPM packages, but not for DEB.

The install script will need to chown any already existing data files.

[dswarbrick]

A common pattern used by the various Prometheus packages on Debian, which require the ability to override command line settings, is to source variables from an external config file.

For example:

$ cat /etc/default/prometheus
# Set the command-line arguments to pass to the server.
ARGS="--storage.tsdb.retention.time=30d --query.max-samples=10000000"

$ systemctl cat prometheus
# /lib/systemd/system/prometheus.service
[Unit]
Description=Monitoring system and time series database
Documentation=https://prometheus.io/docs/introduction/overview/

[Service]
Restart=always
User=prometheus
EnvironmentFile=/etc/default/prometheus
ExecStart=/usr/bin/prometheus $ARGS
ExecReload=/bin/kill -HUP $MAINPID
TimeoutStopSec=20s
SendSIGKILL=no
LimitNOFILE=8192

The EnvironmentFile in the systemd unit, and passing $ARGS to the ExecStart allows you to use a one-size-fits-all unit, whilst still allowing customization of options.

[agarthetiger]

There is also an Ansible Role on GitHub which installs a single VictoriaMetrics instance as a service which I came across today along with this issue. It is at https://github.com/dreamteam-gg/ansible-victoriametrics-role in case it's useful to anyone else who lands here.

[valyala]

There is also https://github.com/VictoriaMetrics/ansible-playbooks

[mactyr]

I think there's a confusing bit of formatting in the initial post here. This should be between two code blocks, not inside a code block, right?

Set the file limits for the service:
mkdir /etc/systemd/system/victoriametrics.service.d
In this folder create a file ulimit.conf with the following:

Actually, I'm not sure that part is necessary at all -- I tried just putting

LimitNOFILE=32000
LimitNPROC=32000

in the [Service] section in my main /etc/systemd/system/victoriametrics.service file and systemd seemed to accept that, so I'm not sure a separate ulimit.conf is necessary at all? I am definitely not a systemd expert, so don't take my word for it...

[dswarbrick]

@mactyr Yes, you are correct about the comments / instructions that have erroneously been included in the code block (which should be two separate blocks).

Systemd allows you to override portions of a unit by placing snippets in /etc/systemd/system/<servicename>.service.d, known as "drop-ins". If you systemctl edit <servicename>, it will create an override.conf in such a directory.

Since 32000 is already a pretty generous FD limit, I doubt there will be that many people who would need to override it with higher limits, so those params could probably just reside in the main service unit.

[denisgolius]

Hi all!
@patsevanton and I have prepared config files for VictoriaMetrics Single, VictoriaMetrics Cluster and vmutils for systemd daemon. You can find them in https://github.com/patsevanton/victoriametrics-rpm. Also this repository is always updated to the latest version of VictoriaMetrics.

PS: we added publishing build for freebsd and openbsd, see https://docs.victoriametrics.com/CHANGELOG.html#v1790
FEATURE: publish binaries for FreeBSD and OpenBSD at releases page.

[denisgolius]

Closing this issue as done.

[erielias]

have you seen a similar setup for vmagent? I wasn't able to find it in the docs...