Skip to content

[FEATURE] - Get binary release for major platforms #25

Open
gtherond opened this issue Oct 12, 2021 · 13 comments
Open

[FEATURE] - Get binary release for major platforms #25

gtherond opened this issue Oct 12, 2021 · 13 comments
Assignees
Labels
enhancement New feature or request

Comments

@gtherond
Copy link

Is your feature request related to a problem? Please describe.
Would it be possible to build binary release for Linux x86_64/aarch64?

Describe the solution you'd like
As for this exporter: https://github.com/openstack-exporter/openstack-exporter/releases
I'd like to be able to download an archive of the statically build exporter for linux.

Describe alternatives you've considered
I'll have to get a two step build phase as our platform is an airgapped platform that can't reach internet and so can't build correctly the exporter without cloning the whole github platform.

Additional context
Any offline platform won't be able to build this exporter as dependencies are refering to github or internet hosted repos.

@gtherond gtherond added the enhancement New feature or request label Oct 12, 2021
@AlexZzz
Copy link
Contributor

AlexZzz commented Oct 13, 2021

Hi, thanks for the suggestion!
There's a problem with this exporter: it's not a statically built binary. There's no good way to build it statically, because libvirt-go doesn't support that. Static linking has been used earlier, but there were a lot of building issues and runtime problems like memory leaks, so I decided to throw it away.

It's better to use a container image. I'm going to push it to dockerhub sooner or later. During the build process exporter is linked with with latest libvirt form golang:alpine image. But, it still works with currently installed in the host system libvirtd daemon. There might be some compatibility issues, but I hope that they are caught somehow in communication logic in libvirt:).

There's a way to build it with dynamic linking for your distro. There's a build-with script for it and some examples for Ubuntu-16.04 -- 20.04 which are used in our production.

So, which is better suite for your case? I can add some CI for this repo to publish distro-specific builds, if it is really useful. But, i thought that the docker image is more popular these days.

@gtherond
Copy link
Author

gtherond commented Oct 13, 2021

Oh! Actually using your docker image as a base for our build process (Openstack kolla) would be a great idea!
Thanks a lot for the information about the libvirt-go that can't be used with a statical build process!

@Ubun1
Copy link
Contributor

Ubun1 commented Nov 4, 2021

maybe there is a way to use https://github.com/digitalocean/go-libvirt - as a substitution for libvirt-go ? lib from digital ocean doesn't have C bindings .

@AlexZzz
Copy link
Contributor

AlexZzz commented Nov 10, 2021

@Ubun1 thanks for pointing that out. Yes, this might be a way to go. But I assume it is a huge rewrite and I'm not sure if it supports all the metrics libvirt-exporter exposes.

I'll take a look at it when I have some more free time.

@Ubun1
Copy link
Contributor

Ubun1 commented Nov 11, 2021

function libvirt-go function invocation libvirt api go-libvirt (digital ocean)
collectFromLibvirt NewConnectReadOnly virConnectOpenReadOnly Connect (?)
GetVersion virConnectGetVersion ConnectGetVersion
GetLibVersion virConnectGetLibVersion ConnectGetLibVersion
GetVersion virGetVersion Version
Free virDomainFree -
GetAllDomainStats virConnectGetAllDomainStats Domains
------------------ ------------------------------ --------------------------- -----------------------------
collectDomain GetName virDomainGetName domain.Name
GetUUIDString virDomainGetUUIDString domain.UUID
GetXMLDesc virDomainGetXMLDesc DomainGetXMLDesc
GetInfo virDomainGetInfo DomainGetInfo
GetVcpus virDomainGetVcpus DomainGetVcpus
GetBlockIoTune virDomainGetBlockIoTune DomainGetBlockIOTune
MemoryStats virDomainMemoryStats DomainGetMemoryParameters (?)
------------------ ------------------------------ --------------------------- -----------------------------

so, this is the table with mappings between libvirt-go functions which is used in this project and go-libvirt functions (maybe it's not complete). I'm starting to gradually rewriting invocation of library functions and compare results by running to exporters - one unchanged (build with libvirt-go) and one updated (build with go-libvirt). So far, for libvirt_versions_info metric similarity tests passed )

@AlexZzz
Copy link
Contributor

AlexZzz commented Nov 13, 2021

@gtherond , I've released version 2.2.1 which is available on dockerhub.

@AlexZzz
Copy link
Contributor

AlexZzz commented Nov 13, 2021

@Ubun1 thanks a lot for this! Would you be able to prepare a PR with these changes?
btw, what version of libvirtd do you use during tests?

@frittentheke
Copy link

frittentheke commented Nov 22, 2021

@AlexZzz @Ubun1 moving to https://github.com/digitalocean/go-libvirt is a great idea!
If you read https://github.com/digitalocean/go-libvirt#libvirt you'll see that most of the code there is auto-generated, so it will stay current with updates to libvirt itself. Pure go is just much more approachable and portable and with the data model and marshaling being mostly the same this is a big change yes, but certainly not a rewrite.

BTW, I've been using https://github.com/zhangjianweibj/prometheus-libvirt-exporter for a while which also makes use of go-libvirt and is yet another implementation of an exporter. It works really well and I was about to start extending it with more info metrics for Nova + OS. But having your (more) widely used libvirt exporter as pure go would just be awesome. Especially when considering the rest of the ecosystem: metric naming, info_metrics, Grafana Dashboards, Prometheus Alerts, ...

@Ubun1 - please let me know if you need any help with testing your PR.

@gtherond
Copy link
Author

Thanks a lot everyone about this issue, that's really awesome, and yeah, I assume using go-libvirt would be cool but will require a slightly large rewrite as the logics seems not really the same.

@Ubun1
Copy link
Contributor

Ubun1 commented Nov 23, 2021

@Ubun1 thanks a lot for this! Would you be able to prepare a PR with these changes? btw, what version of libvirtd do you use during tests?

7.0.0-x from fedora 34 repos

@frittentheke
Copy link

@Ubun1 I am, as stated above (#25 (comment)), really looking forward to this change.

Are you still working on this? Could you maybe provide a WiP PR for me to build and test already?

@frittentheke
Copy link

@Ubun1 @AlexZzz is this change still something you consider doing? Having a more portable exporter would just be awesome! May I kindly ask again if there is WiP code that you need help reviewing or testing?

@AlexZzz
Copy link
Contributor

AlexZzz commented Mar 10, 2023

@frittentheke, sorry, i don't support this project actively anymore, as i don't work on cloud infrastructure:(
New maintaner is @k3nny0ne, i hope he can help here

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants