Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Netdata updater doesn't update to 1.42.1 #15851

Closed
jurgenhaas opened this issue Aug 21, 2023 · 9 comments
Closed

[Bug]: Netdata updater doesn't update to 1.42.1 #15851

jurgenhaas opened this issue Aug 21, 2023 · 9 comments
Labels
area/packaging Packaging and operating systems support question

Comments

@jurgenhaas
Copy link
Contributor

Bug description

Running /etc/cron.daily/netdata-updater doesn't install 1.42.1 and we can't figure out why. Here is the output:

Mon 21 Aug 09:21:14 CEST 2023 : INFO: netdata-updater:  Checking if a newer version of the updater script is available.
Mon 21 Aug 09:21:14 CEST 2023 : INFO: netdata-updater:  Entering /tmp/netdata-updater-FVv5VvC1H8
Mon 21 Aug 09:21:14 CEST 2023 : INFO: netdata-updater:  Current Version: 00104200000063
Mon 21 Aug 09:21:14 CEST 2023 : INFO: netdata-updater:  Latest Version: 00104200000063
Mon 21 Aug 09:21:14 CEST 2023 : INFO: netdata-updater:  Newest version (current=00104200000063 >= latest=00104200000063) is already installed

We've installed it with this command:

kickstart.sh --non-interactive --stable-channel --reinstall-even-if-unsafe --disable-telemetry --static-only

Expected behavior

It should update to 1.42.1

Steps to reproduce

  1. /etc/cron.daily/netdata-updater

Installation method

kickstart.sh

System info

/etc/lsb-release:DISTRIB_ID=Ubuntu
/etc/lsb-release:DISTRIB_RELEASE=22.04
/etc/lsb-release:DISTRIB_CODENAME=jammy
/etc/lsb-release:DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS"
/etc/os-release:PRETTY_NAME="Ubuntu 22.04.3 LTS"
/etc/os-release:NAME="Ubuntu"
/etc/os-release:VERSION_ID="22.04"
/etc/os-release:VERSION="22.04.3 LTS (Jammy Jellyfish)"
/etc/os-release:VERSION_CODENAME=jammy
/etc/os-release:ID=ubuntu
/etc/os-release:ID_LIKE=debian
/etc/os-release:UBUNTU_CODENAME=jammy

Netdata build info

Packaging:
    Netdata Version ____________________________________________ : v1.42.0-63-gc26af4455
    Installation Type __________________________________________ : kickstart-static
    Package Architecture _______________________________________ : x86_64
    Package Distro _____________________________________________ : unknown
    Configure Options __________________________________________ :  '--prefix=/opt/netdata/usr' '--sysconfdir=/opt/netdata/etc' '--localstatedir=/opt/netdata/var' '--libexecdir=/opt/netdata/usr/libexec' '--libdir=/opt/netdata/usr/lib' '--with-math' '--with-user=netdata' '--enable-cloud' '--without-bundled-protobuf' '--disable-dependency-tracking' '--enable-lto' 'CFLAGS=-ffunction-sections -fdata-sections -static -O2 -funroll-loops -I/openssl-static/include -I/libnetfilter-acct-static/include/libnetfilter_acct -I/usr/include/libmnl -pipe' 'LDFLAGS=-Wl,--gc-sections -static -L/openssl-static/lib -L/libnetfilter-acct-static/lib -lnetfilter_acct -L/usr/lib -lmnl' 'PKG_CONFIG=pkg-config --static' 'PKG_CONFIG_PATH=/openssl-static/lib/pkgconfig:/libnetfilter-acct-static/lib/pkgconfig:/usr/lib/pkgconfig'
Default Directories:
    User Configurations ________________________________________ : /opt/netdata/etc/netdata
    Stock Configurations _______________________________________ : /opt/netdata/usr/lib/netdata/conf.d
    Ephemeral Databases (metrics data, metadata) _______________ : /opt/netdata/var/cache/netdata
    Permanent Databases ________________________________________ : /opt/netdata/var/lib/netdata
    Plugins ____________________________________________________ : /opt/netdata/usr/libexec/netdata/plugins.d
    Static Web Files ___________________________________________ : /opt/netdata/usr/share/netdata/web
    Log Files __________________________________________________ : /opt/netdata/var/log/netdata
    Lock Files _________________________________________________ : /opt/netdata/var/lib/netdata/lock
    Home _______________________________________________________ : /opt/netdata/var/lib/netdata
Operating System:
    Kernel _____________________________________________________ : Linux
    Kernel Version _____________________________________________ : 5.15.0-79-generic
    Operating System ___________________________________________ : Ubuntu
    Operating System ID ________________________________________ : ubuntu
    Operating System ID Like ___________________________________ : debian
    Operating System Version ___________________________________ : 22.04.3 LTS (Jammy Jellyfish)
    Operating System Version ID ________________________________ : none
    Detection __________________________________________________ : /etc/os-release
Hardware:
    CPU Cores __________________________________________________ : 12
    CPU Frequency ______________________________________________ : 3200000000
    CPU Architecture ___________________________________________ : 67287355392
    RAM Bytes __________________________________________________ : 1024220381184
    Disk Capacity ______________________________________________ : x86_64
    Virtualization Technology __________________________________ : none
    Virtualization Detection ___________________________________ : systemd-detect-virt
Container:
    Container __________________________________________________ : none
    Container Detection ________________________________________ : systemd-detect-virt
    Container Orchestrator _____________________________________ : none
    Container Operating System _________________________________ : none
    Container Operating System ID ______________________________ : none
    Container Operating System ID Like _________________________ : none
    Container Operating System Version _________________________ : none
    Container Operating System Version ID ______________________ : none
    Container Operating System Detection _______________________ : none
Features:
    Built For __________________________________________________ : Linux
    Netdata Cloud ______________________________________________ : YES
    Health (trigger alerts and send notifications) _____________ : YES
    Streaming (stream metrics to parent Netdata servers) _______ : YES
    Replication (fill the gaps of parent Netdata servers) ______ : YES
    Streaming and Replication Compression ______________________ : YES (lz4)
    Contexts (index all active and archived metrics) ___________ : YES
    Tiering (multiple dbs with different metrics resolution) ___ : YES (5)
    Machine Learning ___________________________________________ : YES
Database Engines:
    dbengine ___________________________________________________ : YES
    alloc ______________________________________________________ : YES
    ram ________________________________________________________ : YES
    map ________________________________________________________ : YES
    save _______________________________________________________ : YES
    none _______________________________________________________ : YES
Connectivity Capabilities:
    ACLK (Agent-Cloud Link: MQTT over WebSockets over TLS) _____ : YES
    static (Netdata internal web server) _______________________ : YES
    h2o (web server) ___________________________________________ : NO
    WebRTC (experimental) ______________________________________ : NO
    Native HTTPS (TLS Support) _________________________________ : YES
    TLS Host Verification ______________________________________ : YES
Libraries:
    LZ4 (extremely fast lossless compression algorithm) ________ : YES
    zlib (lossless data-compression library) ___________________ : YES
    Judy (high-performance dynamic arrays and hashtables) ______ : YES (bundled)
    dlib (robust machine learning toolkit) _____________________ : YES (bundled)
    protobuf (platform-neutral data serialization protocol) ____ : YES (system)
    OpenSSL (cryptography) _____________________________________ : YES
    libdatachannel (stand-alone WebRTC data channels) __________ : NO
    JSON-C (lightweight JSON manipulation) _____________________ : YES
    libcap (Linux capabilities system operations) ______________ : NO
    libcrypto (cryptographic functions) ________________________ : YES
    libm (mathematical functions) ______________________________ : YES
    jemalloc ___________________________________________________ : NO
    TCMalloc ___________________________________________________ : NO
Plugins:
    apps (monitor processes) ___________________________________ : YES
    cgroups (monitor containers and VMs) _______________________ : YES
    cgroup-network (associate interfaces to CGROUPS) ___________ : YES
    proc (monitor Linux systems) _______________________________ : YES
    tc (monitor Linux network QoS) _____________________________ : YES
    diskspace (monitor Linux mount points) _____________________ : YES
    freebsd (monitor FreeBSD systems) __________________________ : NO
    macos (monitor MacOS systems) ______________________________ : NO
    statsd (collect custom application metrics) ________________ : YES
    timex (check system clock synchronization) _________________ : YES
    idlejitter (check system latency and jitter) _______________ : YES
    bash (support shell data collection jobs - charts.d) _______ : YES
    debugfs (kernel debugging metrics) _________________________ : YES
    cups (monitor printers and print jobs) _____________________ : NO
    ebpf (monitor system calls) ________________________________ : YES
    freeipmi (monitor enterprise server H/W) ___________________ : NO
    nfacct (gather netfilter accounting) _______________________ : YES
    perf (collect kernel performance events) ___________________ : YES
    slabinfo (monitor kernel object caching) ___________________ : YES
    Xen ________________________________________________________ : NO
    Xen VBD Error Tracking _____________________________________ : NO
Exporters:
    AWS Kinesis ________________________________________________ : NO
    GCP PubSub _________________________________________________ : NO
    MongoDB ____________________________________________________ : NO
    Prometheus (OpenMetrics) Exporter __________________________ : YES
    Prometheus Remote Write ____________________________________ : YES
    Graphite ___________________________________________________ : YES
    Graphite HTTP / HTTPS ______________________________________ : YES
    JSON _______________________________________________________ : YES
    JSON HTTP / HTTPS __________________________________________ : YES
    OpenTSDB ___________________________________________________ : YES
    OpenTSDB HTTP / HTTPS ______________________________________ : YES
    All Metrics API ____________________________________________ : YES
    Shell (use metrics in shell scripts) _______________________ : YES
Debug/Developer Features:
    Trace All Netdata Allocations (with charts) ________________ : NO
    Developer Mode (more runtime checks, slower) _______________ : NO

Additional info

No response

@jurgenhaas jurgenhaas added bug needs triage Issues which need to be manually labelled labels Aug 21, 2023
@ilyam8 ilyam8 added area/packaging Packaging and operating systems support question and removed bug needs triage Issues which need to be manually labelled labels Aug 21, 2023
@ilyam8
Copy link
Member

ilyam8 commented Aug 21, 2023

Hi, @jurgenhaas. ACtually v1.42.0-63 > v1.42.1, so you are good (you are using nightly channel btw). This is confusing, indeed - see #14851. There is a feature request to fix the problem - #14855.

@ilyam8 ilyam8 closed this as completed Aug 21, 2023
@jurgenhaas
Copy link
Contributor Author

@ilyam8 hmm, strange. How can I be on nightly if I installed with kickstart.sh --non-interactive --stable-channel --reinstall-even-if-unsafe --disable-telemetry --static-only ?

@ilyam8
Copy link
Member

ilyam8 commented Aug 21, 2023

@jurgenhaas I think a bug in our installer - it sets the wrong RELEASE_CHANNEL in /opt/netdata/etc/netdata/.environment. Let me check.

@jurgenhaas
Copy link
Contributor Author

Confirmed, that's what it is. Even if I set the channel to stable there and run the updater, it does successfully update to 1.42.1 but resets the release channel to nightly in the environment file.

@ilyam8
Copy link
Member

ilyam8 commented Aug 21, 2023

@jurgenhaas I think I found the problem #15854

@jurgenhaas
Copy link
Contributor Author

Oh, so it was commented out? Do we get that change in one of the next nightlies and can then update all our .environment files so that they remain unchanged? Ah, maybe not, because if it then installs stable again, that fix won't be there any longer and will fall back to nightly again. Sounds like we have to wait until that fix lands in another stable, right?

@ilyam8
Copy link
Member

ilyam8 commented Aug 21, 2023

It wasn't commented, but the sed command didn't work for the install-or-update.sh (static build) file, so the RELEASE_CHANNEL was always nightly.

@ilyam8
Copy link
Member

ilyam8 commented Aug 21, 2023

Sounds like we have to wait until that fix lands in another stable, right?

  1. Change manually RELEASE_CHANNEL to stable.
  2. Run the updater script (it will install v1.42.1)
  3. Change manually RELEASE_CHANNEL to stable again.

@netdata-community-bot
Copy link

This issue has been mentioned on the Netdata Community Forums. There might be relevant details there:

https://community.netdata.cloud/t/cloud-shows-false-alarm-about-outdated-agent-versions/3187/7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/packaging Packaging and operating systems support question
Projects
None yet
Development

No branches or pull requests

3 participants