Field Notes

Field notes

Closed reports from the tracker, grouped by what you would actually observe. All are fixed: check your version before debugging, because most of these look like configuration errors and are not.

For anything about wrong usage figures, read How usage is measured first. For what is mature and what is not, see Feature status.

Containers never scale down

Seemingly below the lower memory threshold, and sometimes scaled back up right after being manually scaled down. The daemon was counting reclaimable page cache as used memory. Reported with the diagnosis in #51.

ValueError: invalid literal for int() with base 10: ''

An empty string where a number was expected: a command returned nothing and the result was parsed anyway. It kills the main loop rather than skipping the container, so one bad reading stops the whole cycle. Reported repeatedly, #9, #14, #16.

/bin/sh: 1: Syntax error: ")" unexpected

Command 'free -m | awk '/^MemTotal:/ {print $2}')' failed

A malformed shell command, note the unbalanced parenthesis, built and sent as-is. Reported in #17. Fixed, and largely made moot since usage moved to cgroup reads instead of shelling out.

Ignored containers are scaled anyway

ignore_lxc stopped being honoured, so containers deliberately excluded, typically Docker hosts, were resized. Reported in #19. If you rely on the exclusion list, confirm in the log that the daemon reports the set it is ignoring at the start of each cycle.

Tier settings do not apply

A tier looks configured and the container is scaled with the defaults instead. Reported in #13 and #25, the latter with no tier applied and no cores added.

This family has a modern relative worth knowing about: in horizontal scaling, a documented key was read under a different name in code and silently discarded (#71). When a setting appears to do nothing, check that the daemon logs it back, rather than assuming it was accepted.

Every container jumps to 100% CPU on each cycle

Observed at every five-minute run, with all containers pinned at 100%. Reported in #23, a measurement artefact rather than real load, but one the daemon would have acted on.

cpu_pinning is set in a tier and nothing is pinned

Or worse, the container is pinned to every core, which looks like a pin and is not one. Three separate defects, all closed: #77 refused every write as a symlink attack, #75 read a sysfs attribute that does not exist, and #78 wrote unvalidated ranges that could stop a container from starting.

The common shape is the one worth remembering: the daemon reported success and nothing had happened. If a tier setting appears to be accepted, confirm the effect on the host rather than the intent in the log. For pinning that is grep -n 'cpuset' /etc/pve/lxc/*.conf, and the cgroup the container actually got is cat /sys/fs/cgroup/lxc/<ctid>/cpuset.cpus.effective.

The tier is pinned and its CPU scaling stopped working

Not a bug in the daemon, an interaction with Proxmox, and it surprises everyone. PVE derives a container's affinity from cores only when the configuration carries no explicit cpuset line. Setting cpu_pinning writes one, so from the container's next start cores no longer controls anything. The daemon keeps computing increments and logging "Increase Cores" for a value the hypervisor has stopped reading.

Verified on PVE 9.1: a container with cores: 1 and a pin of 0-1 reported two CPUs to nproc after a restart. Use cpu_pinning or CPU scaling on a given tier, not both.

The service does not start, or the timer is irregular

• lxc_autoscale.service failing to start: #8
• Installation failing with a failed status check: #5
• Execution intervals drifting or firing inconsistently: #12
• SSH connection dropping before the command was sent, in remote mode: #10

Logging to file does not work

The configured log file stayed empty while the journal had the output. Reported in #20. Current logging behaviour, including the rotating JSON metrics log, is documented in the logging guide.