Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update host-upgrades addon to use pharos-host-upgrades #393

Merged
merged 22 commits into from
Jun 20, 2018
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 30 additions & 4 deletions addons/host-upgrades/addon.rb
Original file line number Diff line number Diff line change
@@ -1,20 +1,46 @@
# frozen_string_literal: true

Pharos.addon 'host-upgrades' do
version '0.1.0'
version '0.2.0'
license 'Apache License 2.0'

config {
attribute :interval, Pharos::Types::String
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we wanted to make the 1.2 upgrade smoother, we could continue to accept existing addon configurations with interval, and just translate that to schedule: "@every ...".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it's needed if the new addon complains when schedule is missing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everyone using the existing host-upgrades addon will need to edit their cluster.yml when upgrading to 1.2. That could be avoided.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's ok to edit cluster.yml in this case.

attribute :schedule, Pharos::Types::String
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This absolutely needs to have some regexp for validation, because anyone configuring this is going to be more likely to get the syntax wrong rather than right, and then be left debugging their CrashLoopBackoff pods... the feedback cycle for fixing this configuration is too long and painful otherwise.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to merge this sooner than later.. if regexp takes time then we should open a new issue and mark it to the 1.2.0 milestone.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need, I already have something, it just needs specs to test it.

attribute :schedule_window, Pharos::Types::String
Copy link
Contributor Author

@SpComb SpComb Jun 18, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: The --schedule-window defaults to 1h within pharos-host-upgrades itself, which might be a mistake... it should still be possible to disable it by using schedule_window: 0 - or something else that makes sense given the schedule, like schedule_window: 24h.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will now get overridden with --schedule-window=0 by default, which is IMO good behavior.

attribute :reboot, Pharos::Types::Bool.default(false)
attribute :drain, Pharos::Types::Bool.default(true)
}

config_schema {
required(:interval).filled(:str?, :duration?)
required(:schedule).filled(:str?, :cron?)
optional(:schedule_window).filled(:str?, :duration?)
optional(:reboot).filled(:bool?)
optional(:drain).filled(:bool?)
}

# @return [String]
def schedule
cron = Fugit::Cron.parse(config.schedule)

cron.to_cron_s
end

# @return [String]
def schedule_window
return '0' if !config.schedule_window

s = Fugit::Duration.parse(config.schedule_window).to_sec

"#{s}s"
end

install {
apply_resources(
interval: duration.parse(config.interval).to_sec
schedule: schedule,
schedule_window: schedule_window, # only supports h, m, s; not D, M, Y
reboot: config.reboot,
drain: config.reboot && config.drain,
journal: false, # disabled due to https://github.com/kontena/pharos-host-upgrades/issues/15
)
}
end
62 changes: 62 additions & 0 deletions addons/host-upgrades/resources/10-clusterrole.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: host-upgrades
rules:
# locking
- apiGroups:
- apps
resources:
- daemonsets
verbs:
- get
- watch
- apiGroups:
- apps
resources:
- daemonsets
resourceNames:
- host-upgrades
verbs:
- update
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- nodes/status
verbs:
- update

# drain
- apiGroups:
- ""
resources:
- nodes
verbs:
- update
- patch
- apiGroups:
- ""
resources:
- pods
verbs:
- list
- get
- delete
- apiGroups:
- extensions
resources:
- daemonsets
verbs:
- get
- apiGroups:
- ""
resources:
- pods/eviction
verbs:
- create
5 changes: 5 additions & 0 deletions addons/host-upgrades/resources/20-serviceaccount.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
apiVersion: v1
kind: ServiceAccount
metadata:
namespace: kube-system
name: host-upgrades
12 changes: 12 additions & 0 deletions addons/host-upgrades/resources/30-clusterrolebinding.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: host-upgrades
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: host-upgrades
subjects:
- kind: ServiceAccount
namespace: kube-system
name: host-upgrades
159 changes: 159 additions & 0 deletions addons/host-upgrades/resources/40-configmap.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
kind: ConfigMap
apiVersion: v1
metadata:
namespace: kube-system
name: host-upgrades
labels:
app: host-upgrades
data:
yum-cron.conf: |
[commands]
# What kind of update to use:
# default = yum upgrade
# security = yum --security upgrade
# security-severity:Critical = yum --sec-severity=Critical upgrade
# minimal = yum --bugfix update-minimal
# minimal-security = yum --security update-minimal
# minimal-security-severity:Critical = --sec-severity=Critical update-minimal
update_cmd = default
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this will install all available updated package from all yum repos... my understanding is that CentOS is missing the required repo metadata for things like minimal-security to work, those are only available in RHEL:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, CentOS is missing security metadata. We lock kube/docker packages with yum-lockversion which should make this a bit more safe. For RHEL we could use security or similar?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably still consider testing the combination of yum-lockversion and the yum-cron configuration to make sure it doesn't upgrade pinned packages.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested, yum-versionlock works with yum-cron.


# Whether a message should be emitted when updates are available,
# were downloaded, or applied.
update_messages = yes

# Whether updates should be downloaded when they are available.
download_updates = yes

# Whether updates should be applied when they are available. Note
# that download_updates must also be yes for the update to be applied.
apply_updates = yes

# Maximum amout of time to randomly sleep, in minutes. The program
# will sleep for a random amount of time between 0 and random_sleep
# minutes before running. This is useful for e.g. staggering the
# times that multiple systems will access update servers. If
# random_sleep is 0 or negative, the program will run immediately.
# 6*60 = 360
random_sleep = 0

[emitters]
# Name to use for this system in messages that are emitted. If
# system_name is None, the hostname will be used.
system_name = None

# How to send messages. Valid options are stdio and email. If
# emit_via includes stdio, messages will be sent to stdout; this is useful
# to have cron send the messages. If emit_via includes email, this
# program will send email itself according to the configured options.
# If emit_via is None or left blank, no messages will be sent.
emit_via = stdio

# The width, in characters, that messages that are emitted should be
# formatted to.
output_width = 80


[email]
# The address to send email messages from.
# NOTE: 'localhost' will be replaced with the value of system_name.
email_from = root@localhost

# List of addresses to send messages to.
email_to = root

# Name of the host to connect to to send email messages.
email_host = localhost


[groups]
# NOTE: This only works when group_command != objects, which is now the default
# List of groups to update
group_list = None

# The types of group packages to install
group_package_types = mandatory, default

[base]
# This section overrides yum.conf

# Use this to filter Yum core messages
# -4: critical
# -3: critical+errors
# -2: critical+errors+warnings (default)
debuglevel = 1

# skip_broken = True
mdpolicy = group:main

# Uncomment to auto-import new gpg keys (dangerous)
# assumeyes = True

unattended-upgrades.conf: |
// Override system /etc/apt/apt.conf.d/50unattended-upgrades
#clear "Unattended-Upgrade::Allowed-Origins";

// Automatically upgrade packages from these (origin:archive) pairs
Unattended-Upgrade::Allowed-Origins {
"${distro_id}:${distro_codename}";
"${distro_id}:${distro_codename}-security";
// Extended Security Maintenance; doesn't necessarily exist for
// every release and this system may not have it installed, but if
// available, the policy for updates is such that unattended-upgrades
// should also install from here by default.
"${distro_id}ESM:${distro_codename}";
// "${distro_id}:${distro_codename}-updates";
// "${distro_id}:${distro_codename}-proposed";
// "${distro_id}:${distro_codename}-backports";
};

// List of packages to not update (regexp are supported)
Unattended-Upgrade::Package-Blacklist {
// "vim";
// "libc6";
// "libc6-dev";
// "libc6-i686";
};

// This option allows you to control if on a unclean dpkg exit
// unattended-upgrades will automatically run
// dpkg --force-confold --configure -a
// The default is true, to ensure updates keep getting installed
//Unattended-Upgrade::AutoFixInterruptedDpkg "false";

// Split the upgrade into the smallest possible chunks so that
// they can be interrupted with SIGUSR1. This makes the upgrade
// a bit slower but it has the benefit that shutdown while a upgrade
// is running is possible (with a small delay)
//Unattended-Upgrade::MinimalSteps "true";

// Install all unattended-upgrades when the machine is shuting down
// instead of doing it in the background while the machine is running
// This will (obviously) make shutdown slower
//Unattended-Upgrade::InstallOnShutdown "true";

// Send email to this address for problems or packages upgrades
// If empty or unset then no email is sent, make sure that you
// have a working mail setup on your system. A package that provides
// 'mailx' must be installed. E.g. "user@example.com"
//Unattended-Upgrade::Mail "root";

// Set this value to "true" to get emails only on errors. Default
// is to always send a mail if Unattended-Upgrade::Mail is set
//Unattended-Upgrade::MailOnlyOnError "true";

// Do automatic removal of new unused dependencies after the upgrade
// (equivalent to apt-get autoremove)
Unattended-Upgrade::Remove-Unused-Dependencies "true";

// Automatically reboot *WITHOUT CONFIRMATION*
// if the file /var/run/reboot-required is found after the upgrade
//Unattended-Upgrade::Automatic-Reboot "false";

// If automatic reboot is enabled and needed, reboot at the specific
// time instead of immediately
// Default: "now"
//Unattended-Upgrade::Automatic-Reboot-Time "02:00";

// Use apt bandwidth limit feature, this example limits the download
// speed to 70kb/sec
//Acquire::http::Dl-Limit "70";
79 changes: 79 additions & 0 deletions addons/host-upgrades/resources/50-daemonset.yml.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: host-upgrades
namespace: kube-system
labels:
app: host-upgrades
spec:
updateStrategy:
type: RollingUpdate
template:
metadata:
labels:
app: host-upgrades
spec:
serviceAccountName: host-upgrades
containers:
- name: host-upgrades
image: "quay.io/kontena/pharos-host-upgrades-<%= arch.name %>:<%= version %>"
imagePullPolicy: IfNotPresent
command:
- pharos-host-upgrades
args:
- "--schedule=<%= schedule %>"
<% if schedule_window %>
- "--schedule-window=<%= schedule_window %>"
<% end %>
<% if reboot %>
- --reboot
<% end %>
<% if drain %>
- --drain
<% end %>
env:
- name: KUBE_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: KUBE_DAEMONSET
value: host-upgrades
- name: KUBE_NODE
valueFrom:
fieldRef:
fieldPath: spec.nodeName
securityContext:
privileged: true
volumeMounts:
- name: config
mountPath: /etc/host-upgrades
- name: host
mountPath: /run/host-upgrades
- name: dbus
mountPath: /var/run/dbus
<% if journal %>
- name: journal
mountPath: /run/log/journal
<% end %>
volumes:
- name: config
configMap:
name: host-upgrades
- name: host
hostPath:
path: /run/host-upgrades
type: DirectoryOrCreate
- name: dbus
hostPath:
path: /var/run/dbus
type: Directory
<% if journal %>
- name: journal
hostPath:
path: /var/log/journal
type: Directory
<% end %>
restartPolicy: Always
tolerations:
- effect: NoSchedule
operator: Exists
Loading