Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nightly disk usage report microservice #23

Merged
merged 19 commits into from
Mar 13, 2019
Merged

Conversation

domq
Copy link
Member

@domq domq commented Feb 25, 2019

  • Generate qdirstat report every night using a modified version of the
    script referenced by https://superuser.com/a/1028907/48763 (freeware
    / public domain)

  • Run the script in a slightly modified version of the "perl" stock
    Docker image

  • Plumb down all the Kubernetes / OpenShift objects to build the
    image, host the script (from a ConfigMap), and run it (as a k8s
    CronJob)

  • Teach our Ansible infrastructure to manage said Kubernetes state

Dominique Quatravaux added 3 commits February 28, 2019 14:47
* Generate qdirstat report every night using a modified version of the
  script referenced by https://superuser.com/a/1028907/48763 (freeware
  / public domain)

* Run the script in a slightly modified version of the "perl" stock
  Docker image

* Plumb down all the Kubernetes / OpenShift objects to build the
  image, host the script (from a ConfigMap), and run it (as a k8s
  CronJob)

* Jury-rigged Makefile to work around
  openshift/origin#8971 and share the YAML
  between test and production
Coming next: wordpress-service, which applies to something different
than all-wordpresses
@domq domq force-pushed the wwp-nightly-disk-usage-report branch 8 times, most recently from 7c0dede to 4db5843 Compare March 5, 2019 15:27
Dominique Quatravaux added 16 commits March 5, 2019 17:50
This paves the way for the `deploy` script to be used for other
purposes than deploying WordPress instances
* Playbook has now multiple plays per resource type

* New role "wordpress-nas-share", does nothing for now

* New "host" groups for dev and prod, describing exactly one NAS share
  each
* Use 'openshift' as the Ansible task name and 'oc' as the command

* do *not* fall back to kubectl (some platforms don't have
  the latter e.g. Arch Linux)
* Use the newly-minted "openshift" Ansible task

* Vary by namespace using Ansible's macro substitution feature, as
  opposed to a kludge in a Makefile

* Use a template file for the Perl script (boy, was that a nightmare
  to pipe through all the layers; see in-line Jinja comments which I
  expect to remain useful to future generations)
* Obtain current and putative new states in JSON format (availability
  of PyYAML on a remote Python being uncertain at best)
* Diff them with a crude but seemingly effective method
* In case of no diff, have `replace` return a no-op status
* Fully recursive

* Igores fields called "metadata", no matter how deep
* We no longer need an exception for "metadata" and "status"

* We can now set "state: latest" more liberally
This lets one e.g. reset object labels by saying "labels: {}", and it
works (Ansible turns back to green on the second run).

Naturally, on some other field that gets auto-populated K8S-side (e.g.
metadata), that would not work; still, the required change would be
attempted at least, i.e. the "state: latest" semantics is now closer
to being the correct one.

* Gracefully descend into the comparison with None if finding a key
  that doesn't exist; this will let labels: {} config-side match no
  labels at all prod-side, which is the correct thing to do (results
  in green status)

* In the same line of thought, allow a None field (foo: ~ in YAML) to
  match a nonexistent field, by comparing favorably None (found in
  YAML) with itself (defaulted as per the previous item's algorithm change)
* Rename method and parameters to outline the asymmetry between both sides

* Provide moar comments and a docstring

* Treat equality in the sense of == as a bypass, doing away with the
  enumeration of scalar types

* Local functions is_list and is_dict for readability
@domq domq force-pushed the wwp-nightly-disk-usage-report branch from 4db5843 to 59d6e95 Compare March 5, 2019 16:51
Copy link
Contributor

@GregLeBarbar GregLeBarbar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lucien et moi venons de parcourir rapidement la PR.
C'est oki pour nous même si on comprend pas tous les détails (euphémisme).
Serais-tu ok pour la pousser en prod toi même ?

@domq
Copy link
Member Author

domq commented Mar 13, 2019

OK pour être l'opérateur demain matin !

Je pushe.

@domq domq merged commit 95617d7 into master Mar 13, 2019
@domq domq deleted the wwp-nightly-disk-usage-report branch March 21, 2019 12:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants