Directly access data objects stored in etcd by kubernetes.
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
cmd Upgrade to kubernetes 1.12.1, fix mistake in value extraction Oct 17, 2018
pkg/encoding etcd wal log analysis tool etcd-dump-logs support Aug 29, 2018
.gitignore Rename to 'auger' Sep 19, 2017
.travis.yml Enable Travis CI. Add license to all yaml files. Add OWNER file. Sep 15, 2017
CONTRIBUTING.md Add OSS license and policy files. Jul 31, 2017
Dockerfile Dockerfile: add a Dockerfile to easily build and use auger May 30, 2018
LICENSE Add OSS license and policy files. Jul 31, 2017
Makefile Rename to 'auger' Sep 19, 2017
OWNER glide, pkg/encoding: update and simplify dependencies up to Kubernete… May 30, 2018
README.md document minimal steps to install locally Oct 17, 2018
REVIEWING.md Add OSS license and policy files. Jul 31, 2017
code-of-conduct.md
glide.lock
glide.yaml Upgrade to kubernetes 1.12.1, fix mistake in value extraction Oct 17, 2018
main.go Rename to 'auger' Sep 19, 2017

README.md

Auger

Directly access data objects stored in etcd by kubernetes.

Encodes and decodes Kubernetes objects from the binary storage encoding used to store data to etcd. Supports data conversion to YAML, JSON and Protobuf.

Automatically determines if etcd data is stored in JSON (kubernetes 1.5 and earlier) or binary (kubernetes 1.6 and newer) and decodes accordingly.

Why?

In earlier versions of kubernetes, data written to etcd was stored as JSON and could easily be inspected or manipulated using standard tools such as etcdctl. In kubernetes 1.6+, for efficiency reasons, much of the data is now stored in a binary storage representation, and is non-trivial to decode-- it contains a enveloped payload that must be unpacked, type resolved and decoded.

This tool provides kubernetes developers and cluster operators with simple way to access the binary storage data via YAML and JSON.

Installation

Check out and build:

git clone https://github.com/jpbetz/auger
cd auger
make release

Run:

build/auger -h

Use cases

Access data via etcdctl

A kubernetes developer or cluster operator needs to inspect the data actually stored to etcd for a particular kubernetes object.

E.g., decode a pod from etcd v3, where <pod-name> is the name of one of your pods:

ETCDCTL_API=3 etcdctl get /registry/pods/default/<pod-name> | auger decode
> apiVersion: v1
> kind: Pod
> metadata:
>   annotations: ...
>   creationTimestamp: 2017-06-27T16:35:34Z
> ...

Modify data via etcdctl

A kubernetes developer or etcd developer needs to modify state of an object stored in etcd.

E.g. Write an updated pod to etcd v3:

cat updated-pod.yaml | auger encode | ETCDCTL_API=3 etcdctl put /registry/pods/default/<pod-name>

Access data directly from db file

A cluster operator, kubernetes developer or etcd developer is needs to inspect etcd data without starting etcd. In extreme cases, it may not be possible to start etcd and inspecting the data may help a etcd developer understand what state it is in.

E.g. find an etcd value by it's key and extract it from a boltdb file:

auger extract -f <boltdb-file> -k /registry/pods/default/<pod-name>
> apiVersion: v1
> kind: Pod
> metadata:
>   annotations: ...
>   creationTimestamp: 2017-06-27T16:35:34Z
> ...

TODO

  • Warn if attempting to read data written by a different version of kubernetes
  • Add detection of unrecognized fields in stored data, which would suggest data was written with newer version of proto schema
  • Enable travis CI
  • Build and publish releases for all recent kubernetes versions (1.6+)