Skip to content
Data center construction tools
Branch: master
Clone or download
tapih Merge pull request #281 from cybozu-go/sabakan-state-setter-reapply
Revert "Revert "Merge pull request #250 from cybozu-go/sabakan-state-setter""
Latest commit 674fa77 May 21, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.circleci Remove temporary upgrade test job Apr 16, 2019
.github/ISSUE_TEMPLATE Update issue templates Apr 8, 2019
bin Delete run-dctest-reboot.sh Mar 11, 2019
dctest Revert "Revert "Merge pull request #250 from cybozu-go/sabakan-state-… May 21, 2019
debian Revert "Revert "Merge pull request #250 from cybozu-go/sabakan-state-… May 21, 2019
docs Revert "Revert "Merge pull request #250 from cybozu-go/sabakan-state-… May 21, 2019
etc Revert "Revert "Merge pull request #250 from cybozu-go/sabakan-state-… May 21, 2019
ext Use different context when updater sends slack notification Apr 3, 2019
gcp [necogcp] Fix a bug May 14, 2019
generator [generator] ignore branch tags Apr 22, 2019
ignitions [ignition] enable lvm2-monitor.service May 14, 2019
menu Revert "Revert "Merge pull request #250 from cybozu-go/sabakan-state-… May 21, 2019
pkg Revert "Revert "Merge pull request #250 from cybozu-go/sabakan-state-… May 21, 2019
progs Revert "Revert "Merge pull request #250 from cybozu-go/sabakan-state-… May 21, 2019
setup
storage
updater [updater] fix bogus timeout error Apr 23, 2019
vendor Revert "Revert "Merge pull request #250 from cybozu-go/sabakan-state-… May 21, 2019
worker Revert "Revert "Merge pull request #250 from cybozu-go/sabakan-state-… May 21, 2019
.gitignore fix Mar 18, 2019
LICENSE Add documents Oct 26, 2018
Makefile Revert "Revert "Merge pull request #250 from cybozu-go/sabakan-state-… May 21, 2019
Makefile.common build node_exporter in BUILDDIR Mar 4, 2019
Makefile.gcp Fix Feb 18, 2019
Makefile.tools Update for cke-1.14 Apr 22, 2019
README.md Update README.md Apr 26, 2019
address_util.go fixes Nov 3, 2018
artifacts.go update artifacts.go May 21, 2019
constants.go Revert "Revert "Merge pull request #250 from cybozu-go/sabakan-state-… May 21, 2019
deb.go Fix CircieCI jobs dependency. Feb 14, 2019
deb_test.go Fix neco-worker to fail when dpkg failed. Feb 14, 2019
etcd.go [neco] implement "vault" subcommands Nov 4, 2018
file.go Implement all cke deployment functions Nov 26, 2018
github.go Tidy GitHub API usage. Nov 9, 2018
go.mod Revert "Revert "Merge pull request #250 from cybozu-go/sabakan-state-… May 21, 2019
hardware.go Return HWTypeNil Nov 26, 2018
identity.go wip Nov 1, 2018
identity_test.go
ignitions_test.go Compare yaml and file tree recursively Apr 18, 2019
neco_logo.svg Add files via upload Apr 26, 2019
rkt.go Make quay.io secrets optional Feb 1, 2019
rkt_test.go wip Nov 30, 2018
sabakan.go Fix ignitions Dec 6, 2018
sleep.go [WIP] upload assets Nov 30, 2018
sleep_test.go Sleep and retry on updating container linux Nov 29, 2018
systemd.go [necogcp] implement "create vmx-enabled" and "setup" for vmx-enabled Feb 15, 2019
types.go [generator] do not choose major version upgrade for release Feb 19, 2019
update.go Suppress update if config "env" is not set. Jan 18, 2019
update_test.go wip Nov 7, 2018
vault.go Maintenance CKE Nov 26, 2018

README.md

GitHub release CircleCI GoDoc Go Report Card

neco logo

Neco is a collection of tools to build and maintain large data center infrastructure. It is used in cybozu.com, the top B2B groupware service in Japan.

Project Status: Perpetual beta.

Architecture

Neco adopts network booting to manage thousands of servers efficiently. The center of Neco is therefore a few netboot servers.

neco-ubuntu creates a custom Ubuntu installer to setup netboot servers.

Sabakan, a product from Neco, implements functions necessary for netboot including DHCP, UEFI HTTP Boot, iPXE, ignition and file server service to download assets.

sabakan_architecture

To share and keep data between netboot servers, etcd is used widely. To protect data and authorize users, Vault is used too.

etcd_vault_architecture

As to network, Neco uses layer-3 technologies only. Specifically, BGP is used to advertise/receive routes together with BFD to reduce route convergence and ECMP for high availability.

network_architecture

Repository contents

neco

neco is a deploy automation tool written in Go. It installs and updates miscellaneous utilities in boot servers.

See docs/neco.md

neco-updater

neco-updater is a background service to detect new releases of neco. When detected, it sends the update information to neco-worker through etcd.

Neco itself is released as a Debian package in releases.

See docs/neco-updater.md

neco-worker

neco-worker is an automate maintenance service. It installs/updates neco package, applications, and sabakan contents when receives information from neco-updater through etcd.

See docs/neco-worker.md

generate-artifacts

generate-artifacts is a command-line tool to generate artifacts.go which is a collection of latest components.

See docs/generate-artifacts.md

Test suite

dctest directory contains test suites to run integration tests in a virtual data center environment.

Placemat is a tool to create arbitrarily complex network topology and virtual servers using Linux networking stacks, rkt, and QEMU.

See docs/dctest.md

CI/CD

CI/CD in Neco is running by CircleCI. Then neco-updater service is updated automatically.

See docs/cicd.md

Run unit tests at local machine

First, start up etcd server manually.

$ make start-etcd

Then, run go test on another terminal.

$ go test -v -count=1 -race -mod=vendor ./...

GCP suite

gcp directory contains utilities for provisioning the Google Compute Engine services for Neco project. Test suite, and other Neco github projects deploy GCE instance based on an image created by this tool.

See docs/gcp

git-neco

git-neco is a git extension utility to help Neco developers.

Documentation

docs directory contains documents about designs and specifications.

You can’t perform that action at this time.