Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Easy to use Cortex: Single binary, single process #1262

Draft
wants to merge 10 commits into
base: master
from

Conversation

Projects
None yet
3 participants
@tomwilkie
Copy link
Member

commented Mar 6, 2019

Fixes #1136

Build a single binary Cortex which can be told to start as an ingester, distributor, querier etc. Declare the various dependancies between modules and use code to start and stop them in the right order. Support the loading of config from a YAML file, and ship a config that allows you to start as a single process.

This replaces all the previous binaries and lite, and avoids the problem of having two (or more) copies of the init code that can get out of sync.

The follow flags have been removed:

  • -ingester.max-concurrent-streams: use -server.grpc-max-concurrent-streams
  • -query-frontend.max-recv-message-size-bytes: use -server.grpc-max-recv-msg-size-bytes
  • -memcached.* on the the frontend: use -frontend.memcached.*

TODO:

  • Single binary
  • Figure out what to do about ruler and alertmanager.
  • Test.
  • Config for single process mode.
  • Add getting started guide.
  • Test it all still works as a bunch of microservices.

@tomwilkie tomwilkie force-pushed the grafana:single-binary branch from 7225c6a to 4fd62d2 Mar 14, 2019

@tomwilkie tomwilkie force-pushed the grafana:single-binary branch 4 times, most recently from 3d1daa6 to fd10a6b Mar 22, 2019

@tomwilkie tomwilkie force-pushed the grafana:single-binary branch from fd10a6b to fe2f248 Apr 7, 2019

Show resolved Hide resolved docs/getting_started.md Outdated
Show resolved Hide resolved docs/getting_started.md
)

// Client is what the ruler and altermanger needs from a config store to process rules.
type Client interface {

This comment has been minimized.

Copy link
@khaines

khaines Apr 7, 2019

Member

This interface looks to be missing a GetTemplates method for alert template files, which were added in PR #1237.

This comment has been minimized.

Copy link
@tomwilkie

tomwilkie Apr 10, 2019

Author Member

Hi @khaines - I don't understand - I don't see a GetTemplates method in that PR (or anywhere in the codebase). What am I missing?

This comment has been minimized.

Copy link
@tomwilkie

tomwilkie Apr 10, 2019

Author Member

Yeah, I think we're good - GetAlerts returns a ConfigsResponse which contains the templates, like before.

Show resolved Hide resolved pkg/cortex/cortex.go Outdated
Show resolved Hide resolved pkg/cortex/cortex.go Outdated
Show resolved Hide resolved pkg/cortex/cortex.go Outdated

@tomwilkie tomwilkie force-pushed the grafana:single-binary branch 3 times, most recently from bd6846e to ee3773e Apr 10, 2019

tomwilkie added some commits Mar 6, 2019

Update weaveworks/common.
We needed:
- the new server-wide gRPC flags.
- Level.MarshalYAML

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Single binary, single process Cortex.
Add a new module-loading system that declaritvely states which module depends on which, and then initialises them in the correct order.

This also allows for a single-process mode where all the different modules run in the same process.

Also:
- Remove the migrations copying, and we just have a single docker image now.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Unify the config client between the alertmanager and the ruler.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Prefix frontend memcache flags with 'frontend.'
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Don't register the config DB flags twice.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Remove the FromStr parsing hack in schema config, do it with a custom…
… type.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Add yaml struct tags to more config fields; Manage lifecycle of overi…
…ddes consistently.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Add getting started guide and example config for running as a single …
…process.

This is built into the docer image to make the getting started guide a bit simplier.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Add -print.config flag, which causes Cortex to print config and exit.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

@tomwilkie tomwilkie force-pushed the grafana:single-binary branch from e33d94e to b1c43ae Apr 10, 2019

Review feedback & minor fixups.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

@tomwilkie tomwilkie force-pushed the grafana:single-binary branch from b1c43ae to b5bb25a Apr 10, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.