Add multi local namespace support #1335

benraskin92 · 2019-01-31T16:21:34Z

No description provided.

codecov · 2019-01-31T16:36:58Z

Codecov Report

Merging #1335 into master will increase coverage by <.1%.
The diff coverage is 34.7%.

@@           Coverage Diff            @@
##           master   #1335     +/-   ##
========================================
+ Coverage    70.6%   70.7%   +<.1%     
========================================
  Files         823     822      -1     
  Lines       70217   70179     -38     
========================================
+ Hits        49599   49624     +25     
+ Misses      17384   17323     -61     
+ Partials     3234    3232      -2

Flag	Coverage Δ
#aggregator	`81.6% <ø> (ø)`	⬆️
#cluster	`85.6% <ø> (ø)`	⬆️
#collector	`78.4% <ø> (ø)`	⬆️
#dbnode	`80.8% <ø> (+0.1%)`	⬆️
#m3em	`73.2% <ø> (ø)`	⬆️
#m3ninx	`74.2% <ø> (ø)`	⬆️
#m3nsch	`51.1% <ø> (ø)`	⬆️
#metrics	`17.8% <ø> (ø)`	⬆️
#msg	`74.9% <ø> (ø)`	⬆️
#query	`64.3% <34.7%> (+0.1%)`	⬆️
#x	`76.2% <ø> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 806e93b...59dbb99. Read the comment docs.

codecov · 2019-01-31T16:37:05Z

Codecov Report

Merging #1335 into master will increase coverage by 0.1%.
The diff coverage is 34.7%.

@@           Coverage Diff            @@
##           master   #1335     +/-   ##
========================================
+ Coverage    70.6%   70.7%   +0.1%     
========================================
  Files         823     822      -1     
  Lines       70217   70179     -38     
========================================
+ Hits        49599   49645     +46     
+ Misses      17384   17304     -80     
+ Partials     3234    3230      -4

Flag	Coverage Δ
#aggregator	`81.6% <ø> (ø)`	⬆️
#cluster	`85.6% <ø> (ø)`	⬆️
#collector	`78.4% <ø> (ø)`	⬆️
#dbnode	`80.9% <ø> (+0.2%)`	⬆️
#m3em	`73.2% <ø> (ø)`	⬆️
#m3ninx	`74.2% <ø> (ø)`	⬆️
#m3nsch	`51.1% <ø> (ø)`	⬆️
#metrics	`17.8% <ø> (ø)`	⬆️
#msg	`74.9% <ø> (ø)`	⬆️
#query	`64.3% <34.7%> (+0.1%)`	⬆️
#x	`76.2% <ø> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 806e93b...81f03e0. Read the comment docs.

richardartoul · 2019-01-31T17:06:56Z

docs/how_to/single_node.md

@@ -16,9 +16,11 @@ directory on your host for durability:

 ```
 docker pull quay.io/m3/m3dbnode:latest
-docker run -p 7201:7201 -p 9003:9003 --name m3db -v $(pwd)/m3db_data:/var/lib/m3db quay.io/m3/m3dbnode:latest
+docker run -p 7201:7201 -p 7203:7203 -p 9003:9003 --name m3db -v $(pwd)/m3db_data:/var/lib/m3db -v $GOPATH/src/github.com/m3db/m3/src/dbnode/config/m3dbnode-local-etcd.yml:/etc/m3dbnode/m3dbnode.yml quay.io/m3/m3dbnode:latest


can we add a little comment about what each port being exposed does?

yo so this command now includes a $GOPATH which means it will only work for users who have already cloned . the m3 repository into their $GOPATH.

I think instead you should link to the config file in Github and tell them to copy and paste that into a file locally or something

And then add something like: "note that this configuration file configuration for both coordinator AND M3DB because its running as a single binary, if you were to run these applications separately then they would have separate configuration.

I know it is kind of saying the same thing over and over again but it it pretty confusing stuff

I like the idea of just having a command you can copy and paste into your terminal - I'll make a note though if you don't have a GOPATH setup

I really think you should remove the $GOPATH. Copy paste and working is fine but "copy and paste this config file into your local directory" is way easier than:

1. Install golang 2. Setup your $GOPATH 3. Clone the M3 repository and make sure you clone it into $GOPATH/src/....

richardartoul · 2019-01-31T17:07:22Z

docs/how_to/single_node.md

 ```

+**Note:** This setup runs `M3DB` and `M3Coordinator` as one application and should only be used for testing/development purposes. If you want to run a clustered `M3DB` with a separate `M3Coordinator`, please [see here](cluster_hard_way.md).


"with a separate . M3Coordinator process (which is our recommended production setup)...`

richardartoul · 2019-01-31T17:28:54Z

docs/how_to/single_node.md

 ```

+**Note:** This setup runs `M3DB` and `M3Coordinator` as one application and should only be used for testing/development purposes. If you want to run a clustered `M3DB` with a separate `M3Coordinator`, please [see here](cluster_hard_way.md).
+
 <!-- TODO: link to docs containing explanations of what namespaces, the coordinator,


we have explanatory docs for namespaces/placement under operational guides if you want to link to them

docs/how_to/single_node.md

richardartoul · 2019-01-31T17:30:05Z

docs/how_to/single_node.md

@@ -37,6 +39,20 @@ curl -X POST http://localhost:7201/api/v1/database/create -d '{
 }'
 ```

+**Note:** If you want to create more than one namespace, you should follow the [instructions here](../operational_guide/namespace_configuration.md) and also add the namespace you created to the `local` section of the `m3dbnode-local-etcd.yml` file used in the `docker run` command above with the appropriate aggregation options specified. For example:


"with the appropriate aggregation options specified" could you add: "for more information on our aggregation functionality, check out our M3Query documentation"

richardartoul · 2019-01-31T17:31:58Z

docs/integrations/prometheus.md

@@ -92,3 +92,11 @@ remote_read:
 remote_write:
  - url: "http://localhost:7201/api/v1/prom/remote/write"
 ```
+
+Also, if you want to see metrics generated by the `m3coordinator` or `m3query` process such as number of queries, etc., you should add the following job under `scrape_configs`:


How about:

"Also, we recommend adding M3DB and M3Coordinator/`M3Query' to your list of jobs so that you can monitor them with Prometheus. With Prometheus scraping setup, you can also use our pre-configured M3DB Grafana dashboard"

richardartoul · 2019-01-31T17:32:09Z

docs/integrations/prometheus.md

+```json
+- job_name: 'query'
+  static_configs:
+    - targets: ['localhost:7203']


can we add a scrape job for M3DB too?

richardartoul · 2019-01-31T17:32:27Z

kube/bundle.yaml

@@ -117,6 +117,11 @@ data:
      listenAddress:
        type: "config"
        value: "0.0.0.0:7201"
+      local:


is this correct for kube?

Yeah - @schallert said to update here as well

richardartoul · 2019-01-31T17:33:20Z

scripts/development/m3_stack/m3dbnode.yml

@@ -3,6 +3,12 @@ coordinator:
    type: "config"
    value: "0.0.0.0:7201"

+local:


hm...I don't think this is needed. m3_stack runs m3coordinator as an external binary, so I don't . think you need to set this here. To be honest I'm not sure why the coordinator config is here at all

Hmm I'll leave it here for now.

but its not even the correct list of namespaces, m3_stack configures a bunch of other namespaces

So you think I should remove?

I think so, it'll be less confusing. We should probably remove the coordinator config entirely from this file but its a bigger change.

richardartoul · 2019-01-31T17:34:05Z

src/dbnode/config/m3dbnode-local-etcd.yml

@@ -3,6 +3,12 @@ coordinator:
    type: "config"
    value: "0.0.0.0:7201"

+  # local:


why is this commented out

Was testing - will uncomment.

richardartoul

LGTM

richardartoul · 2019-01-31T18:43:53Z

docs/how_to/single_node.md

@@ -16,9 +16,11 @@ directory on your host for durability:

 ```
 docker pull quay.io/m3/m3dbnode:latest
-docker run -p 7201:7201 -p 9003:9003 --name m3db -v $(pwd)/m3db_data:/var/lib/m3db quay.io/m3/m3dbnode:latest
+docker run -p 7201:7201 -p 7203:7203 -p 9003:9003 --name m3db -v $(pwd)/m3db_data:/var/lib/m3db -v $GOPATH/src/github.com/m3db/m3/src/dbnode/config/m3dbnode-local-etcd.yml:/etc/m3dbnode/m3dbnode.yml quay.io/m3/m3dbnode:latest


I really think you should remove the $GOPATH. Copy paste and working is fine but "copy and paste this config file into your local directory" is way easier than:

1. Install golang 2. Setup your $GOPATH 3. Clone the M3 repository and make sure you clone it into $GOPATH/src/....

richardartoul · 2019-01-31T18:46:04Z

scripts/development/m3_stack/m3dbnode.yml

@@ -3,6 +3,12 @@ coordinator:
    type: "config"
    value: "0.0.0.0:7201"

+local:


but its not even the correct list of namespaces, m3_stack configures a bunch of other namespaces

Benjamin Raskin and others added 6 commits January 31, 2019 10:07

Update m3db single node docs

7130123

Fix some things

974c158

Fix cmd

093ffa9

Add support for multiple local namespace definitions

8462817

Update configs

5694931

Caps

59dbb99

benraskin92 requested review from robskillington and richardartoul January 31, 2019 16:35

benraskin92 mentioned this pull request Jan 31, 2019

Update m3db single node docs #1333

Closed

richardartoul reviewed Jan 31, 2019

View reviewed changes

docs/how_to/single_node.md Show resolved Hide resolved

richardartoul reviewed Jan 31, 2019

View reviewed changes

Address comments

81f03e0

richardartoul approved these changes Jan 31, 2019

View reviewed changes

benraskin92 merged commit e80adde into master Jan 31, 2019

justinjc deleted the braskin/multi_local_namespace branch June 17, 2019 21:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add multi local namespace support #1335

Add multi local namespace support #1335

benraskin92 commented Jan 31, 2019

codecov bot commented Jan 31, 2019

codecov bot commented Jan 31, 2019 •

edited

Loading

richardartoul Jan 31, 2019

richardartoul Jan 31, 2019

richardartoul Jan 31, 2019

benraskin92 Jan 31, 2019

richardartoul Jan 31, 2019

richardartoul Jan 31, 2019

richardartoul Jan 31, 2019

richardartoul Jan 31, 2019

richardartoul Jan 31, 2019

richardartoul Jan 31, 2019

richardartoul Jan 31, 2019

benraskin92 Jan 31, 2019

richardartoul Jan 31, 2019

benraskin92 Jan 31, 2019

richardartoul Jan 31, 2019

benraskin92 Jan 31, 2019

richardartoul Jan 31, 2019

richardartoul Jan 31, 2019

benraskin92 Jan 31, 2019

richardartoul left a comment

richardartoul Jan 31, 2019

richardartoul Jan 31, 2019

		```

		Note: This setup runs `M3DB` and `M3Coordinator` as one application and should only be used for testing/development purposes. If you want to run a clustered `M3DB` with a separate `M3Coordinator`, please [see here](cluster_hard_way.md).

Add multi local namespace support #1335

Add multi local namespace support #1335

Conversation

benraskin92 commented Jan 31, 2019

codecov bot commented Jan 31, 2019

Codecov Report

codecov bot commented Jan 31, 2019 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

richardartoul left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jan 31, 2019 •

edited

Loading