Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ReadTheDocs documentation for Table Configs, Monitoring, and Deployment #3975

Merged
merged 13 commits into from
Mar 27, 2019

Conversation

jgutmann
Copy link
Contributor

First pass of 'Running Pinot in Production' docs, which includes
sections on

  • Deploying Pinot and the ordering
  • Table configs and what each setting does
  • Monitoring (still needs proper metric names added)

First pass of 'Running Pinot in Production' docs, which includes
sections on

* Deploying Pinot and the ordering
* Table configs and what each setting does
* Monitoring (still needs proper metric names added)
@codecov-io
Copy link

codecov-io commented Mar 14, 2019

Codecov Report

Merging #3975 into master will increase coverage by 0.05%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #3975      +/-   ##
============================================
+ Coverage     66.99%   67.05%   +0.05%     
- Complexity        4       20      +16     
============================================
  Files          1032     1033       +1     
  Lines         51047    51338     +291     
  Branches       7126     7181      +55     
============================================
+ Hits          34200    34424     +224     
- Misses        14502    14537      +35     
- Partials       2345     2377      +32
Impacted Files Coverage Δ Complexity Δ
...a/manager/realtime/RealtimeSegmentDataManager.java 50% <0%> (-50%) 0% <0%> (ø)
...mpl/dictionary/DoubleOffHeapMutableDictionary.java 67.27% <0%> (-9.1%) 0% <0%> (ø)
...elix/core/relocation/RealtimeSegmentRelocator.java 73.62% <0%> (-7.7%) 0% <0%> (ø)
.../apache/pinot/common/config/RealtimeTagConfig.java 93.33% <0%> (-6.67%) 0% <0%> (ø)
.../impl/dictionary/LongOffHeapMutableDictionary.java 87.27% <0%> (-5.46%) 0% <0%> (ø)
...impl/dictionary/FloatOffHeapMutableDictionary.java 87.27% <0%> (-5.46%) 0% <0%> (ø)
...a/org/apache/pinot/common/config/TagNameUtils.java 64.1% <0%> (-5.13%) 0% <0%> (ø)
...ot/controller/helix/core/util/HelixSetupUtils.java 62.31% <0%> (-4.35%) 0% <0%> (ø)
...api/resources/PinotTableConfigRestletResource.java 24.35% <0%> (-3.28%) 0% <0%> (ø)
...time/converter/stats/RealtimeColumnStatistics.java 47.22% <0%> (-2.06%) 0% <0%> (ø)
... and 44 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5867b16...ee39148. Read the comment docs.

Changes include:

1. Breaking out the table config section from the "Running Pinot in
Production" section. I think this makes the doc easier to follow for
table configs. I have a section here which includes a sample offline and
realtime table config. I am not sure if there are some configs included
as a way for users to run a local quick start. If there is I can link to
those files in the repo, otherwise I can include the full configs on
this page for users.

2. Adding links to metrics which I am able to find in the code. I need
to collect feedback on what to do with the others - Remove or I need
pointers to where I can find them in the code.
@jgutmann jgutmann changed the title First pass of 'Running Pinot in Production' docs ReadTheDocs documentation for Table Configs, Monitoring, and Deployment Mar 19, 2019
The is two items I need before this is completed

1. I notice that the controller metrics appear to be generated dynamically
due to table names being present in most of the metrics. I need some advice
from the devs on how I should note these metric names so I can link to them
in the code. I want to make sure it is clear to the users.

2. I want to add sample table configs in the "in_production.rst"
section. I feel this would be best to be added in the git repo since
they will be useful for the users in the quick start guides. I need Jack
to tell me where in the repo I should put these files. Once they are in
the repo I will link to them from that section.
docs/in_production.rst Show resolved Hide resolved
docs/in_production.rst Outdated Show resolved Hide resolved
docs/in_production.rst Outdated Show resolved Hide resolved
docs/in_production.rst Outdated Show resolved Hide resolved
docs/in_production.rst Outdated Show resolved Hide resolved
docs/tableconfig_schema.rst Outdated Show resolved Hide resolved
docs/tableconfig_schema.rst Outdated Show resolved Hide resolved
docs/tableconfig_schema.rst Outdated Show resolved Hide resolved
docs/tableconfig_schema.rst Outdated Show resolved Hide resolved
docs/tableconfig_schema.rst Outdated Show resolved Hide resolved
John Gutmann added 3 commits March 19, 2019 13:19
General cleanup and re-formatting based on feedback for my PR.
Copy link
Contributor

@mcvsubbu mcvsubbu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partial

docs/tableconfig_schema.rst Show resolved Hide resolved
docs/tableconfig_schema.rst Outdated Show resolved Hide resolved
docs/tableconfig_schema.rst Outdated Show resolved Hide resolved
docs/tableconfig_schema.rst Show resolved Hide resolved
docs/tableconfig_schema.rst Outdated Show resolved Hide resolved
docs/tableconfig_schema.rst Outdated Show resolved Hide resolved
docs/tableconfig_schema.rst Outdated Show resolved Hide resolved
docs/tableconfig_schema.rst Outdated Show resolved Hide resolved
docs/tableconfig_schema.rst Outdated Show resolved Hide resolved
docs/in_production.rst Outdated Show resolved Hide resolved
docs/in_production.rst Outdated Show resolved Hide resolved
docs/in_production.rst Show resolved Hide resolved
Copy link
Contributor

@mcvsubbu mcvsubbu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks


Monitoring Pinot
~~~~~~~~~~~~~~~~

Pinot exposes several metrics to monitor the service and ensure that pinot users are not experiencing issues. In this section we discuss some of the key metrics that are useful to monitor. A full list of metrics is available in the `Metrics <customizations.html#metrics>`_ section.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we dont have a 'full list' so to speak.
so maybe we should remove this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah i see you have a todo. no problem. your call.

@mcvsubbu mcvsubbu merged commit e4bb117 into apache:master Mar 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants