-
Notifications
You must be signed in to change notification settings - Fork 1
/
chef-expander-cluster.8.html
164 lines (136 loc) · 7.26 KB
/
chef-expander-cluster.8.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
<!DOCTYPE html>
<html>
<head>
<meta http-equiv='content-type' value='text/html;charset=utf8'>
<meta name='generator' value='Ronn/v0.7.3 (http://github.com/rtomayko/ronn/tree/0.7.3)'>
<title>chef-expander-cluster(8) - fetches messages from RabbitMQ, processes, and loads into chef-solr</title>
<style type='text/css' media='all'>
/* style: man */
body#manpage {margin:0}
.mp {max-width:100ex;padding:0 9ex 1ex 4ex}
.mp p,.mp pre,.mp ul,.mp ol,.mp dl {margin:0 0 20px 0}
.mp h2 {margin:10px 0 0 0}
.mp > p,.mp > pre,.mp > ul,.mp > ol,.mp > dl {margin-left:8ex}
.mp h3 {margin:0 0 0 4ex}
.mp dt {margin:0;clear:left}
.mp dt.flush {float:left;width:8ex}
.mp dd {margin:0 0 0 9ex}
.mp h1,.mp h2,.mp h3,.mp h4 {clear:left}
.mp pre {margin-bottom:20px}
.mp pre+h2,.mp pre+h3 {margin-top:22px}
.mp h2+pre,.mp h3+pre {margin-top:5px}
.mp img {display:block;margin:auto}
.mp h1.man-title {display:none}
.mp,.mp code,.mp pre,.mp tt,.mp kbd,.mp samp,.mp h3,.mp h4 {font-family:monospace;font-size:14px;line-height:1.42857142857143}
.mp h2 {font-size:16px;line-height:1.25}
.mp h1 {font-size:20px;line-height:2}
.mp {text-align:justify;background:#fff}
.mp,.mp code,.mp pre,.mp pre code,.mp tt,.mp kbd,.mp samp {color:#131211}
.mp h1,.mp h2,.mp h3,.mp h4 {color:#030201}
.mp u {text-decoration:underline}
.mp code,.mp strong,.mp b {font-weight:bold;color:#131211}
.mp em,.mp var {font-style:italic;color:#232221;text-decoration:none}
.mp a,.mp a:link,.mp a:hover,.mp a code,.mp a pre,.mp a tt,.mp a kbd,.mp a samp {color:#0000ff}
.mp b.man-ref {font-weight:normal;color:#434241}
.mp pre {padding:0 4ex}
.mp pre code {font-weight:normal;color:#434241}
.mp h2+pre,h3+pre {padding-left:0}
ol.man-decor,ol.man-decor li {margin:3px 0 10px 0;padding:0;float:left;width:33%;list-style-type:none;text-transform:uppercase;color:#999;letter-spacing:1px}
ol.man-decor {width:100%}
ol.man-decor li.tl {text-align:left}
ol.man-decor li.tc {text-align:center;letter-spacing:4px}
ol.man-decor li.tr {text-align:right;float:right}
</style>
<style type='text/css' media='all'>
/* style: toc */
.man-navigation {display:block !important;position:fixed;top:0;left:113ex;height:100%;width:100%;padding:48px 0 0 0;border-left:1px solid #dbdbdb;background:#eee}
.man-navigation a,.man-navigation a:hover,.man-navigation a:link,.man-navigation a:visited {display:block;margin:0;padding:5px 2px 5px 30px;color:#999;text-decoration:none}
.man-navigation a:hover {color:#111;text-decoration:underline}
</style>
</head>
<!--
The following styles are deprecated and will be removed at some point:
div#man, div#man ol.man, div#man ol.head, div#man ol.man.
The .man-page, .man-decor, .man-head, .man-foot, .man-title, and
.man-navigation should be used instead.
-->
<body id='manpage'>
<div class='mp' id='man'>
<div class='man-navigation' style='display:none'>
<a href="#NAME">NAME</a>
<a href="#SYNOPSIS">SYNOPSIS</a>
<a href="#DESCRIPTION">DESCRIPTION</a>
<a href="#SEE-ALSO">SEE ALSO</a>
<a href="#AUTHOR">AUTHOR</a>
</div>
<ol class='man-decor man-head man head'>
<li class='tl'>chef-expander-cluster(8)</li>
<li class='tc'>Chef Manual</li>
<li class='tr'>chef-expander-cluster(8)</li>
</ol>
<h2 id="NAME">NAME</h2>
<p class="man-name">
<code>chef-expander-cluster</code> - <span class="man-whatis">fetches messages from RabbitMQ, processes, and loads into chef-solr</span>
</p>
<h2 id="SYNOPSIS">SYNOPSIS</h2>
<p><strong>chef-expander-cluster</strong> <em>(options)</em></p>
<dl>
<dt><code>-c</code>, <code>--config CONFIG_FILE</code></dt><dd>a configuration file to use</dd>
<dt><code>-i</code>, <code>--index INDEX</code></dt><dd>the slot this node will occupy in the ring</dd>
<dt><code>-n</code>, <code>--node-count NUMBER</code></dt><dd>the number of nodes in the ring</dd>
<dt><code>-l</code>, <code>--log-level LOG_LEVEL</code></dt><dd>set the log level</dd>
<dt><code>-L</code>, <code>--logfile LOG_LOCATION</code></dt><dd>Logfile to use</dd>
<dt><code>-d</code>, <code>--daemonize</code></dt><dd>fork into the background</dd>
<dt><code>-P</code>, <code>--pid PIDFILE</code></dt><dd>PID file</dd>
<dt><code>-h</code>, <code>--help</code></dt><dd>show help message</dd>
<dt><code>-v</code>, <code>--version</code></dt><dd>show the version and exit</dd>
</dl>
<h2 id="DESCRIPTION">DESCRIPTION</h2>
<p>Chef Expander fetches messages from RabbitMQ, processes them into the
correct format to be loaded into Solr and loads them into Solr.</p>
<p><strong>Running Chef Expander</strong></p>
<p>Chef Expander is designed for clustered operation, though small
installations will only need one worker process. To run Chef
Expander with one worker process, run chef-expander-cluster -n 1.
You will then have a master and worker process, which looks like
this in ps:</p>
<pre><code>your-shell> ps aux|grep expander
you 52110 0.1 0.7 2515476 62748 s003 S+ 3:49PM 0:00.80 chef-expander worker #1 (vnodes 0-1023)
you 52108 0.1 0.5 2492880 41696 s003 S+ 3:49PM 0:00.91 ruby bin/chef-expander-cluster -n 1
</code></pre>
<p>Workers are single threaded and therefore cannot use more than 100%
of a single CPU. If you find that your queues are getting backlogged,
increase the number of workers</p>
<p><strong>Design</strong></p>
<p>Chef Expander uses 1024 queues (called vnodes in some places) to allow
you to scale the number of Chef Expander workers to meet the needs of
your infrastructure. When objects are saved in the API server, they are
added to queues based on their database IDs. These queues can be assigned
to different Chef Expander workers to distribute the load of processing
the index updates.</p>
<p><strong>Chef Expander Operation and Troubleshooting</strong></p>
<p>Chef Expander includes chef-expanderctl, a management program that allows
you to get status information or change the logging verbosity (without
restarting).</p>
<p>See <strong>chef-expanderctl</strong>(8) for details.</p>
<h2 id="SEE-ALSO">SEE ALSO</h2>
<p><strong>chef-expanderctl</strong>(8)
<strong>chef-solr</strong>(8)</p>
<p>Full documentation for Chef and chef-server is located on the Chef
wiki, http://wiki.opscode.com/display/chef/Home.</p>
<h2 id="AUTHOR">AUTHOR</h2>
<p>Chef was written by Adam Jacob <a href="mailto:adam@ospcode.com" data-bare-link="true">adam@ospcode.com</a> of Opscode
(http://www.opscode.com), with contributions from the community. This
manual page was created by Nuo Yan <a href="mailto:nuo@opscode.com" data-bare-link="true">nuo@opscode.com</a>. Permission is
granted to copy, distribute and / or modify this document under the
terms of the Apache 2.0 License.</p>
<p>On Debian systems, the complete text of the Apache 2.0 License can be
found in /usr/share/common-licenses/Apache-2.0.</p>
<ol class='man-decor man-foot man foot'>
<li class='tl'>Chef 0.10.0.beta.10</li>
<li class='tc'>April 2011</li>
<li class='tr'>chef-expander-cluster(8)</li>
</ol>
</div>
</body>
</html>