Skip to content

Commit

Permalink
added store level throttling setting to data import
Browse files Browse the repository at this point in the history
best practice
  • Loading branch information
chaudum committed Oct 20, 2014
1 parent 76c6066 commit 33c154b
Showing 1 changed file with 21 additions and 0 deletions.
21 changes: 21 additions & 0 deletions docs/best_practice/data_import.txt
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,27 @@ a reasonable value (time in ms)::
- :ref:`refresh_data`
- :ref:`sql_ref_refresh_interval`

Store Level Throttling
.......................

If you do not need to query your data during import, which is the case
most of the time, you can change the storage policy of how Lucene indices
are merged. Since merging may affect IO performance, turning merging off
by setting the ``indices.store.throttle.type`` to ``none`` can help
to improve indexing performance.

::

cr> SET GLOBAL TRANSIENT indices.store.throttle.type = 'none';
SET OK (... sec)

However if you want to keep merging turned on during import you can
throttle the merge operation by decreasing the maximum bytes per second
(default ``20mb``)::

cr> SET GLOBAL TRANSIENT indices.store.throttle.max_bytes_per_sec = '5mb';
SET OK (... sec)


Importing Data Using COPY FROM
------------------------------
Expand Down

0 comments on commit 33c154b

Please sign in to comment.