Skip to content
Browse files

add performance tutorial covering db indices

  • Loading branch information...
1 parent f118f5b commit 63255a75cab7309f7dbfc5e19a2d70ea938ea3d7 @computron computron committed Jul 10, 2013
Showing with 34 additions and 5 deletions.
  1. +0 −2 README.rst
  2. +1 −1 docs/duplicates_tutorial.rst
  3. +1 −0 docs/index.rst
  4. +32 −0 docs/performance_tutorial.rst
  5. +0 −2 docs/todos.rst
View
2 README.rst
@@ -4,8 +4,6 @@ FireWorks
FireWorks helps run calculation workflows, with a centralized workflow server controlling many worker nodes.
-It is currently in development mode.
-
For more about FireWorks, please visit the official documentation_.
.. _documentation: http://packages.python.org/FireWorks/
View
2 docs/duplicates_tutorial.rst
@@ -112,4 +112,4 @@ The built-in duplicate finder, ``Exact Dupe Finder``, suffers from two limitatio
* performance is not great when the number of FireWorks is large
* matching is limited to *exact* matches of the FireWork spec. You cannot, for example, define two FireWorks to be duplicated if a portion of the spec matches within some numerical tolerance.
-In the future, we will include a tutorial on implementing custom Dupe Finders for your application that overcome these limitations. For now, we suggest that you refer to the internal docs or contact us for help. (see :ref:`contributing-label`)
+In the future, we will include a tutorial on implementing custom Dupe Finders for your application that overcome these limitations. For now, we suggest that you refer to the internal docs or contact us for help. (see :ref:`contributing-label`). You can also try to improve performance by manually adding database indices to :doc:`improve performance <performance_tutorial>`.
View
1 docs/index.rst
@@ -129,6 +129,7 @@ Deploying FireWorks in production
security_tutorial
config_tutorial
+ performance_tutorial
Additional features and topics
==============================
View
32 docs/performance_tutorial.rst
@@ -0,0 +1,32 @@
+===============================
+Improving FireWorks performance
+===============================
+
+For the most part, you shouldn't need to tinker with FireWork's performance. One issue you might run into is speed of querying FireWorks by their spec or workflows by a field in their metadata. This issue might also manifest itself as slow duplicate checking queries. You can add *indices* to certain fields of your FireWork specification in order to improve query performances.
+
+How to add an index to a FW spec
+================================
+
+If you are initializing the LaunchPad using Python, set the ``user_indices`` argument to contain an array the fields you want to index. Make sure to prefix the names with ``spec``. For example, you might put ``["spec.parameter1", "spec.parameter2"]``.
+
+If you are using a ``my_launchpad.yaml`` file, add the array in YAML format to a key called ``user_indices``. e.g., add the following::
+
+ user_indices:
+ - spec.parameter1
+ - spec.parameter2
+
+How to add an index to Workflow metadata
+========================================
+
+If you are initializing the LaunchPad using Python, set the ``wf_user_indices`` argument to contain an array the fields you want to index. Make sure to prefix the names with ``metadata``. For example, you might put ``["metadata.parameter1", "metadata.parameter2"]``.
+
+If you are using a ``my_launchpad.yaml`` file, add the array in YAML format to a key called ``wf_user_indices``. e.g., add the following::
+
+ wf_user_indices:
+ - metadata.parameter1
+ - metadata.parameter2
+
+Further performance tweaks
+==========================
+
+A few other (very minor) performance tuning parameters are available via the :doc:`FW configuration <config_tutorial>`, although in most cases you shouldn't need to change these.
View
2 docs/todos.rst
@@ -28,8 +28,6 @@ FW Docs
* Detailed tutorial on implementing dynamic jobs
-* Document allow_fizzled_parents
-
* Separate out the tutorial on running arbitrary Python code
* Give examples on how to implement certain things, e.g. a workflow where one step gives an output file used as input by the second step.

0 comments on commit 63255a7

Please sign in to comment.
Something went wrong with that request. Please try again.