Skip to content

Commit

Permalink
Britt docs (#58)
Browse files Browse the repository at this point in the history
* Grammar and typo edits to KCONF documents

* Additional cmd file edits for KSCONF

* Final Edit of KSCONF documentation

* Lowell breaking more docs...

* Another possible spelling of ksconf

* Ran Pre-Commit Hooks

* Splunk Documentation Edits

* Grammar edits for setup.py

* Minor python package description typo

* Switch back to 'Ksconf'  (ed8e1fe)

- Revert movement to 'KSConf'.  I'm just not ready to endorse that spelling yet.
  After and official decision is made, we can use this commit to find all
  instances Brittany identified where a proper noun spelling is appropriate.
  Isolating these changes allows me to easily change my mind later ;-)

I suspect there will always be 3 spellings:
  (1) lower case for CLI examples,
  (2) UPPER for titles (plain-text emphasis), and
  (3) proper noun or title, which for now remains 'Ksconf'.
  • Loading branch information
bbarnett07 authored and lowell80 committed Jul 25, 2019
1 parent 3fda6f2 commit 77e61ec
Show file tree
Hide file tree
Showing 42 changed files with 373 additions and 371 deletions.
2 changes: 1 addition & 1 deletion .splunkbase/description.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
Git and Splunk don't always fit together perfectly. Introducing KSCONF, a tool written to alleviate painful tasks associated with managing Splunk apps in a git repository. This open source tool supports many functions to help both admins and developers manage Splunk content in git in a simple way without getting stuck in the details.
Git and Splunk don't always fit together perfectly. Introducing KSCONF, a tool written to alleviate arduos tasks associated with managing Splunk apps in a git repository. This open source tool supports many functions to help both admins and developers manage Splunk content in git in a simple way without getting stuck in the details.

https://github.com/Kintyre/ksconf
18 changes: 9 additions & 9 deletions .splunkbase/details.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@

# What is KSCONF?

KSCONF is a command-line tool that helps administrators and developers manage their Splunk environments by enhancing control of their configuration files. The interface is modular so that each function (or subcommand) can be learned quickly and used independently. While most users will probably only use a subset of the total capabilities of this tool, it’s reassuring to have a deep toolbox of power goodies ready to be unleashed at a moments notice. Ksconf works with (and does not replace) your existing Splunk deployment mechanisms and version control tools.
KSCONF is a command-line tool that helps administrators and developers manage their Splunk environments by enhancing control of their configuration files. The interface is modular so that each function (or subcommand) can be learned quickly and used independently. While most users will probably only use a subset of the total capabilities of this tool, it’s reassuring to have a comprehensive toolbox of powerful assets ready to be utilized at a moment's notice. Ksconf works with, rather than replace, your existing Splunk deployment mechanisms and version control tools.

KSCONF is open source and an open development effort. Check us out on [GitHub](https://github.com/Kintyre/ksconf#kintyres-splunk-configuration-tool)

Expand All @@ -11,8 +11,8 @@ Pronounced: k·s·kȯnf

- *Ksconf is a toolbox.* - Each tool has a specific purpose and function that works independently. Borrowing from the Unix philosophy, each command should do one small thing well and be easily combined to handle higher-order tasks.
- *When possible, be familiar.* - Various commands borrow from popular UNIX command line tools such as “grep” and “diff”. The overall modular nature of the command is similar to the modular interface used by “git” and the “splunk” cli.
- *Don’t impose workflow.* - Ksconf works with or without version control and independently of your deployment mechanisms. (If you are looking to implement these things, ksconf is a great building block)
- *Embrace automated testing.* - It’s impractical to check every scenarios between each release, but significant work has gone into unittesting the CLI to avoid breaks between releases.
- *Don’t impose workflow.* - Ksconf works with or without version control and independently of your deployment mechanisms. If you are looking to implement these things, ksconf is a great building block.
- *Embrace automated testing.* - It’s impractical to check every scenario between each release, but significant work has gone into unit testing the CLI to avoid breaks between releases.

## Common uses for ksconf
- Promote changes from “local” to “default”
Expand All @@ -26,12 +26,12 @@ Pronounced: k·s·kȯnf

This Splunk app comes bundled with a CLI tool that helps manage other Splunk apps. While this is not a traditional use case for a Splunk app, it is a very quick and easy way to deploy ksconf.

Why did we make this a Splunk app? Well, while ksconf is technically just a Python package that can be deployed in a variety of ways, we found that the logistics of getting it deployed can be quite difficult due to a packaging issues, legacy cruft, and OS limitations. This approach avoids all that mess.
Why did we make this a Splunk app? While ksconf is technically just a Python package that can be deployed in a variety of ways, we found that the logistics of getting it deployed can be quite difficult due to a packaging issues, legacy cruft, and OS limitations. This approach avoids all that mess.


# Getting Started

Full documentation for ksconf and, therefore this app, is hosted at read-the-docs. A full copy of the `ksconf` documentation is also included, just like how Splunk ships with a fully copy of the docs in the system/README folder. (And all the air-gapped people rejoice! but sadly, no one could hear them.)
Full documentation for ksconf, including this app, is hosted at read-the-docs. A full copy of the `ksconf` documentation is also included, similar to how Splunk ships with a full copy of the docs in the system/README folder. (And all the air-gapped people rejoice! but sadly, no one could hear them.)


## Docs
Expand All @@ -53,20 +53,20 @@ Full documentation for ksconf and, therefore this app, is hosted at read-the-doc

## Roadmap

Additional Splunk UI feature are planned, but currently not implemented.
Additional Splunk UI features are planned, but currently not implemented.

* Dashboard to track all changes coordinated by `ksconf`
* Configuration snapshot tracking
* Custom SPL command to give visibility into the what exists in the `local` folder. (The built-in `rest` command only shows you the final merged view of your settings; and sometimes you have to look deeper.)
* Custom SPL command to give visibility into what exists in the `local` folder. (The built-in `rest` command only shows you the final merged view of your settings; and sometimes you have to look deeper.)

## Installation & Configuration

See the [Install an add-on](https://docs.splunk.com/Documentation/AddOns/released/Overview/Singleserverinstall) in Splunk's official documentation. There is one manual step required to active the CLI portion of this app, if you choose to do so. See the [Installation docs](https://ksconf.readthedocs.io/en/latest/install.html) for more details.
See the [Install an add-on](https://docs.splunk.com/Documentation/AddOns/released/Overview/Singleserverinstall) in Splunk's official documentation. There is one manual step required to activate the CLI portion of this app, if you choose to do so. See the [Installation docs](https://ksconf.readthedocs.io/en/latest/install.html) for more details.

## Support

Community support is available on best-effort basis. For information about commercial support, contact [Kintyre](mailto:hello@kintyre.co)
Issues are tracked via [GitHub](https://github.com/Kintyre/ksconf/issues)

## History
See the full [Change log](https://ksconf.readthedocs.io/en/latest/changelog.html)
See the full [Change log](https://ksconf.readthedocs.io/en/latest/changelog.html)
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@
![Ksconf logo][logo]

This utility handles a number of common Splunk app maintenance tasks in an installable python
package. Specifically, this tools deals with many of the nuances with storing Splunk apps in a
version control system like git and pointing live Splunk apps to a working tree, merging changes
from the live system's (local) folder to the version controlled (default) folder, and dealing with
more than one layer of "default" (which splunk can't handle natively).
package. Specifically, this tools deals with many of the nuances of storing Splunk apps in a
version control system like git and pointing live Splunk apps to a working tree. Merging changes
from the live system's (local) folder to the version controlled (default) folder and dealing with
more than one layer of "default" are all supported tasks which are not native to Splunk.


## Install
Expand Down
26 changes: 14 additions & 12 deletions docs/source/cheatsheet.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ Here's a quick rundown of handy ``ksconf`` commands:

.. note::

Note that for clarity, most of the command line arguments are given in their long form.
Many options also have a short form too.
Note that for clarity, most of the command line arguments are given in their long form,
but many options also have a short form.

Long commands may be broken across line for readability. When this happens, a trailing
backslash (``\``) is added so the command could still be copied verbatim into most shells.
Expand Down Expand Up @@ -41,7 +41,7 @@ Show the differences between two conf files using :ref:`ksconf_cmd_diff`.
Sorting content
~~~~~~~~~~~~~~~

Create a normalized version a configuration file, making conf files easier to merge with :command:`git`.
Create a normalized version of a configuration file, making conf files easier to merge with :command:`git`.
Run an in-place sort like so:

.. code-block:: sh
Expand Down Expand Up @@ -97,7 +97,9 @@ Cleaning up
Reduce cruft in local
~~~~~~~~~~~~~~~~~~~~~~~

If you're in the habit of copying the *default* files to *local* in the TAs you deploy, here a quick way to 'minimize' your files. This will reduce the *local* file by removing all the *default* settings you copied but didn't change. (The importance of this is outlined in :ref:`minimizing_files`.)
If you're in the habit of copying the *default* files to *local* in the TAs you deploy, here is a quick way to 'minimize' your files.
This will reduce the *local* file by removing all the *default* settings you copied but didn't change.
(The importance of this is outlined in :ref:`minimizing_files`.)

.. code-block:: sh
Expand All @@ -107,14 +109,14 @@ If you're in the habit of copying the *default* files to *local* in the TAs you
Pushing local changes to default
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

App developers can push changes from the :file:`local` folder over to the :file:`default` folder:
App developers can push changes from the :file:`local` folder to the :file:`default` folder:

.. code-block:: sh
ksconf promote --interactive myapp/local/props.conf myapp/default/props.conf
You will be prompted to pick which items you want to promote.
Or use the ``--batch`` option to promote everything in one step, without reviewing the changes first.
Alternatively, use the ``--batch`` option to promote everything in one step, without reviewing the changes first.



Expand All @@ -127,15 +129,15 @@ Migrating content between apps


Say you want to move a bunch of savedsearches from ``search`` into a more appropriate app.
First create a file that list all the names of your searches (one per line) in :file:`corp_searches.txt`.
First create a file that lists all the names of your searches (one per line) in :file:`corp_searches.txt`.
Next, copy just the desired stanzas, those named in the 'corp_searches' file, over to your new :file:`corp_app` application.

.. code-block:: sh
ksconf filter --match string --stanza 'file://corp_searches.txt' \
search/local/savedsearches.conf --output corp_app/default/savedsearches.conf
And now, to avoid duplication and confusion, you want to remove that exact same set of searches from the search app.
Now, to avoid duplication and confusion, you want to remove that exact same set of searches from the search app.

.. code-block:: sh
Expand Down Expand Up @@ -164,7 +166,7 @@ Migrating the 'users' folder
Say you stood up a new Splunk server and the migration took longer than expected.
Now you have two :file:`users` folders and don't want to loose all the goodies stored in either one.
You've copied the users folder to :file:`user_old`.
You're working from the new server and would generally prefer to keep whatever on the new server over what's on the old.
You're working from the new server and would generally prefer to keep whatever is on the new server over what is on the old.
(This is because some of your users copied over some of their critical alerts manually while waiting for the migration to complete, and they've made updates they don't want to lose.)


Expand Down Expand Up @@ -208,7 +210,7 @@ Putting it all together
Pulling out a stanza defined in both default and local
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Say wanted to count the number of searches containing the word ``error``
Say you wanted to count the number of searches containing the word ``error``


.. code-block:: sh
Expand All @@ -228,7 +230,7 @@ Building an all-in one TA for your indexing tier
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Say you need to build a single TA containing all the index-time settings for your indexing tier.
(Note: Enterprise Security does something similar this whenever they generate the indexer app.)
(Note: Enterprise Security does something similar when generating the indexer app.)

.. code-block:: sh
Expand All @@ -241,7 +243,7 @@ Say you need to build a single TA containing all the index-time settings for you
--include-attr 'EVENT_BREAKER*' \
--include-attr 'LINE_BREAKER*'
This example is incomplete because it doesn't list *every* index-time :file:`props.conf` attribute, and leaves out file:`transforms.conf` and :file:`fields.conf`, but hopefully you get the idea.
This example is incomplete because it doesn't list *every* index-time :file:`props.conf` attribute, and leaves out :file:`transforms.conf` and :file:`fields.conf`, but hopefully you get the idea.



Expand Down
4 changes: 2 additions & 2 deletions docs/source/cmd.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ The ksconf command documentation is provided in the following ways:
1. A detailed listing of each sub-command is provided in this section.
This includes relevant background descriptions, typical use cases, examples, and discussion of
relevant topics. An expanded descriptions of CLI arguments and their usage is provided here.
If you've not used a particular command before, start here.
2. The :doc:`Command line reference<dyn/cli>` provides a quick an convenient reference when
If you have not used a particular command before, start here.
2. The :doc:`Command line reference<dyn/cli>` provides a quick and convenient reference when
the command line is unavailable. The same information is available by typing ``ksconf <CMD> --help``.
This is most helpful if you're already familiar with a command, but need a quick refresher.

Expand Down
12 changes: 6 additions & 6 deletions docs/source/cmd_check.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ ksconf check
How 'check' differs from btool's validation
--------------------------------------------

Keep in mind that ksconf idea of *valid* is different than Splunk's. Specifically,
Keep in mind that idea of *valid* in ksconf is different than within Splunk. Specifically,

- **Ksconf is more picky syntactically.** Dangling stanzas and junk lines are picked up by
ksconf in general (the 'check' command or others), but silently ignored Splunk.
- **Btool handles content validation.** The :command:`btool check` mode does a great job of check
ksconf in general (the 'check' command or others), but silently by ignored Splunk.
- **Btool handles content validation.** The :command:`btool check` mode does a great job of checking
stanza names, attribute names, and values. Btool does this well and ksconf tries to not repeat
things that Splunk already does well.

Expand All @@ -50,8 +50,8 @@ Can you spot the error in this :file:`props.conf`?
That's right, line 7 contains the stanza ``myapp:total:junk`` that doesn't have a closing ``]``.
How Splunk handle this? It ignores the broken stanza header completely and therefore ``TRANSFORMS-drop`` gets added
to the ``myapp:web:access`` sourcetype and very likely going to start loosing data.
How does Splunk handle this? It ignores the broken stanza header completely and therefore ``TRANSFORMS-drop`` gets added
to the ``myapp:web:access`` sourcetype, which will likely result in the loss of data.


Splunk also ignores entries like this:
Expand All @@ -65,7 +65,7 @@ ignores it.

.. tip::

If you want to see how different this is. Run ksconf check against the system default files:
If you want to see how different this is, run ksconf check against the system default files:

.. code-block:: sh
Expand Down
17 changes: 8 additions & 9 deletions docs/source/cmd_combine.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,16 +22,15 @@ ksconf combine

You may have noticed similarities between the ``combine`` and :ref:`merge <ksconf_cmd_merge>`
subcommands. That's because under the covers they are using much of the same code. The combine
operations essentially does a recursive merge between a set of directories. One big difference is
that ``combine`` command will gracefully handle non-conf files intelligently, not just conf files.
operation essentially does a recursive merge between a set of directories. One big difference is
that ``combine`` command will handle non-conf files intelligently, not just conf files.


.. note:: Mixing layers

Just like all layers can be managed independently, they can also be combined in any way you'd
like. While this workflow is out side the scope of the examples provided here, it's very doable.
This also allows for different layers to be mixed-and-matched by selectively including which
layers to combine.
Just like all layers can be managed independently, they can also be combined in any way you would like.
While this workflow is outside of the scope of the examples provided here, it's certainly feasible.
This also allows for different layers to be mixed-and-matched by selectively including which layers to combine.

Examples
--------
Expand Down Expand Up @@ -91,8 +90,8 @@ In this structure, you can see several layers of configurations at play:
Note that since ``user_tracking.xml`` is not a ``.conf`` file it will fully replace the
upstream default version (that is, the file in ``10-upstream``)

Here's are the commands that could be used to generate a new (merged) ``default`` folder from all
these layers shown above.
Here are the commands that could be used to generate a new (merged) ``default`` folder from all
of the layers shown above.

.. code-block:: sh
Expand All @@ -109,5 +108,5 @@ these layers shown above.
Consolidating 'users' directories
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The ``combine`` can consolidate 'users' directory across several instances after a phased server migration.
The ``combine`` command can consolidate 'users' directory across several instances after a phased server migration.
See :ref:`example_combine_user_folder`.

0 comments on commit 77e61ec

Please sign in to comment.