Skip to content

Commit

Permalink
Offline install docs
Browse files Browse the repository at this point in the history
- Added "Offline" installation notes and CentOS/RHEL install notes
- Added release notes for v0.5.2 (whoops)
- Fixed issue in CLI main() where exceptions were raised instead of exiting
  with the appropriate return code.
  • Loading branch information
lowell80 committed Sep 5, 2018
1 parent bb93d61 commit fb6686c
Show file tree
Hide file tree
Showing 4 changed files with 117 additions and 11 deletions.
17 changes: 13 additions & 4 deletions docs/source/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,23 @@

Add Python 3 support, external command plugins, tox and vagrant for testing.

### Release v0.5.3 (UNRELEASED)
* Expand install docs specifically for offline mode and some OS-specific notes.
* Add changelog for v0.5.2 (whoops)

### Release v0.5.2 (2018-08-13)
* Expand CLI output for `--help` and `--version`
* Internal cleanup of CLI entry point module name. Now the ksconf CLI can be invoked as
`python -m ksconf`, you know, for anyone who's into that sort of thing.
* Minor docs and CI/testing improvements.

### Release v0.5.1 (2018-06-28)
* Support external ksconf command plugins through custom 'entry_points', allowing for others to
develop their own custom extensions as needed.
* Many internal changes: Refactoring of all CLI commands to use new entry_points as well as pave
the way for future CLI unittest improvements.
* Docs cleanup / improvements.


### Release v0.5.0 (2018-06-26)
* Python 3 support.
* Many bug fixes and improvements resulting from wider testing.
Expand Down Expand Up @@ -45,7 +54,7 @@ registration (installation via `pip install` and, online docs.

### Release v0.4.3 (2018-06-04)
* Rename PyPI package `kintyre-splunk-conf`
* Add support for building a standalone executable (zipapp).
* Add support for building a standalone executable (zipapp).
* Revamp install docs and location
* Add GitHub release for the standalone executable.

Expand All @@ -57,7 +66,7 @@ registration (installation via `pip install` and, online docs.

### Release v0.4.0 (2018-05-19)
* Refactor entire code base. Switched from monolithic all-in-one file to clean-cut modules.
* Versioning is now discoverable via `ksconf --version`, and controlled via git tags (via
* Versioning is now discoverable via `ksconf --version`, and controlled via git tags (via
`git describe --tags`).

#### Module layout
Expand All @@ -75,7 +84,7 @@ First public releases.
### Release v0.3.2 (2018-04-24)
* Add AppVeyor for Windows platform testing
* Add codecov integration
* Created ConfFileProxy.dump()
* Created ConfFileProxy.dump()

### Release v0.3.1 (2018-04-21)
* Setup automation via Travis CI
Expand Down
99 changes: 97 additions & 2 deletions docs/source/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ Please change `venv` to a suitable path for your environment.
***Note: This requires admin access.***

This is the absolute easiest install method where 'ksconf' is available to all users on the system
but it requires root access.
but it requires root access and `pip` must be installed and up-to-date.

On Mac or Linux, run:

Expand All @@ -110,6 +110,18 @@ On Windows, run this commands from an Administrator console.
pip install kintyre-splunk-conf


### CentOS (RedHat derived) distros

# Enable the EPEL repo so that `pip` can be installed.
sudo yum install -y epel-release

# Install pip
sudo yum install -y python-pip

# Install ksconf (globally, for all users)
sudo pip install kintyre-splunk-conf


### Install from GIT

If you'd like to contribute to ksconf, or just build the latest and greatest, then install from the
Expand All @@ -127,7 +139,7 @@ See [developer docs](devel.html) for additional details about contributing to ks

Ksconf can be installed as a standalone executable zip app. This approach still requires a python
interpreter to be present either from the OS or the one embedded with Splunk Enterprise. This works
well for testing or when all other options fail.
well for testing or when all other options fail.

From the [GitHub releases][gh-releases] page, grab the file name `ksconf-*.pyz`, download it, copy
it to a `bin` folder in your PATH and rename it `ksconf`. The default shebang looks for 'python' in
Expand Down Expand Up @@ -288,6 +300,89 @@ OS-specific notes:
subsystem for Windows instead.


## Offline installation

Installing ksconf to an offline or network restricted computer requires three steps: (1) download
the latest packages from the Internet to a staging location, (2) transfer the staged content (often
as a zip file) to the restricted host, and (3) use pip to install packages from the staged copy.
Fortunately, pip makes offline workflows quite easy to achieve. Pip can download a python package
with all dependencies stored as wheels files into a single directory, and pip can be told to install
from that directory instead of attempting to talk to the Internet.

The process of transferring these files is very organization-specific. The example below shows the
creation of a tarball (since `tar` is universally available on Unix systems), but any acceptable
method is fine. If security is a high concern, this step is frequently where safety checks are
implemented. For example, antivirus scans, static code analysis, manual inspection, and/or
comparison of cryptographic file hashes.

One additional use-case for this workflow is to ensure the exact same version of all packages are
deployed consistently across all servers and environments. Often building a `requirements.txt` file
by running `pip freeze` is a more appropriate solution, but this is technically more secure.


### Offline installation steps

**NOTE:** Pip must be installed on the destination server for this process to work. If pip is NOT
installed see the *Offline installation of pip* section below.


**Step 1**: Use pip to download the latest package and their dependencies. Be sure to use the same
version of python that is running on destination machine

# download packages
python2.7 -m pip download -d ksconf-packages kintyre-splunk-conf

A new directory named 'ksconf-packages' will be created and will contain the neccesary `*.whl` files.

**Step 2**: Transfer the directory or archive to the remote computer. Insert whatever security and
file copy procedures necessary for your organization.

# Compress directory (on staging computer)
tar -czvf ksconf-packages.tgz ksconf-packages

# Copy file using whatever means
scp ksconf-packages.tgz user@server:/tmp/ksconf-packages.tgz

# Extract the archive (on destination server)
tar -xzvf ksconf-packages.tgz

**Step 3**:

# Install ksconf package with pip
pip install --no-index --find-links ksconf-packages kntyre-splunk-conf

# Test the installation
ksconf --version

The `ksconf-packages` folder can now safely be removed.


### Offline installation of pip

Use the recommended `pip` install procedures listed elsewhere if possible. But if a remote
bootstrap of pip is your only option, then here are the steps. (This process mirrors the steps
above and can be combined, if needed.)


**Step 1**: Fetch boostrap script and necessary wheels

mkdir ksconf-packages
curl https://bootstrap.pypa.io/get-pip.py -o ksconf-packages/get-pip.py
python2.7 -m pip download -d /tmp/my_packages pip setuptools wheel

The `ksconf-pacakges` folder should contain 1 script, and 3 wheel (`*.whl) files.

**Step 2**: Archive and/or copy to offline server

**Step 3**: Boostrap pip

sudo python get-pip.py --no-index --find-links=ksconf-packages/

# Test with
pip --version



## Frequent gotchas

### PIP Install TLS Error
Expand Down
3 changes: 2 additions & 1 deletion ksconf/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import ksconf.util
from ksconf.commands import KsconfCmd, MyDescriptionHelpFormatter, get_entrypoints
from ksconf.util.completers import autocomplete
from ksconf.consts import EXIT_CODE_INTERNAL_ERROR

###################################################################################################
## CLI definition
Expand Down Expand Up @@ -117,8 +118,8 @@ def cli(argv=None, _unittest=False):
try:
return_code = args.funct(args)
except Exception as e: # pragma: no cover
# Todo: Make a CLI arg or ENV var to enable stacktrace for debugging
sys.stderr.write("Unhandled top-level exception. {0}\n".format(e))
raise
return_code = EXIT_CODE_INTERNAL_ERROR

if _unittest:
Expand Down
9 changes: 5 additions & 4 deletions ksconf/setup_entrypoints.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
"""
""" Defines all command prompt entry points for CLI actions
This is a silly hack that serves 2 purposes:
(1) It works around and apparent Python 3.4/3.5 bug on Windows where [options.entry_point] in
setup.cfg is ignored.. hence 'ksconf' isn't installed and custom ksconf_* entry points are
not available. (So everything breaks)
setup.cfg is ignored hence 'ksconf' isn't installed as a console script and custom ksconf_*
entry points are not available. (So no CLI commands are available)
(2) It allows for fallback mechanism when
(a) running unit tests (can happen before install)
(b) if entrypoints or pkg_resources are not available at run time.
(b) if entrypoints or pkg_resources are not available at run time (Splunk's embedded python)
"""

from __future__ import absolute_import, unicode_literals
Expand Down

0 comments on commit fb6686c

Please sign in to comment.