A tool for discovery, inspection, collection/deduplication, and reporting on an IT environment
Clone or download
kholdaway Updates to ensure partial results are recorded for multiple source sc…
…ans (#1601)

* Updates to ensure partial results are recorded for multiple source scans
* Improve satellite 5 scanning
Latest commit f661e28 Oct 12, 2018
Permalink
Failed to load latest commit information.
.github/ISSUE_TEMPLATE Update issue templates for feature request (#1409) Aug 30, 2018
bin work around ansible bug where command must be called ssh (#1283) Apr 24, 2018
client fix(ui linter): ui activate linter (#1588) Oct 9, 2018
deploy Add async merge jobs endpoints (#1557) Oct 4, 2018
docs Renamed job command to match prefered syntax of insights #1585 (#1587) Oct 9, 2018
install Installation scripts modified to support release upgrades (#1538) Oct 4, 2018
qpc Renamed job command to match prefered syntax of insights #1585 (#1587) Oct 9, 2018
quipucords Updates to ensure partial results are recorded for multiple source sc… Oct 12, 2018
rel-eng COPR no longer builds fedora 26 so this updates to 27/28 (#1448) Sep 10, 2018
roles Fix yum_enabled_repolist fact (#1577) Oct 7, 2018
test_util Timeout individual Ansible tasks (#536) Jan 30, 2018
.coveragerc Remove elasticsearch hooks (#699) Feb 12, 2018
.dockerignore Fix role issue injected when moved to using supervisord. Closes #646. ( Feb 7, 2018
.env.example Add instructions for setting up .env (#1598) Oct 11, 2018
.gitignore Updates to ensure partial results are recorded for multiple source sc… Oct 12, 2018
.landscape.yml Add landscape.io and requires.io integration to quipucords. Closes #759 Feb 16, 2018
.pyup.yml add dev locked requirments to pyup (#1581) Oct 8, 2018
.travis.yml Update pip before installing packages on Travis Oct 3, 2018
AUTHORS.rst Add https support to server with defaulted key and cert. Closes #428. ( Jan 30, 2018
CHANGES.rst Create initial project/server structure. (#12) Sep 8, 2017
CONTRIBUTING.rst Reverse the order that dependencies are installed (Make dev-requireme… Sep 20, 2018
Dockerfile copy locked-requirments over (#1565) Oct 4, 2018
Jenkinsfile Build and test quipucords on a single pipeline Oct 10, 2018
LICENSE Initial commit Jun 15, 2017
Makefile Move fingerprint engine into a task (#1544) Oct 3, 2018
README.rst Add instructions for setting up .env (#1598) Oct 11, 2018
dev-locked-requirements.txt Move pycrypto to locked dependencies (#1568) Oct 5, 2018
dev-requirements.txt Scheduled weekly dependency update for week 40 (#1582) Oct 8, 2018
locked-requirements.txt Move pycrypto to locked dependencies (#1568) Oct 5, 2018
qpc.spec Add qpc server status command to CLI (#1579) Oct 8, 2018
requirements.txt Move pycrypto to locked dependencies (#1568) Oct 5, 2018
setup.py Use flake8-import-order in linting. Closes #1167. (#1168) Apr 3, 2018

README.rst

https://travis-ci.org/quipucords/quipucords.svg?branch=master Code Health Requirements Status Documentation Status CLI RPM Build Status

quipucords - Tool for discovery, inspection, collection, deduplication, and reporting on an IT environment

quipucords is a tool for discovery, inspection, collection, deduplication, and reporting on an IT environment.

This README file contains information about the installation and development of quipucords, as well as instructions about where to find basic usage, known issue, and best practices information.

Introduction to quipucords

quipucords is a Python based information gathering tool. quipucords provides a server base infrastructure for process tasks that discover and inspect remote systems by utilizing Ansible while additionally looking to integrate and extract data from systems management solutions. quipucords collects basic information about the operating system, hardware, and application data for each system. quipucords is intended to help simplify some of the basic system administrator tasks that are a part of the larger goal of managing licensing renewals and new deployments.

Requirements and Assumptions

Before installing quipucords on a system, review the following guidelines about installing and running quipucords:

  • quipucords is written to run on RHEL or Fedora servers.
  • The system that quipucords is installed on must have access to the systems to be discovered and inspected.
  • The target systems must be running SSH.
  • The user account that quipucords uses for the SSH connection into the target systems must have adequate permissions to run commands and read certain files, such as privilege escalation required for the systemctl command.
  • The user account that quipucords uses for a machine requires an sh shell or a similar shell. For example, the shell cannot be a /sbin/nologin or /bin/false shell.

The Python packages that are required for running quipucords on a system can be found in the dev-requirements.txt file. The Python packages that are required to build and test quipucords from source can be found in the requirements.txt and dev-requirements.txt files.

Installation

quipucords is delivered with an RPM command line tool and a server container image. The following information contains instructions for installing each of these items.

Command Line

qpc is available for download from the Fedora COPR.

  1. Enable the EPEL repo for the server. You can find the appropriate architecture and version on the EPEL wiki:

    rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
    
  2. Add the COPR repo to your server. You can find the appropriate architecture and version on the COPR qpc page:

    wget -O /etc/yum.repos.d/group_quipucords-qpc-epel-7.repo \
    https://copr.fedorainfracloud.org/coprs/g/quipucords/qpc/repo/epel-7/group_quipucords-qpc-epel-7.repo
    
  3. Install the qpc package:

    yum -y install qpc
    

Command Syntax and Usage

The complete list of options for each qpc command and subcommand are listed in the qpc man page. The man page information also contains usage examples and some best practice recommendations.

For expanded information on credential entries, sources, scanning, and output, see the syntax and usage document.

Development

To work with the quipucords code, begin by cloning the repository:

git clone git@github.com:quipucords/quipucords.git

quipucords currently supports Python 3.4, 3.5 and 3.6. If you do not have Python on your system, follow these instructions.

Setting Up a Virtual Environment

Developing inside a virtual environment is recommended. Add desired environment variables to the .env file before creating your virtual environment. You can copy .env.example to get started.

On Mac run the following command to set up a virtual environment:

brew install pipenv
pipenv shell
pip install -r dev-requirements.txt

On Linux run the following command to set up a virtual environment:

sudo yum install python-tools (or dnf for Fedora)
pip3 install pipenv
pipenv shell
pip install -r dev-requirements.txt

Database Options

Quipucords currently supports development in both SQLite and Postgres. The default database is an internal postgres container.

  • Using a Postgres container:

    make setup-postgres
    docker ps
    
  • Using a SQLite DB:

    export QPC_DBMS=SQLite
    

Initializing the Server

  1. To initialize the server with Postgres, run the following command:

    make server-init
    

Both of the above commands create a superuser with name admin and password of pass.

Running the Server

  1. To run the development server using Postgres, run the following command:

    make serve
    

To log in to the server, you must connect to http://127.0.0.1:8000/admin/ and provide the superuser credentials.

After logging in, you can change the password and also go to some of the browsable APIs such as http://127.0.0.1:8000/api/v1/credentials/. To use the command line interface, you can configure access to the server by entering qpc server config. You can then log in by using qpc server login.

If you intend to run on Mac OS, there are several more steps that are required.

  • Increase the maxfile limit as described here.
  • Install sshpass as described here.
  • Install coreutils to obtain the gtimeout command. To do this step, run the brew install coreutils command.
  • If you are running macOS 10.13 or later and you encounter unexpected crashes when running scans, set the environment variable OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES before starting the server. See the explanation for this step here.
  • Install gtimeout using brew install coreutils

Linting

To lint changes that are made to the source code, run the following command:

make lint

Testing

Unit Testing

To run the unit tests, use the following command:

make test

Advanced Topics

Container Image

The quipucords container image can be created from source. This quipucords repository includes a Dockerfile that contains instructions for the image creation of the server. You must have Docker installed to create the image and run the container.

  1. Clone the repository:

    git clone git@github.com:quipucords/quipucords.git
    
  2. Optional - Build UI.:

    make build-ui
    
NOTE: You will need to install NodeJS. See https://nodejs.org/.
  1. Build the Docker image:

    docker -D build . -t quipucords:1.0.0
    
NOTE: The need to use sudo for this step is dependent upon on your system configuration.
  1. There are 3 different options for running the QPC server.
    1. Run the Docker image with Postgres container:

      docker run --name qpc-db -e POSTGRES_PASSWORD=password -d postgres:9.6.10
      docker run --name quipucords --link qpc-db:qpc-link -d -e QPC_DBMS_HOST=qpc-db -p443:443 -i quipucords:1.0.0
      
    2. Run the Docker image with external Postgres container:

      ifconfig (get your computer's external IP if Postgres is local)
      docker run -d --name quipucords -e "QPC_DBMS_PASSWORD=password" -e"QPC_DBMS_HOST=EXTERNAL_IP" -p443:443 -i quipucords:1.0.0
      
    3. Run the Docker image with SQLite:

      docker run -d --name quipucords -e "QPC_DBMS=sqlite" -p443:443 -i quipucords:1.0.0
      
  2. Configure the CLI by using the following commands:

    qpc server config --host 127.0.0.1
    qpc server login
    
  3. You can work with the APIs, the CLI, and UI (visit https://127.0.0.1/ if you installed the UI in step 2 above).

Running quipucords server in gunicorn

You can run the server locally inside of gunicorn. This can be a useful way to debug.

  1. Clone the repository:

    git clone git@github.com:quipucords/quipucords.git
    cd quipucords
    
  2. Switch to quipucords django app module:

    cd quipucords
    
  3. Make symbolic link to ansible roles:

    ln -s ../roles/ roles
    
  4. Install gunicorn:

    pip install gunicorn==19.7.1
    
  5. Start gunicorn:

    gunicorn quipucords.wsgi -c ./local_gunicorn.conf.py
    
  6. Configure the CLI by using the following commands:

    qpc server config --host 127.0.0.1 --port 8000
    qpc server login
    

Issues

To report bugs for quipucords open issues against this repository in Github. Complete the issue template when opening a new bug to improve investigation and resolution time.

Changes

Track and find changes to the tool in CHANGES.

Authors

Authorship and current maintainer information can be found in AUTHORS.

Contributing

See the CONTRIBUTING guide for information about contributing to the project.

Copyright and License

Copyright 2017-2018, Red Hat, Inc.

quipucords is released under the GNU Public License version 3