The landscaper constructs a graph describing a computing infrastructure.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
data
documentation
landscaper
requirements
samples
sphinx
tests
.gitignore
AUTHORS
CHANGELOG.md
Dockerfile
LICENSE
NOTICE
README.md
docker-compose.yaml
landscaper.cfg
landscaper.py
requirements.txt
setup.py
start.sh
tox.ini

README.md

Landscaper

  1. Introduction
  2. Installation
  3. Running the Landscaper
  4. Creating a Landscape
  5. Docker Collector
  6. About Landscaper

Introduction

The landscaper constructs a graph describing a computing infrastructure. The graph details what software stacks are running on what virtual infrastructure, and on what physical infrastructure the virtual infrastructure is running.

The data within the landscape is collected via collectors and event listeners. Collectors are provided for physical hardware information (via the output of hwloc and cpuinfo) and for OpenStack environments. Event Listeners are provided for OpenStack.

Installation

Retrieve the code

# Clone the repository 
git clone https://github.com/IntelLabsEurope/landscaper.git
# Change directory to the landscaper root.
cd landscaper

Install python packages

pip install -r requirements.txt

Note that on Ubuntu 16.04.03 LTS, various errors may arise which may be resolved with

sudo apt-get install libssl-dev libcurl4-openssl-dev
sudo pip install -r requirements.txt

Install Neo4j

Follow the neo4j installation manual: https://neo4j.com/docs/operations-manual/current/installation/

The landscaper has been tested against neo4j Community Edition 3.0.9.

Once installed, and with the neo4j service started, navigate to http://localhost:7474. This is neo4j's graphical browser. Set a username and password. Make note of these credentials, as they will need to be configured in the landscaper configuration file, landscaper.cfg.

The landscaper has a pluggable architecture to allow the graph to be persisted to any database. Neo4j is the initial backend supported.

Configure Landscaper

Update the file landscaper.cfg with the appropriate user and password for neo4j.

Running the Landscaper

The landscaper/data directory contains raw files which the landscaper collectors use to construct a landscape. By default, the landscaper comes with sample files that define a simple two node cluster.

Start the Landscaper

From the top level landscaper directory run:

python landscaper.py

The landscaper physical host and physical network collectors parse relevant files in landscaper/data and construct an appropriate landscape in the neo4j database.

The landscape can be browsed using the neo4j web browser.

The sample landscape includes some components at the physical layer. To create a more detailed cloud infrastructure landscape see section "Creating a Landscape" below.

Start the REST API

Set the python path to the landscaper directory. From the top level landscaper directory run:

export PYTHONPATH=`pwd`

Run the webserver.

cd landscaper/web
gunicorn -w 2 application:APP

This will start a webserver running on the localhost at port 8000

Querying the Landscape

The project contains a samples folder which contains example queries for the Landscaper. These queries can be run on the default example landscape. From the top level landscaper directory run:

export PYTHONPATH=`pwd`
cd samples
python query_landscaper_test.py

The command will return a list of the physical machines in the landscape. Additional sample queries are visible within query_landscaper_test.py

Creating a Landscape

Collecting Hardware and CPU Information

Hardware and CPU information can be collected for each physical machine in your landscape.

Execute the following commands on each machine to be included in the landscape, and copy the output files to the data directory of the landscaper.

hwloc-ls --of xml > $HOSTNAME"_hwloc.xml"
cat /proc/cpuinfo > $HOSTNAME"_cpuinfo.txt"

The landscaper.cfg should be updated so that machines lists all HOSTNAMES that are to be landscaped:

machines=HOSTNAME1,HOSTNAME2,HOSTNAME3

Collecting OpenStack Information

Set the following OpenStack environment variables as appropriate for your OpenStack environment:

export OS_TENANT_NAME=
export OS_PROJECT_NAME=
export OS_TENANT_ID=
export OS_USERNAME=
export OS_PASSWORD=
export OS_AUTH_URL=

The openstack collectors listen for events from the OpenStack event rabbitMQ queue. This can be configured in landscaper.cfg by setting the following configuration variables.

[rabbitmq]
rb_name=
rb_password=
rb_host=
rb_port=
topic=
notification_queue=
exchanges=	

Flushing the Neo4j Database

To purge all information from the landscape, in the landscaper.cfg file change Flush=False to Flush=True

WARNING: This will delete all data in the Neo4j Database.

Docker Collector

Configuration

Docker must be installed and configured in swarm mode. If using TLS (currently default) client cert and key must be generated and path given to the landscaper.cfg The provided script configure_docker.tls.sh is for development purposes/verification and will generate keys for usage in the folder CA_SRV, read it and understand before executing. client_cert=<path_to>/client-cert.pem client_key=<path_to>/client-priv-key.pem Removing these entires from the config file will allow the collector to attempt to connect without TLS

About Landscaper

The landscaper was developed by Intel Labs Europe and is published under the Apache License, Version 2.0.

Contributions from the community are welcome via Pull Requests.