Skip to content

Commit

Permalink
Added more config file documentation.
Browse files Browse the repository at this point in the history
  • Loading branch information
Alex Meadows committed Jul 24, 2014
1 parent a4a0d06 commit 1a9ab15
Show file tree
Hide file tree
Showing 3 changed files with 69 additions and 0 deletions.
32 changes: 32 additions & 0 deletions docs/source/settings/connections.rst
Original file line number Diff line number Diff line change
@@ -1,2 +1,34 @@
Data Connections
================
This section describes the setting file used for all data source and target connectivity.

Default Settings
----------------

Here is the default sample of the ``connections.cfg`` file. This file can be found in the application settings directory, as described in :doc:`Settings <./settings>`

::

[etlUnitTest]
hostname: 127.0.0.1
username: root
password:
port: 3306
type: mysql
dbname: etlUnitTest


While this sample is written for MySQL/MariaDB, you can connect to any data source supported by SQLAlchemy. The full
list of SQLAlchemy supported data sources can be found in their official `documentation <http://docs.sqlalchemy
.org/en/rel_0_9/core/engines.html#database_urls>`_ .

* ``[etlUnitTest]`` - The distinct name of the data source/target. Can be any valid string,
as long as it does not break configuration file standards.
* ``hostname:`` - The host name or the ip address of the system that hosts the data source/target.
* ``username:`` - The username that will be used to connect to the data source/target.
* ``password:`` - The password for the user account connecting to the data source/target.
* ``port:`` - The port on the host that allows for connections to the data source/target.
* ``type:`` - The type of data source/target being connected to. Must be compliant with the types of SQLAlchemy
dialects.
* ``dbname:`` - The name of the schema/database being connected to. Does not have to match the name used to define
the data source/target.
10 changes: 10 additions & 0 deletions docs/source/settings/data_integration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ Here is the default sample for ``tools.yml``. This file can be found in the app
::

PDI:
host_name: localhost
port:
user_name:
password:
private_key: '~/.ssh/id_rsa'
tool_path: ${TOOL_PATH}
code_path: ${ETL_TEST_ROOT}/etltest/samples/etl/
process_param: "/file:"
Expand All @@ -22,6 +27,11 @@ Here is the default sample for ``tools.yml``. This file can be found in the app
While the sample is written for Pentaho Data Integration, it can be configured for any data integration tool that can be run from the command line.

* ``PDI:`` - The unique name of the tool. Can be any string as long as it does not break YAML standards.
* ``host_name:`` - The unique name or ip address of the host the tool lives on.
* ``port:`` - The port used to ssh onto the host box.
* ``user_name:`` - The name of the user account that is used to run data integration code.
* ``password:`` - The password of the user account that is used to run data integration code.
* ``private_key:`` - The private key to tunnel onto the box, if needed.
* ``tool_path:`` - The install location of the data integration tool.
* ``code_path:`` - The location of the data integration tool's code base. This is where etlTest will look for executable code.
* ``process_param:`` - Any custom parameters that have to be used to call the code. In PDI's case, files are called with the ``/file:`` property.
Expand Down
27 changes: 27 additions & 0 deletions docs/source/settings/properties.rst
Original file line number Diff line number Diff line change
@@ -1,2 +1,29 @@
etlTest User Properties Settings
================================
This section describes the settings file used for the user's environment settings.

Default Settings
----------------
Here is the default sample for ``properties.cfg``. This file can be found in the application settings directory,
as described in :doc:`Settings <./settings>`
::

[Locations]
tests: ${ETL_TEST_ROOT}/Documents/etlTest/tests
data: ${ETL_TEST_ROOT}/Documents/etlTest/data
output: ${ETL_TEST_ROOT}/Documents/etlTest/output
[Results]
Verbose: True
FailureRate: 10
ReportType: Normal

While the sample is written with example paths, any valid directory path can be used. If the directory does not
exist, it will be created.

* ``[Locations]`` - This section of the properties configuration file contains locations for the various inputs and
outputs of etlTest.
* ``tests:`` - The location where the YAML test files are stored.
* ``data:`` - The location where the YAML data files are stored.
* ``output:`` - The location where the generated test scripts are created.
* ``[Results]`` - This section is currently not in use. The intent is to create user/environment based parameters
for how tests are run and the results shown. The parameters underneath are just examples and are ignored by etlTest.

0 comments on commit 1a9ab15

Please sign in to comment.