Skip to content
Pentaho Data Integration ( ETL ) a.k.a Kettle
Java JavaScript CSS HTML Shell Batchfile
Branch: master
Clone or download

Latest commit

gkuntsevich Merge pull request #7488 from pentaho/PDI-18625
[PDI-18625] Fixed issue with creating report using streaming HTML output
Latest commit cb21276 Jun 3, 2020

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
assemblies [BISERVER-3497] Encr command line fix and dependency preparation May 20, 2020
core {BACKLOG-33987] Add decryption support to simple-jndi Jun 2, 2020
dbdialog [PDI-18603] Added CTRL-SPACE functionality to Impala and Hive 2 conne… Feb 3, 2020
engine-ext [CLEANUP] updated versions via release version merger Jan 14, 2020
engine Merge pull request #7488 from pentaho/PDI-18625 Jun 3, 2020
integration [PPP-4495] Use of Vulnerable Component: Components/legion-of-the-boun… May 7, 2020
plugins [BISERVER-3497] Encr command line fix and dependency preparation May 20, 2020
ui Merge pull request #7477 from tkafalas/feature Jun 3, 2020
.gitignore BACKLOG-20096 - Fixed error with JavaScript failing when run in AEL /… Dec 4, 2017
Carte-jmeter.jmx BACKLOG-2249 As an OEM/Embedder, I would like PDI Carte web services … Mar 3, 2015
LICENSE.txt PDI-7147 : Change Kettle license to Apache Jan 16, 2012
README.md [BACKLOG-26768] Task to test devoptics Dec 7, 2018
Translator.bat Changes requestet in review. Aug 23, 2019
pom.xml [PPP-4486] Use of Vulnerable Component: commons-codec [Multiple Versi… Feb 17, 2020
translator.sh [BACKLOG-27160] - 8.3 - Update Copyright Information for Applications… May 8, 2019
translator.xml [BACKLOG-30801] - Kettle5-log4j plugin is no longer used Sep 17, 2019

README.md

Pentaho Data Integration

Pentaho Data Integration ( ETL ) a.k.a Kettle

Project Structure

  • assemblies: Project distribution archive is produced under this module
  • core: Core implementation
  • dbdialog: Database dialog
  • ui: User interface
  • engine: PDI engine
  • engine-ext: PDI engine extensions
  • plugins: PDI core plugins
  • integration: Integration tests

How to build

Pentaho Data Integration uses the maven framework.

Pre-requisites for building the project:

  • Maven, version 3+
  • Java JDK 1.8
  • This settings.xml in your /.m2 directory

Building it

This is a maven project, and to build it use the following command

$ mvn clean install

Optionally you can specify -Drelease to trigger obfuscation and/or uglification (as needed)

Optionally you can specify -Dmaven.test.skip=true to skip the tests (even though you shouldn't as you know)

The build result will be a Pentaho package located in target.

Running the tests

Unit tests

This will run all unit tests in the project (and sub-modules). To run integration tests as well, see Integration Tests below.

$ mvn test

If you want to remote debug a single java unit test (default port is 5005):

$ cd core
$ mvn test -Dtest=<<YourTest>> -Dmaven.surefire.debug

Integration tests

In addition to the unit tests, there are integration tests that test cross-module operation. This will run the integration tests.

$ mvn verify -DrunITs

To run a single integration test:

$ mvn verify -DrunITs -Dit.test=<<YourIT>>

To run a single integration test in debug mode (for remote debugging in an IDE) on the default port of 5005:

$ mvn verify -DrunITs -Dit.test=<<YourIT>> -Dmaven.failsafe.debug

To skip test

$ mvn clean install -DskipTests

To get log as text file

$ mvn clean install test >log.txt

IntelliJ

  • Don't use IntelliJ's built-in maven. Make it use the same one you use from the commandline.
    • Project Preferences -> Build, Execution, Deployment -> Build Tools -> Maven ==> Maven home directory

Contributing

  1. Submit a pull request, referencing the relevant Jira case
  2. Attach a Git patch file to the relevant Jira case

Use of the Pentaho checkstyle format (via mvn checkstyle:check and reviewing the report) and developing working Unit Tests helps to ensure that pull requests for bugs and improvements are processed quickly.

When writing unit tests, you have at your disposal a couple of ClassRules that can be used to maintain a healthy test environment. Use RestorePDIEnvironment and RestorePDIEngineEnvironment for core and engine tests respectively.

pex.:

public class MyTest {
  @ClassRule public static RestorePDIEnvironment env = new RestorePDIEnvironment();
  #setUp()...
  @Test public void testSomething() { 
    assertTrue( myMethod() ); 
  }
}

Asking for help

Please go to https://community.hitachivantara.com/community/products-and-solutions/pentaho/ to ask questions and get help.

You can’t perform that action at this time.