Skip to content
Pentaho Data Integration ( ETL ) a.k.a Kettle
Java JavaScript CSS HTML Shell Batchfile
Branch: master
Clone or download

Latest commit

pdesai16 [BACKLOG-34028] Pdi-core is not sending any ExtensionPoint events if … (
#7479)

* [BACKLOG-34028] Pdi-core is not sending any ExtensionPoint events if there are any failure during transformation preparation step
Trans.java is throwing a KettleException when there is any error in the transformation
preparation step but it is not sending any Kettle ExtensionPointEvents
to indicate that the transformation had finished with errors. This makes
it impossible for the plugins that are relying on the
ExtensionPointEvents to track the status of the execution.

Added code in PanCommandExecutor to send TransformationFinish event in
case there is any error in transformation Preparation

* Adding unit test for the change
Latest commit dfa2e99 May 29, 2020

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
assemblies [BISERVER-3497] Encr command line fix and dependency preparation May 20, 2020
core [BISERVER-3497] Encr command line fix and dependency preparation May 20, 2020
dbdialog [PDI-18603] Added CTRL-SPACE functionality to Impala and Hive 2 conne… Feb 3, 2020
engine-ext [CLEANUP] updated versions via release version merger Jan 14, 2020
engine [BACKLOG-34028] Pdi-core is not sending any ExtensionPoint events if … ( May 29, 2020
integration [PPP-4495] Use of Vulnerable Component: Components/legion-of-the-boun… May 7, 2020
plugins [BISERVER-3497] Encr command line fix and dependency preparation May 20, 2020
ui [BACKLOG-33865] Update prefetch label on streaming steps May 22, 2020
.gitignore
Carte-jmeter.jmx BACKLOG-2249 As an OEM/Embedder, I would like PDI Carte web services … Mar 3, 2015
LICENSE.txt PDI-7147 : Change Kettle license to Apache Jan 16, 2012
README.md [BACKLOG-26768] Task to test devoptics Dec 7, 2018
Translator.bat Changes requestet in review. Aug 23, 2019
pom.xml [PPP-4486] Use of Vulnerable Component: commons-codec [Multiple Versi… Feb 17, 2020
translator.sh [BACKLOG-27160] - 8.3 - Update Copyright Information for Applications… May 8, 2019
translator.xml [BACKLOG-30801] - Kettle5-log4j plugin is no longer used Sep 17, 2019

README.md

Pentaho Data Integration

Pentaho Data Integration ( ETL ) a.k.a Kettle

Project Structure

  • assemblies: Project distribution archive is produced under this module
  • core: Core implementation
  • dbdialog: Database dialog
  • ui: User interface
  • engine: PDI engine
  • engine-ext: PDI engine extensions
  • plugins: PDI core plugins
  • integration: Integration tests

How to build

Pentaho Data Integration uses the maven framework.

Pre-requisites for building the project:

  • Maven, version 3+
  • Java JDK 1.8
  • This settings.xml in your /.m2 directory

Building it

This is a maven project, and to build it use the following command

$ mvn clean install

Optionally you can specify -Drelease to trigger obfuscation and/or uglification (as needed)

Optionally you can specify -Dmaven.test.skip=true to skip the tests (even though you shouldn't as you know)

The build result will be a Pentaho package located in target.

Running the tests

Unit tests

This will run all unit tests in the project (and sub-modules). To run integration tests as well, see Integration Tests below.

$ mvn test

If you want to remote debug a single java unit test (default port is 5005):

$ cd core
$ mvn test -Dtest=<<YourTest>> -Dmaven.surefire.debug

Integration tests

In addition to the unit tests, there are integration tests that test cross-module operation. This will run the integration tests.

$ mvn verify -DrunITs

To run a single integration test:

$ mvn verify -DrunITs -Dit.test=<<YourIT>>

To run a single integration test in debug mode (for remote debugging in an IDE) on the default port of 5005:

$ mvn verify -DrunITs -Dit.test=<<YourIT>> -Dmaven.failsafe.debug

To skip test

$ mvn clean install -DskipTests

To get log as text file

$ mvn clean install test >log.txt

IntelliJ

  • Don't use IntelliJ's built-in maven. Make it use the same one you use from the commandline.
    • Project Preferences -> Build, Execution, Deployment -> Build Tools -> Maven ==> Maven home directory

Contributing

  1. Submit a pull request, referencing the relevant Jira case
  2. Attach a Git patch file to the relevant Jira case

Use of the Pentaho checkstyle format (via mvn checkstyle:check and reviewing the report) and developing working Unit Tests helps to ensure that pull requests for bugs and improvements are processed quickly.

When writing unit tests, you have at your disposal a couple of ClassRules that can be used to maintain a healthy test environment. Use RestorePDIEnvironment and RestorePDIEngineEnvironment for core and engine tests respectively.

pex.:

public class MyTest {
  @ClassRule public static RestorePDIEnvironment env = new RestorePDIEnvironment();
  #setUp()...
  @Test public void testSomething() { 
    assertTrue( myMethod() ); 
  }
}

Asking for help

Please go to https://community.hitachivantara.com/community/products-and-solutions/pentaho/ to ask questions and get help.

You can’t perform that action at this time.