Partitioning data on clustered Pentaho Kettle ETL transformations

This is the second article on clustering ETL transformations with Pentaho Kettle (Pentaho Data Integration). It is highly recommended that you read the first article Creating a clustered transformation in Pentaho Kettle before continuing with this one. Make sure that the slave and master servers are running and the cluster schema is defined - as outlined in the first article.Prerequisites:Current version of PDI installed.Download the sample transformations...

Read More

Creating a clustered transformation in Pentaho Kettle

Prerequisites:Current version of PDI installed.Download the sample transformations from here.Navigate to the PDI root directory. Let’s start three local carte instances for testing (Make sure these ports are not in use beforehand):sh carte.sh localhost 8077sh carte.sh localhost 8078sh carte.sh localhost 8079In PDI Spoon create a new transformation.Click on the View tab on the left hand side and right click on Slave server and choose New. Add the...

Read More

Pentaho Kettle (PDI): Get Pan and Kitchen Exit Code

Various monitoring applications require the exit code/status of a process as an input. A simple example (test1.sh):#!/bin/bashecho "Hi"exit $?Let’s run it:$ ./test1.shLet’s check the exit status (of the last command) which can be accessed via $?:$ echo $?0Let’s take a look at how we can get the exit status from Pan and Kitchen:For demonstration purposes we create a very simple dummy transformation which just outputs some data to the log: Now create...

Read More