Today, I will discuss about how to add files in the mail component present in the job section. In order to do this , you have to use the “Add filenames to result” component. See the below SS for the same.Here, you need to perform two steps.1. Use wildcard for text files in the “Add filenames” component in case you need to attach text files. select wildcard according to the need of your project. See the below SS for the same. 2. Do the below Settings in the mail component.…
Category: Pentaho
cassandra input component in Pentaho
Today, I will discuss about “How to use cassandra input component in Pentaho”. For this, the first and foremost criteria is that cassandra database should be downloaded and installed in your local Machine.Once installation is completed, you can start the Apache-cassandra services using below command.cassandra.bat -f ( This batch file is present inside bin folder).In order to create a sample table in the Cassandra database, you need to open another command prompt session where you need to run the cqlsh command(Run this command inside bin folder) which helps to run…
call stored procedure using Table Input component in Pentaho
Today, I will discuss about the scenario which might comes in someone’s development phase wherein we need to call procedure or function using Table input component. Here, I have created the sample function and procedure using sql server Database. See the code snippets. See the Data which is present in the TBL_DIM_CUSTOMER Now, we will call function using select statement inside Table input component. See the below SS. Here , I have passed customer_id value as 5. Click on Preview, you will get required result. In the similar Fashion, we…
Dimension Lookup in Pentaho
Today, I will discuss about the Dimension Lookup in Pentaho. I have seen various blogs and forums where users share their experience about the issues encountered during the implementation of this component. One of the major issue is null values in the Dimension tables. There are two scenarios. 1. When you use this component with “Use Auto Incremental Key” for Technical Field, then it inserts a null values on every run which is a concern .2. When you use this component using “Use table Maximum +1” for Technical Fields, then…
How to pass parameter in carte
Today, I will discuss about “Call Job using Carte with parameters”. I have already explained How to setup carte in windows machine and How to execute job or Transformation using Carte. https://www.allabouttechnologies.co.in/pentaho/run-transformation-as-web-service-using-carte-in-pentaho/ Now, we will extend this blog by passing the parameter to a job. Here I have a job where I have set parameter “file_name”. See the Below images for the same. This file_name parameter is used inside the transformation which can see below. Now, we will run the Job using carte . In order to do this, first…
Avro Output in Pentaho
Today, I will discuss about Avro Output Component in Pentaho. In my previous blog, I have share my experience about Avro input component where Data Deserialization happens. In this Component, Data Serialization Happens. So, if you have data in a text format, you can convert the same in Avro format as well. As soon as you do this conversion, a Schema file also get generated along the Avro file.This all can be achieved through Avro Output Component in Pentaho I have designed a very simple Transformation wherein we have csv…
Avro File Input In Pentaho
Today, I will discussing about the Avro input component in Pentaho. Avro uses the concept of serialization and De-serialization. Serialization means processing the data into binary format. Its very clear that if we have data in binary format ,its unreadable and hence very effective way to transfer over the network. Therefore, many Organization are adopting this technique due to data security concerns. Deserialization means convert the binary formatted data into a readable form. Now the question comes how binary data is deserialized. Here , comes the concept of Schema file.…
INTERVIEW QUESTIONS IN PENTAHO SET-4
1.How to create ETL code which will behave same as case statement in Database. Name the component which you will use for the same. 2.Can we run the ETL code until the condition met or you can say infinite loop like while statement saying 1==1 in shell script. 3.How to get number of fields in csv file in Pentaho without using shell Script. 4.Have you ever used Sample rows component. If yes, where you actually need this component. 5.What is Apache kafka . How you will setup the same in…
Check Number of fields in csv file in Pentaho
Today, I will discuss about the “How to check Number of fields in csv file or any other file in Pentaho without using shell Scripting. Below is the code snippet for the same. Here, we have used components like Text file input, Sample rows, Split fields to rows,Write to log. In Text File input, we have intentionally kept wrong delimiter, so that we will get only one field in the fields section. Please see how data looks like. Now, comes the Sample rows,Using this component, we will fetch only one…
Wildcard In pentaho
Today, I will discuss about the “Wildcards in Pentaho”. I have tried to cover all possible Scenarios. Use the below wildcard for below mentioned Scenarios. 1. Get all files present inside a particular Directory. .+ 2. Get all CSV files. .+\.csv 3. Get all CSV files having special character like UnderScore (_) .+._.+\.csv 4. Get all CSV files starting with Particular Pattern . For Example abc*****.csv abc.+\.csv 5. Get all CSV files having pattern in between.For example ***abc***.csv .+.abc.+\.csv 6. Get all CSV files which are ending with Numbers. .+[0-9]\.csv…