Today i will share my experience how to use S3 file input component in Pentaho. Before we start and discuss about the ETL code, you should have S3 Access key and S3 secret key handy with you in order to connect with Amazon S3 bucket. Secondly, If you want to test the below ETL code and you are not using S3 bucket for file storage, then you can create your own Amazon account(Free for one year with limited features and limited storage).
Below is the ETL code regarding “How to read the data from S3 bucket using S3 file input”.
Here , I have used S3 file input and write to log component inside the transformation. Below are the values which we need to fill in S3 file input.
- S3 Access key
- S3 Secret Key
- S3 bucket(Click on “Select Bucket” to select the S3 bucket).
- File Name
See the below SS for the same.
Here, i used the Bucket S3-techie and file “Mar-SAL.csv”(This file i used in my last post as well).
Click on Get fields to get all the fields of the file. See the below image for the same.
When you execute the ETL code, you will see the data written on console logging section. See the below image for the same.
