create table dynamically based on Metadata file and load the source file into that table in pentaho

admin

Today , i will discuss about creating the table based on metadata file and then loading the source file into that table.Below is the image of ETL job.

First, loading the metadata file into a table, means loading the metadata file which has details like column name ,column type and column length.

In this ktr (loading_metadata_file), loading the metadata file to a table TBL_METADATA_STORE.


Below is the DDL of this table.I am using mysql as database.

CREATE TABLE `tbl_metadata_store` (
`column_name` varchar(100) DEFAULT NULL,
`data_type` varchar(100) DEFAULT NULL,
`col_len` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Once the metadata is loaded into table, now i will create the dynamic query using the data which is present in the TBL_METADATA_STORE and eventually when i execute the dynamic query, table gets created as per the metadata defined.

Name of the ktr is call-dynamic proc. Below is the image of this code.

You can alter the procedure(used to create the table dynamically) as per your metadata file and DB used.
Now, simplest task is left which is loading the source file to that table.Below is the image for the same.

ETL code, procedure, DDL , sample metadata file and source file checked in to git. you can use this code as reference for your use-case development.

https://github.com/Karan-Arora-13/technicalstuff/

ETL_CODE_PENTAHO_DYNAMIC_TABLE_CREATION.zip

 

Related posts

6 Thoughts to “create table dynamically based on Metadata file and load the source file into that table in pentaho”

  1. Dedneips
    at

    What a lovely story! The tale in this YouTube video that is posted here is in fact a fastidious one with having nice picture quality.
    [url=http://noreferer.win/]Anonymous links[/url]

  2. Buistino
    at

    Es ist nichts zu sagen – schweigen Sie still, damit, das Thema nicht zu verunreinigen.
    http://losslan75.full-design.com/
    http://www.versisforum.com/viewtopic.php?f=3&t=319543

    http://btstore.top/

    http://drstore.top/

  3. Buistino
    at

    Les mots sont plus grands!
    http://btstore.top/
    http://drstore.top/
    http://randu.xyz/

    http://www.dreamatrix.net/forum/viewtopic.php?f=23&t=460230

    https://twitter.com/choisehelper

  4. Buistino
    at

    Felicito, este pensamiento magnГ­fico tiene que justamente a propГіsito
    http://tiedrink88.blogminds.com/free2play-first-person-shooter-rexuiz-fps-4155623

    http://turkeypocket89.blogdon.net/rexuiz-4847499
    http://www.blackplanet.com/your_page/blog/index.html?profile_id=89640335&profile_name=locketwind76&user_id=89640335&username=locketwind76

    http://bbs.he.99.com/showthread.php?p=534498#post534498

  5. Buistino
    at

    Credo che lei abbia sbagliato. Sono sicuro. Sono in grado di provarlo. Scrivere a me in PM, parlare.
    http://noreferer.win/

    http://mishushinavoq.tumblr.com/
    http://roberttierrez.blogspot.com/
    https://twitter.com/top_choise

    http://randu.xyz/

  6. Buistino
    at

    Entre nosotros hablando, prueben buscar la respuesta a su pregunta en google.com
    http://360square.com/forum/viewtopic.php?p=11102#11102
    http://forum.glyks.com/viewtopic.php?f=5&t=138318
    http://www.studynexus.net/forums/viewtopic.php?f=7&t=278573
    http://difmuhendislik.com/difforum/index.php?topic=80489.new#new
    http://russian-anonymous-forum.biz/showthread.php?tid=5346

    http://forum.plan-a-academy.com/index.php/topic,133478.new.html#new
    http://abagreatbook.com/viewtopic.php?f=4&t=91765
    http://www.vwloc.com/forum/viewtopic.php?f=21&t=9503
    http://makeverythingreatagain.com/viewtopic.php?f=32&t=56310

    http://forum.glyks.com/viewtopic.php?f=6&t=137198
    http://difmuhendislik.com/difforum/index.php?topic=41830.new#new
    http://test.if-forum.com/viewtopic.php?f=13&t=29817
    http://www.cz58.com/forum.php?mod=viewthread&tid=1000226&extra=

Comments are closed.