Talend community forum / Integrate data from many csv files

mandy1211 · 2010-03-11 10:28:09

Hi!

I have Campaign A, Campaign B and Campaign C. Each campaign has a set of statistics exported in csv files.
Campaign A: A1,A2,A3 (A1,A2,A3 are names of csv files). I need to integrate data in these 3 files and store in a table X in Mysql DB. After create a job A on JasperETL to do this, table X will have 1 row which store integrated data about campaign A.

Campaign B also has B1,B2,B3 as names of csv files. I need to integrate data in these 3 files and store in 2nd row of table X.

The procedure to integrate campaign B is exactly the same as integrating campaign A.

I am thinking to create a new job B and copy all the components in job A to B. However, doing this, if I have 5 campaigns, I will have to create 5 similar jobs. If I have n campaigns, I will have to create n similar jobs.

How would you suggest me to do with this? Is there any way to automate the process?

Thanks a million!!

Mandy

mgainhao · 2010-03-11 14:07:48

Hi,
your schema from your Campaign A is the same of campaign B or not?

Maybe if it is the same schema you can try with contexts.

mandy1211 · 2010-03-16 04:15:52

Hi,

My campaign A and campaign B have exactly the same schema.
Currently, I created a job for campaign A. I imported 3 csv files which store data about campaign A. I integrated these 3 files and export to a table X in my database.
Now I would like to integrate 3 files about campaign B and insert the data into table X. The process of integration is the same, the schema is the same.

Can you explain more how can I use context to solve my issue?
And...Is there anyway that TOS can pull data from csv files in a folder located locally in my pc and import to metadata?

Thanks for your time!

Mandy

Announcement

Unanswered posts

#1 2010-03-11 10:28:09

Integrate data from many csv files

#2 2010-03-11 14:07:48

Re: Integrate data from many csv files

#3 2010-03-16 04:15:52

Re: Integrate data from many csv files

Board footer