Talend community forum / Merging two flows of datas using the same schema in an iterative flow

Diane2 · 2009-04-16 09:29:56

Hi,

Is is possible to merge two data flows that have the same schema and are on the same iterative branch of a job, like in the following case

____ tSampleRow (Table1, Schema 1)
/
tOracleInput ----> tMap -< Merging possible?
\____ tSampleRow (Table2, Schema 1)

Thanks in advance,
Diane2

nSmolyniec · 2009-04-16 10:05:45

Hi,

From what I know, merging is possible with "tUnite" component, but you won't be able to merge your 2 tSampleRows outputs because you cannot perform any loop. Maybe use a temp file.
Anyway, why merging ? You can't find another way with you tMap ?

Regards,

Nicolas

shong · 2009-04-16 10:24:33

Hello Diane2

Talend don't allow a cycle flow in a job, see [Forum, topic 1468] multiple tAgregateRow and tMap components.

Best regards

shong

Diane2 · 2009-04-16 15:33:18

Hi, thanks for your answer, I read carefully the topic you linked, which explain why it isn't possible. I also floowed a link in this thread pointing to another topic expalining how to bypass this restriction in two distincts manners by using fileDelimited.
I tested them and they fullfil my wishes but I can't help noticing that in my case this solution is less efficient than duplicating whole subjob (meaning that reloading all data twice and making same tMap twice). This is mainly because of poor performance in insertion from a DelimitedFile.

Does Somebody know anyway to improve speed in this kind of operation? What about a similar solution using tOutputBulk and its familly? I couldn't find a solution using these components by myself.

Cheers,
Diane2

shong · 2009-04-17 09:57:23

Hello

(meaning that reloading all data twice and making same tMap twice)

Can you upload a screenshot of your current job?

Does Somebody know anyway to improve speed in this kind of operation? What about a similar solution using tOutputBulk and its familly?

1)Yes, the txxxOutputBulk component will improve performance than txxxOuput component. You can read the user documentation and learn its usage.
2)Try to add filter conditon in your sql statement if you can.

Best regards

shong

Announcement

Unanswered posts

#1 2009-04-16 09:29:56

Merging two flows of datas using the same schema in an iterative flow

#2 2009-04-16 10:05:45

Re: Merging two flows of datas using the same schema in an iterative flow

#3 2009-04-16 10:24:33

Re: Merging two flows of datas using the same schema in an iterative flow

#4 2009-04-16 15:33:18

Re: Merging two flows of datas using the same schema in an iterative flow

#5 2009-04-17 09:57:23

Re: Merging two flows of datas using the same schema in an iterative flow

Board footer